Slop Eval

Slop Eval

Evaluate a design the way skill-evaluation evaluates a skill: every finding cites concrete evidence, every axis gets a 0–100 score, arithmetic runs through a script, and the output is a structured report — never a vibe check.

The tell catalog lives in references/tells.md; read it before sweeping. The positive rubric (signature formula, cohesion checks, slop→premium pairs, and the "Adding Soul" guide) lives in references/premium-markers.md; read it before scoring Axes 7–8 and when writing fix prescriptions. The design context guide lives in references/contexts.md; read it to adjust priorities and tolerances based on the type of design being evaluated.

Source

• The pols.dev anti-slop design law — the tell catalog, absolute rules, and signature formula are distilled from it.
• Method modeled on skill-evaluation (cite-or-cut, weighted axes, scripted scoring, failure-mode diagnosis).

Parameters

Write the report in the language the user is speaking; keep tell IDs and names in English so they stay greppable against the catalog.

Evaluation modes

Standard mode (default)

Single evaluation of a design. Produces a Slop Report with scores, tells, section ledger, and prioritized fixes.

Comparison mode

Side-by-side evaluation of two different designs (e.g., competitor analysis, A/B variants). Add --compare pointing to another target or existing report.

Tracking mode

Evaluate the same design over time to measure improvement. Use when:

• Running weekly/sprint design reviews
• Measuring progress after a redesign
• Validating that fixes actually moved the score

Usage:

# First evaluation — establishes baseline
slop-eval --target https://site.com --output ./reports/baseline.md

# Later evaluation — tracks evolution
slop-eval --target https://site.com --output ./reports/week-2.md \
  --compare ./reports/baseline.md

Tracking mode adds to the report:

• Score progression table with trends (✅ improved / ⚠️ regressed)
• Tells resolved (what got fixed)
• New tells (what got introduced)
• Regressions (axes/sections that got worse)
• Section ledger evolution
• Velocity metrics (tells resolved per week, score improvement rate)
• Recommendations for next iteration

See references/output-template.md for the full tracking output format.

Evidence channels

What you can verify depends on what you were given. Never score a check you could not observe — mark it Unverifiable and exclude it (like N/A in skill-evaluation).

With code, grep before you stare: fonts.googleapis|next/font, lucide-react, linear-gradient, box-shadow, border-radius: *9999, backdrop-filter, opacity: *0, initial={{ *opacity: *0, overflow: *hidden, clip-path, position: *fixed. Each hit is a lead, not a verdict — confirm against the catalog entry before recording it.

Evidence acquisition SOP

Route by what the target is; always end with an evidence inventory (what was captured, what is Unverifiable) — it feeds the report header.

Live URL — the richest channel; prefer it whenever reachable. Use whatever browser automation this session has (a browser MCP such as Playwright or Chrome DevTools, or npx playwright screenshot as the no-MCP fallback) and capture, saving every artifact to the scratchpad so findings can cite file + region:

1. Load at desktop (1440×900) and mobile (390×844); wait for network idle.
2. Full-page screenshot of both viewports immediately after load, before any scrolling — sections sitting at opacity:0 waiting for a scroll reveal show up blank here (M1 evidence).
3. Scroll pass top to bottom, then a second full-page capture; diff the two mentally for reveal-gated content, seams (C11, X13), and fold ownership (L16).
4. Interaction pass: hover the primary CTA, one card, one nav link (M2–M4); click every tab, accordion, toggle, and button (M8); Tab through the page and confirm a visible focus ring (X14).
5. Zoom crops at 2x of: anything near a clipped edge (X2), circled/tiled numbers and icons (X1), pricing columns side by side (X3), button labels (X5).
6. Pull the rendered sources for the code-channel greps: font names from the network panel or <link>/@font-face, computed hex values from the stylesheets.

No browser automation available → fetch the HTML/CSS (curl) and run the code channel on it, ask the user for full-page desktop + mobile prints, and mark every visual-only and interaction check Unverifiable until the prints arrive. Never score a visual check from raw HTML.

Screenshots — Read each image. If only partial crops were provided, ask for full-page desktop + mobile before sweeping (a hero-only print cannot support L11, L15, or the cohesion axis). All interaction checks (M1, M8, X14, hover tells) are Unverifiable.

Code path — run the greps, read every file they hit, plus the layout/ page components and global styles. If the project runs locally, start its dev server and continue under the Live URL SOP — code plus a live render is the only combination that can verify everything.

Figma export — treat as Screenshots for visual tells; additionally fonts, hex values, and spacing are exact from the file. Motion and interaction axes are Unverifiable (score NA for Axis 5 unless prototypes were shared).

Axes and weights

Axis 7 carries the heaviest weight on purpose: the law's deepest rule is that dodging the tell list is still slop — a page with zero tells and no signature is unfinished work wearing restraint as an alibi.

Scoring

Axes 1–6 (tell-counted). Count confirmed tells on the axis by severity, then: score = max(0, 100 − 30·critical − 15·major − 5·minor). Run scripts/score.py axis CRIT MAJOR MINOR — don't do it by hand. One tell, one count: a pattern repeated across sections is still one tell (note the repetition in the evidence; repetition may upgrade minor → major where the catalog says so).

Axis 7 (Signature). Score each of the 7 formula elements 0 (absent), 50 (attempted, weak), or 100 (strong) per the rubric in premium-markers.md; the axis is their mean.

Axis 8 (Cohesion). Same 0/50/100 on the 4 cohesion checks; mean.

Compounding rule. Three or more major layout tells on one page cap Axis 4 at 40 — a page assembled from known skeletons is slop no matter how clean each block is.

Gates (pass as --cap to the overall run):

• Signature gate: Axis 7 < 40 caps the overall at 59 (grade C max). No amount of clean spacing rescues a page with no signature.
• Absolute-rule gate: any confirmed critical tell caps the overall at 69 (no grade A with broken execution).

Overall & Slop Index.

overall    = sum(axis_score × weight) / sum(weight)   # capped by gates
Slop Index = 100 − overall

Run scripts/score.py overall 1:80:2 2:65:2 ... [--cap 59] [--cap 69]. Unverifiable axes score NA and drop out of both sums. --fail-below N exits non-zero for CI gating, e.g. gating a PR on its preview deploy:

# .github/workflows/slop-gate.yml (step excerpt)
- name: Slop gate
  run: |
    # run slop-eval against $PREVIEW_URL, export each axis score, then:
    python3 skills/slop-eval/scripts/score.py overall \
      1:$A1:2 2:$A2:2 3:$A3:1 4:$A4:2 5:$A5:1 6:$A6:2 7:$A7:3 8:$A8:2 \
      --fail-below 40

Grade scale

Absolute rules check

Six execution laws, each pass/fail/unverifiable, reported in their own table. Any fail is a critical tell (counts on its axis AND triggers the absolute-rule gate):

1. Content visible by default — nothing gated on an entrance animation (opacity:0 + reveal) (M1)
2. Clear the cut — no text/control sliced by clip, notch, overflow, or fixed height (X2, X11)
3. Parallel alignment — comparable columns share baselines; buttons anchored (X3)
4. Real centering — everything meant to be centered is, mathematically and optically (X1)
5. Legible contrast — every text clears its background by a real value gap (X5)
6. Controls work — every interactive-looking control responds (M8)

Workflow

1. Gather evidence — route the target through the Evidence acquisition SOP above. Done when the evidence inventory states what was captured and what is Unverifiable.
2. Read references/tells.md — the catalog you sweep against.
3. Sweep axes 1–6 — walk the catalog group by group. Cite-or-cut: a tell is only recorded with concrete evidence (hex value, font name, file:line, or screenshot region); no evidence, no tell. Check each candidate against its premium-pair note — the crafted version of a pattern is not the tell. Done when every catalog group has been swept and every recorded tell carries a citation.
4. Run the absolute rules check — all six, pass/fail/unverifiable with evidence.
5. Score Axes 7–8 — read references/premium-markers.md, score the 7 signature elements and 4 cohesion checks with one-line justifications each. Done when all 11 items carry a score and a justification.
6. Compute — score.py axis per tell-counted axis, then score.py overall with weights and any triggered --cap. Never hand-compute.
7. Write the report — read references/output-template.md and emit exactly that structure to output, ending with the 3–5 prioritized fixes that would move the score most (biggest weighted deltas first; a missing signature usually outranks any single tell).