Agent skill

File Run Issues

by adrianco in adrianco/retort

Aggregate a retort run's findings.jsonl into a machine-readable assessment.json summary with severity counts, penalty score, requirement coverage, and top findings.

Apache-2.0Auto-check passed

Install File Run Issues

skills CLI
$ npx skills add adrianco/retort --skill file-run-issues -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI
$ gh skill install adrianco/retort file-run-issues --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy
$ git clone --depth 1 https://github.com/adrianco/retort.git skills-src && mkdir -p .claude/skills && cp -r skills-src/skills/file-run-issues .claude/skills/file-run-issues && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Facts

Skill name
file-run-issues
GitHub stars
207
Token cost
~1.3k tokens
SKILL.md length
463 words
Files
4
Skills in repo
6
Repo updated
First seen
Licence
Apache-2.0

At a glance

Aggregate a retort run's findings.jsonl into a machine-readable assessment.json summary with severity counts, penalty score, requirement coverage, and top findings.

  • Works in 9 steps: Load findings → Filter by severity → Count severities → …
  • SKILL.md covers Overview, Parameters, Steps and Constraints Summary, plus 2 more sections
  • Runs Python and Shell scripts from its folder; calls jq

What it does

File Run Issues is an agent skill from adrianco/retort. Aggregate a retort run's findings.jsonl into a machine-readable assessment.json summary with severity counts, penalty score, requirement coverage, and top findings.

Its SKILL.md is about 1.3k tokens, which your agent loads only when the skill is triggered. The skill folder holds 3 other files (for example `file-run-issues.py`, `file-run-issues.sh` and `test_file_run_issues.py`).

The repository describes itself as: Platform Evolution Engine. Distill the best from the combinatorial mess. The licence is Apache-2.0.

Example prompts

  • “/file-run-issues”

Requirements

  • Python 3
  • A Bash shell

Workflow steps

9 steps, taken from the step headings in SKILL.md.

  1. Load findings
  2. Filter by severity
  3. Count severities
  4. Compute penalty_score
  5. Collect top findings
  6. Compute requirement_coverage
  7. Read model from stack.json
  8. Write assessment.json
  9. Emit a summary

What it can do on your machine

Read from SKILL.md and the folder at commit 1f75769. It shows what the files ask for, not the result of running them.

  • Tool permissions

    Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

    From allowed-tools in the SKILL.md frontmatter.

  • Runs code

    Ships script files (Python and Shell), which the agent can run.

    Shell commands in SKILL.md call:

    • jq

    From the folder's file list and the shell code blocks in SKILL.md.

  • Network

    No URLs in SKILL.md.

    From URLs in SKILL.md, links to its own repository left out.

  • Credentials

    Names no API keys, tokens, secrets or passwords.

    From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

File Run Issues loads about 1.3k tokens when it runs. Until then it costs about 45 tokens; SKILL.md has 463 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it
~45
When it runs · the whole SKILL.md, loaded when a task matches
~1.3k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check passed

The automated check found no risky patterns in SKILL.md.

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from adrianco/retort at commit 1f75769, republished under its Apache-2.0 licence (© adrianco). 463 words, ~1,259 tokens.

Download SKILL.mdSave it as .claude/skills/file-run-issues/SKILL.md (or your agent's skills folder). This skill also uses 3 other files; get the full folder from GitHub.
name
file-run-issues
description
Aggregate a retort run's findings.jsonl into a machine-readable assessment.json summary with severity counts, penalty score, requirement coverage, and top findings.
type
anthropic-skill
version
2.0

File Run Issues

Overview

evaluate-run produces a findings.jsonl per run — one JSON object per observation. This skill aggregates those findings into assessment.json, a compact machine-readable summary used by FindingsScorer and downstream reporting. It does not create beads issues or GitHub issues.

Parameters

  • run_dir (required): Same path evaluate-run used, e.g. experiment-1/runs/language=rust_model=opus_tooling=beads/rep2/
  • min_severity (optional, default: info): Skip findings below this severity when computing counts. Options: critical, high, medium, low, info
  • dry_run (optional, default: false): Print the assessment JSON without writing assessment.json

Steps

1. Load findings
bash
test -f {run_dir}/findings.jsonl || { echo "no findings.jsonl — run evaluate-run first"; exit 1; }

Read the file line-by-line. Each line is one finding with at minimum these fields:

json
{"id": "R3", "kind": "requirement_missing", "severity": "high", "title": "...", "evidence": "...", "suggestion": "..."}
2. Filter by severity

Drop findings whose severity is below min_severity. The severity ordering from highest to lowest is: critical, high, medium, low, info.

3. Count severities

Count how many findings fall into each severity bucket:

json
{"critical": 0, "high": 2, "medium": 5, "low": 3, "info": 1}
4. Compute penalty_score
start = 1.0
subtract: critical * 0.25 + high * 0.10 + medium * 0.03 + low * 0.01
clamp result to [0.0, 1.0]

A run with no findings scores 1.0. A run with one critical finding scores 0.75. A run with four critical findings scores 0.0 (clamped).

5. Collect top findings

Select the top 5 findings by severity (critical first, then high, medium, low, info). Within the same severity level, preserve the original order from findings.jsonl. Include all fields from the original finding object.

6. Compute requirement_coverage

Count findings with kind in requirement_missing or requirement_partial — these represent requirements the agent did not fully implement. Estimate total requirements from R<N> IDs present in findings plus any implemented ones (inferred from evaluation.md if available, otherwise estimate from the highest R-number seen).

requirement_coverage = implemented_count / total_requirements

If total requirements cannot be determined, set requirement_coverage to null.

7. Read model from stack.json
bash
cat {run_dir}/stack.json | jq -r '.model // .agent // "unknown"'

If stack.json is absent or has no model/agent field, use "unknown".

Show full SKILL.md (195 more words)Show less
8. Write assessment.json

Write atomically (via .tmp rename):

json
{
  "severity_counts": {"critical": 0, "high": 2, "medium": 5, "low": 3, "info": 1},
  "penalty_score": 0.67,
  "top_findings": [...],
  "requirement_coverage": 0.75,
  "model": "haiku",
  "evaluated_at": "2026-04-18T21:00:00Z"
}

Constraints:

  • You MUST write atomically — write to {run_dir}/assessment.json.tmp then rename to {run_dir}/assessment.json.
  • evaluated_at MUST be an ISO 8601 UTC timestamp.
  • penalty_score MUST be rounded to 4 decimal places.
  • requirement_coverage MAY be null if total requirements cannot be determined.
9. Emit a summary

Print a terminal-readable summary:

Assessment written to {run_dir}/assessment.json
  Severity counts: critical=0 high=2 medium=5 low=3 info=1
  Penalty score:   0.6700  (1.0 = clean, 0.0 = critical failures)
  Req coverage:    75.0%
  Model:           haiku
  Top finding:     [high] No pagination support on GET /books

If --dry-run was specified, print the JSON to stdout and skip the file write.

Constraints Summary

  • You MUST NOT create beads issues, GitHub issues, or any external tracker records.
  • You MUST write assessment.json atomically.
  • You MUST be safe to re-run repeatedly — re-running overwrites assessment.json with fresh aggregation.
  • You MUST respect --dry-run by only printing what would be written.
  • You MUST finish quickly — this is aggregation only, no LLM calls, no network calls.

Interaction with retort

  • FindingsScorer in src/retort/scoring/scorers/findings.py reads {run_dir}/assessment.json and returns penalty_score directly.
  • The retort CLI MAY invoke this skill automatically after evaluate-run completes.
  • If assessment.json is absent, FindingsScorer returns 0.5 (neutral) rather than failing.

Troubleshooting

findings.jsonl is empty

  • Write assessment.json with all-zero severity counts, penalty_score 1.0, empty top_findings.

stack.json is missing

  • Use model: "unknown". Do not abort.

assessment.json.tmp rename fails (permissions)

  • Fall back to direct write. Log a warning.

© adrianco, Apache-2.0. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

SKILL.md and 3 other files in skills/file-run-issues of adrianco/retort.

  • SKILL.md
  • file-run-issues.py
  • file-run-issues.sh
  • test_file_run_issues.py

Open the folder on GitHubat commit 1f75769

Compare with similar skills

File Run Issues next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

File Run Issues compared with similar skills
SkillStarsUsed inTokensAuto-checkLicenceRepo updated
File Run Issues this skilladrianco/retort207—~1.3kAutomated safety check: PassApache-2.0
Find Releaseflutter/flutter179k—~545Automated safety check: PassBSD-3-Clause
Ddd Aggregateruvnet/ruflo74k—~774Automated safety check: NotesMIT
Fd Findpenpot/penpot61k—~882Automated safety check: PassMPL-2.0
Find Matching Tenderssickn33/agentic-awesome-skills47k1 repos~1kAutomated safety check: PassApache-2.0
Finding Replay For IssuePostHog/posthog40k—~1.7kAutomated safety check: PassCustom licence

Similar skills

  • Find Release

    flutter/flutter

    A skill to find the lowest Dart and Flutter release containing a given commit.

    179k GitHub stars~545 tokensUpdated today
    MobileAuto-check passed
  • Ddd Aggregate

    ruvnet/ruflo

    Scaffold an aggregate root with entity, value objects, repository interface, domain events, and test stubs.

    74k GitHub stars~774 tokensUpdated today
    DevelopmentAuto-check: notes
  • Fd Find

    penpot/penpot

    A fast and user-friendly alternative to 'find' - simple syntax, smart defaults, respects gitignore.

    61k GitHub stars~882 tokensUpdated today
    Auto-check passed
  • Find Matching Tenders

    sickn33/agentic-awesome-skills

    Find open AU/NZ government tenders matching what a company does, ranked by fit with why and gap analysis.

    47k GitHub starsUsed in 1 repo~1k tokens
    Sales & SupportAuto-check passed
  • Official

    Finds the most informative session recording linked to an error tracking issue.

    40k GitHub stars~1.7k tokensUpdated today
    Auto-check passed
  • Find In Notion

    superset-sh/superset

    Find what the workspace already says before you answer or write — search Notion, tell the canonical page from the stale copy, and cite what you used.

    15k GitHub stars~763 tokensUpdated today
    Productivity & AutomationAuto-check passed

More from adrianco/retort

  • Compare Runs

    adrianco/retort

    Compare evaluated runs in a retort experiment along factor dimensions.

    207 GitHub stars~2.2k tokensUpdated today
    Auto-check passed
  • Diagnose Failed Run

    adrianco/retort

    Determine the TRUE cause of a failed retort run before attributing it.

    207 GitHub stars~1.7k tokensUpdated today
    Auto-check passed
  • Evaluate Run

    adrianco/retort

    Evaluate a single retort experiment run. An agent skill from adrianco/retort.

    207 GitHub stars~4.2k tokensUpdated today
    Auto-check passed
  • Run Summary

    adrianco/retort

    Summarize the architecture of code generated by a single retort run.

    207 GitHub stars~1.3k tokensUpdated today
    Auto-check passed
  • Update Optimal Blog

    adrianco/retort

    Refresh the data tables in optimal-blog.md from master.db. An agent skill from adrianco/retort.

    207 GitHub stars~1.4k tokensUpdated today
    Auto-check passed

Questions about File Run Issues

What does File Run Issues do?

Aggregate a retort run's findings.jsonl into a machine-readable assessment.json summary with severity counts, penalty score, requirement coverage, and top findings. File Run Issues is an agent skill from adrianco/retort.json summary with severity counts, penalty score, requirement coverage, and top findings.

How do I install File Run Issues in Claude Code?

Run `npx skills add adrianco/retort --skill file-run-issues -a claude-code`. Or copy the skill folder (skills/file-run-issues in adrianco/retort) into .claude/skills/file-run-issues in your project. Claude Code loads it when a task matches its description.

How do I install File Run Issues in Codex?

Run `npx skills add adrianco/retort --skill file-run-issues -a codex`. Or copy the skill folder (skills/file-run-issues in adrianco/retort) into .agents/skills/file-run-issues in your project. Codex loads it when a task matches its description.

Can I use File Run Issues in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add adrianco/retort --skill file-run-issues -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/file-run-issues, .gemini/skills/file-run-issues, .github/skills/file-run-issues and .opencode/skills/file-run-issues in your project.

What does File Run Issues need to run?

Going by SKILL.md and its folder, File Run Issues needs Python and a shell for the scripts in its folder and the command-line tools its instructions call (jq). Our summary lists: Python 3; A Bash shell.

Does File Run Issues access the network?

SKILL.md contains no URLs. Any network use would come from the scripts or tools the agent runs. This is read from the text; nothing was executed.

Is File Run Issues safe to install?

Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.

What licence does File Run Issues use?

File Run Issues is published under the Apache-2.0 licence (the repository's licence). It allows redistribution, so the full SKILL.md is shown on this page.

How many tokens does File Run Issues use?

About 1.3k tokens (SKILL.md is roughly 5k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full.

What are the alternatives to File Run Issues?

Skills that share tags, products or a category with File Run Issues: Find Release (flutter/flutter, 179k stars), Ddd Aggregate (ruvnet/ruflo, 74k stars), Fd Find (penpot/penpot, 61k stars) and Find Matching Tenders (sickn33/agentic-awesome-skills, 47k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains File Run Issues?

adrianco (a GitHub user) maintains it in adrianco/retort, which has 207 GitHub stars. The repository holds 6 skills in this directory. The repository was last updated on October 9, 2026.

Source: adrianco/retort on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.