New: 98,000+ skills indexed from GitHub repos with 100+ stars · every skill safety-checkedBrowse top skillsTop skills
Esc to closeBrowse all skills
Submit a skillBrowse skills
Submit a skillBrowse skills

Agent skill

Analyze Training Run Iris

by open-thoughts in open-thoughts/OpenThoughts-Agent

Detailed health check for a Levanter/executor TRAINING run on the marin Iris cluster (e.g.

Apache-2.0Auto-check passed

Install Analyze Training Run Iris

skills CLI
$ npx skills add open-thoughts/OpenThoughts-Agent --skill analyze-training-run-iris -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI
$ gh skill install open-thoughts/OpenThoughts-Agent analyze-training-run-iris --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy
$ git clone --depth 1 https://github.com/open-thoughts/OpenThoughts-Agent.git skills-src && mkdir -p .claude/skills && cp -r skills-src/.agents/skills/analyze-training-run-iris .claude/skills/analyze-training-run-iris && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Facts

Skill name
analyze-training-run-iris
GitHub stars
301
Token cost
~2k tokens
SKILL.md length
689 words
Files
1
Skills in repo
44
Repo updated
First seen
Licence
Apache-2.0

At a glance

Detailed health check for a Levanter/executor TRAINING run on the marin Iris cluster (e.g.

  • An executor coordinator (<run-coord) plus its nested <run-coord/checkpoints-<step-<hash training child
  • SKILL.md covers Source 1 — iris job summary:…, Source 2 — W&B per-step…, Source 3 — GCS checkpoint… and Don't mistake setup steps for…, plus 3 more sections
  • Calls gsutil and python; needs WANDB_API_KEY
  • Which the harbor analyzer (analyze-job-history-iris) does NOT cover (training has no harbor trial sidecars

What it does

Analyze Training Run Iris is an agent skill from open-thoughts/OpenThoughts-Agent. Detailed health check for a Levanter/executor TRAINING run on the marin Iris cluster (e.g. the delphi midtraining runs) — step progress vs target, loss/throughput, preemption + MAJOR step-gap detection, and checkpoint cadence. Use for an executor coordinator (<run-coord) plus its nested <run-coord/checkpoints-<step-<hash training child, which the harbor analyzer (analyze-job-history-iris) does NOT cover (training has no harbor trial sidecars, same as GPU-RL). Reads W&B per-step history + iris job summary + GCS…

Its SKILL.md is about 2k tokens, which your agent loads only when the skill is triggered. It is a single SKILL.md file with no bundled scripts.

The repository describes itself as: Data recipes and robust infrastructure for training AI agents. The licence is Apache-2.0.

When your agent uses it

  • An executor coordinator (<run-coord) plus its nested <run-coord/checkpoints-<step-<hash training child
  • Which the harbor analyzer (analyze-job-history-iris) does NOT cover (training has no harbor trial sidecars
  • Same as GPU-RL)

Example prompts

  • “/analyze-training-run-iris”

Requirements

  • Python 3
  • A credential in WANDB_API_KEY

What it can do on your machine

Read from SKILL.md and the folder at commit 3bd1917. It shows what the files ask for, not the result of running them.

  • Tool permissions

    Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

    From allowed-tools in the SKILL.md frontmatter.

  • Runs code

    Shell commands in SKILL.md call:

    • gsutil
    • python

    From the folder's file list and the shell code blocks in SKILL.md.

  • Network

    No URLs in SKILL.md.

    From URLs in SKILL.md, links to its own repository left out.

  • Credentials

    Names these keys or tokens, usually read from environment variables:

    • WANDB_API_KEY

    From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Analyze Training Run Iris loads about 2k tokens when it runs. Until then it costs about 148 tokens; SKILL.md has 689 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it
~148
When it runs · the whole SKILL.md, loaded when a task matches
~2k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check passed

The automated check found no risky patterns in SKILL.md.

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from open-thoughts/OpenThoughts-Agent at commit 3bd1917, republished under its Apache-2.0 licence (© open-thoughts). 689 words, ~2,027 tokens.

Download SKILL.mdSave it as .claude/skills/analyze-training-run-iris/SKILL.md (or your agent's skills folder).
name
analyze-training-run-iris
description
Detailed health check for a Levanter/executor TRAINING run on the marin Iris cluster (e.g. the delphi midtraining runs) — step progress vs target, loss/throughput, preemption + MAJOR step-gap detection, and checkpoint cadence. Use for an executor coordinator (`<run>-coord`) plus its nested `<run>-coord/checkpoints-<step>-<hash>` training child, which the harbor analyzer (analyze-job-history-iris) does NOT cover (training has no harbor trial sidecars, same as GPU-RL). Reads W&B per-step history + `iris job summary` + GCS checkpoints instead of harbor GCS output.

analyze-training-run-iris

📍 Iris orientation — read first. Before acting on anything in this skill, read the Iris tools catalog (.agents/ops/iris/ops.md) and the Iris ops directory (.agents/ops/iris/ — the CoreWeave GPU particulars in ops.md, the TPU marin particulars in ops.md). They carry the binding access/preamble/gotchas and the helper-script inventory the steps below rely on.

A Levanter training run launched through the marin executor surfaces as TWO Iris jobs:

  • a tiny CPU coordinator — /<user>/<run>-coord — the executor_main DAG-walker (it submits the training job and then blocks); and
  • its nested training child — /<user>/<run>-coord/checkpoints-<step>-<hash> — the multi-task v5p job where the actual training steps happen (e.g. 8 tasks for a v5p-64).

Health = the CHILD's step progress + the run's preemption/gap history. The harbor analyzer (analyze-job-history-iris) does NOT apply — a training job has no harbor trial sidecars (just like GPU-RL). Use the three sources below instead. W&B is primary for step/loss/throughput; iris job summary is primary for preemptions/liveness; GCS is primary for checkpoint cadence.

Source 1 — iris job summary: preemptions, liveness, per-task state (always available)

bash
IRIS=/Users/benjaminfeuer/Documents/marin/.venv/bin/iris
$IRIS --cluster=marin job summary <child_job_id>

Report preemptions=N failures=N, tasks running/completed (all N tasks should be running together — a v5p job is gang-scheduled), the longest task DURATION, and PEAK MEM. preemptions>0 is expected on a preemptible v5p — each one means iris restarted the slice and Levanter resumed from the last checkpoint (every preemption costs a wall-clock gap: re-place + reload weights + XLA recompile). failures>0, a shrinking task count, or a crash-restart loop is a red flag → read the child logs for the error.

Source 2 — W&B per-step history: step / loss / throughput + MAJOR GAP detection (primary)

The run logs to W&B project delphi-midtraining (entity nyu-dice-lab); the run name is the GCS output-path hash — the last path segment of gs://marin-us-east5/checkpoints/<run>-<hash> (e.g. delphi-1e23-p33m67-k0p20-lr0.67-b6607e → run delphi-1e23-p33m67-k0p20-lr0.67-b6607e). Per-step history is NOT mirrored by mum — query the W&B API directly (needs WANDB_API_KEY from $DC_AGENT_SECRET_ENV; use the otagent python which has wandb):

bash
source "${DC_AGENT_SECRET_ENV:?set DC_AGENT_SECRET_ENV to the secrets file first}"
/Users/benjaminfeuer/miniconda3/envs/otagent/bin/python - <<'PY'
import wandb
ENTITY, PROJECT, RUN = "nyu-dice-lab", "delphi-midtraining", "<run-hash>"   # <-- the ...-b6607e hash
api = wandb.Api()
r = api.run(f"{ENTITY}/{PROJECT}/{RUN}")
total = r.config.get("trainer", {}).get("num_train_steps") or r.config.get("num_train_steps")
h = r.history(keys=["_step", "_timestamp", "_runtime", "train/loss"], pandas=True)
if h is None or len(h) == 0:
    print("state:", r.state, "-> pre-first-step (still setup/HF-download/XLA-compile); no training step yet")
else:
    h = h.dropna(subset=["_step"]).sort_values("_step")
    cur = int(h["_step"].iloc[-1])
    ts = h["_timestamp"].to_numpy()
    deltas = [b - a for a, b in zip(ts[:-1], ts[1:])]
    med = sorted(deltas)[len(deltas)//2] if deltas else 0.0
    thr = max(300.0, 20.0 * med)                       # same MAJOR-GAP rule as compute_time.md
    gaps = [d for d in deltas if d > thr]
    toks = 0
    bs = (r.config.get("trainer", {}) or {}).get("train_batch_size")
    sl = r.config.get("train_seq_len") or (r.config.get("model", {}) or {}).get("max_seq_len")
    if bs and sl and med: toks = bs * sl / med
    loss = h["train/loss"].dropna()
    loss = float(loss.iloc[-1]) if len(loss) else None
    eta_h = ((total - cur) * med / 3600.0) if (total and med) else None
    print(f"state            : {r.state}")
    print(f"step             : {cur}/{total}  ({round(100*cur/total,1) if total else '?'}%)")
    print(f"train/loss       : {loss}")
    print(f"median step dt   : {round(med,2)} s   -> ~{round(toks):,} tok/s" if med else "median step dt: n/a")
    print(f"MAJOR gaps       : count={len(gaps)}  total={round(sum(gaps)/3600,2)} h  (threshold {round(thr)} s)")
    print(f"ETA to {total}   : ~{round(eta_h,1)} h of compute (excludes future preemption gaps)" if eta_h else "ETA: n/a")
PY
  • step cur/total = progress against the K-budget target (e.g. 29,945 for K=0.20). This is the headline "progress in steps" number.
  • MAJOR gaps = consecutive _timestamp deltas exceeding max(300 s, 20 × median step interval) — preemption / idle gaps (the metric the user wants: "note major gaps"). Cross-check count against iris job summary preemptions — they should be the same order (a gap with NO matching preemption is a silent stall worth flagging). Report gap count + total hours.
  • tok/s = batch × seq / median_dt. Compare across ticks; a sudden drop = contention or a bad slice.
  • Empty history = the run is still in setup / HF-weight download / first XLA compile — that is NOT a gap; report it as pre-first-step and move on.
Show full SKILL.md (275 more words)Show less

Source 3 — GCS checkpoint cadence: resume safety (always available)

bash
gsutil ls gs://marin-us-east5/checkpoints/<run>-<hash>/ | grep -E 'step-[0-9]+' | tail -5
gsutil ls -l gs://marin-us-east5/checkpoints/<run>-<hash>/step-<latest>/ 2>/dev/null | tail -2   # timestamp

Report the latest persisted step + its timestamp (the checkpointer saves on an interval — e.g. save_interval 10m, keep every 1500). The latest checkpoint lagging a bit behind the W&B step is fine (async save). But no step-* checkpoint long after training started is a red flag — under preemption the run would lose all un-checkpointed progress. Only .executor_info / .executor_status* present (no step-*) = still pre-first-checkpoint (early bring-up).

Don't mistake setup steps for training steps

iris job logs <child> early on shows [iris setup] step N/M lines — those are uv-sync SETUP steps, NOT training steps. Do NOT grep step N from the logs for progress. Use the W&B _step (Source 2) as the authoritative training-step counter; only fall back to Levanter's own in-log training-step line if W&B is unreachable.

The compact cron line (one per training run)

<run> state=running step=<cur>/<total> (X%) loss=<L> ~<T>tok/s preempts=<P> gaps=<G>/<H>h ckpt=step-<C> plus a one-line health read: past setup/compile? step rate sane vs the prior tick? loss finite and trending down (not NaN/spiking)? preemptions resuming cleanly (checkpoint advancing)? ETA to the K-budget target.

Running it / offloading to a subagent

The W&B pull is fast (seconds), unlike the harbor analyzer — you usually do NOT need a subagent. If a run is huge or you are sweeping several, the analyze-job-history-iris foreground-and-wait discipline still applies to any slow gsutil/log reads, but the W&B query itself is quick.

  • monitor-cron-sweep-iris — the every-3-hours sweep; its class E (executor/Levanter training) invokes this skill, just as class A/B invoke analyze-job-history-iris.
  • analyze-job-history-iris — the harbor (datagen/eval) analyzer; does NOT apply to training runs.
  • rl-job-health-deep-dive — the GPU-RL equivalent (also no harbor sidecars; uses the finelog).

© open-thoughts, Apache-2.0. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

Just SKILL.md in .agents/skills/analyze-training-run-iris of open-thoughts/OpenThoughts-Agent.

Open the folder on GitHubat commit 3bd1917

Compare with similar skills

Analyze Training Run Iris next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

Analyze Training Run Iris compared with similar skills
SkillStarsUsed inTokensAuto-checkLicenceRepo updated
Analyze Training Run Iris this skillopen-thoughts/OpenThoughts-Agent301—~2kAutomated safety check: PassApache-2.0
Ito Trainingaffaan-m/ECC276k1 repos~1.5kAutomated safety check: PassMIT
Skin Health Analyzersickn33/agentic-awesome-skills47k2 repos~293Automated safety check: PassMIT
Ray Train Distributed TrainingOrchestra-Research/AI-Research-SKILLs13k2 repos~2.7kAutomated safety check: PassMIT
Codebase Health Dashboardgarrytan/gstack136k—~11kAutomated safety check: NotesMIT
Fal Trainnexu-io/open-design100k—~293Automated safety check: PassApache-2.0

Similar skills

  • Ito Training

    affaan-m/ECC

    Inspect the availability of ML training on a completed Itô compute booking and, when the canonical backend becomes available, hand off an explicitly confirmed training manifest.

    276k GitHub starsUsed in 1 repo~1.5k tokens
    AI & LLM EngineeringAuto-check passed

  • Skin Health Analyzer

    sickn33/agentic-awesome-skills

    Analyze skin health data, identify skin problem patterns, assess skin health status.

    47k GitHub starsUsed in 2 repos~293 tokens
    Productivity & AutomationAuto-check passed

  • Ray Train Distributed Training

    Orchestra-Research/AI-Research-SKILLs

    Scales PyTorch, TensorFlow and Hugging Face training from a single GPU to multi-node clusters with Ray Train, including Ray Tune sweeps and checkpoint recovery.

    13k GitHub starsUsed in 2 repos~2.7k tokens
    AI & LLM EngineeringAuto-check passed

  • Codebase Health Dashboard

    garrytan/gstack

    Runs a project's own type checker, linter, test runner, dead-code detector and shell linter, combines them into a weighted 0-10 score and tracks the trend.

    136k GitHub stars~11k tokensUpdated yesterday
    DevelopmentAuto-check: notes

  • Fal Train

    nexu-io/open-design

    Train custom AI models (LoRA) on fal.ai for personalized image generation tailored to a brand, character, or style.

    100k GitHub stars~293 tokensUpdated today
    AI & LLM EngineeringAuto-check passed

  • Health

    agenticnotetaking/arscontexta

    Run condition-based vault health diagnostics. An agent skill from agenticnotetaking/arscontexta.

    3.5k GitHub stars~7.1k tokensUpdated 7 mo ago
    Auto-check: notes

More from open-thoughts/OpenThoughts-Agent

All 44 skills in this repo

  • Analyze Dataset Token Length

    open-thoughts/OpenThoughts-Agent

    Analyze the token length of an OT-Agent conversation-format (ShareGPT-style) dataset — the per-trace distribution (median/p90/max) and/or counts under a token threshold + a metadata predicate (e.g.

    301 GitHub stars~1.5k tokensUpdated 12 days ago
    Auto-check passed

  • Analyze Id Eval Ranking

    open-thoughts/OpenThoughts-Agent

    Given a list of models (HF name stubs) that have valid agentic ID eval scores in Supabase, build a ranking table: raw per-benchmark accuracy on the 3 ID benchmarks (SWE-Bench-100…

    301 GitHub stars~3.1k tokensUpdated 12 days ago
    Auto-check passed

  • Analyze Job History Iris

    open-thoughts/OpenThoughts-Agent

    Run the Iris harbor job-history analyzer (scripts/iris/analyzeirisharborjob.py) on a datagen/eval job and read its JSON sidecar for trustworthy throughput / preemption / productive-trial stats.

    301 GitHub stars~2.9k tokensUpdated 12 days ago
    Auto-check passed

  • Analyze Rl Behavior

    open-thoughts/OpenThoughts-Agent

    Run the full RL behavioral-analysis pipeline (scripts/analysis/analyzerlbehavior.py) on a trained RL model to understand WHAT changed vs its pre-RL baseline, WHY, whether it PERSISTS, and its EVAL…

    301 GitHub stars~4.2k tokensUpdated 12 days ago
    Auto-check passed

  • Code Create Staged Plan

    open-thoughts/OpenThoughts-Agent

    DESIGN a non-trivial codebase change (Harbor / MarinSkyRL / vLLM / OT-Agent / LLaMA-Factory) as a dependency-ordered STAGED PLAN before writing code — a feature port, a multi-step fix with parity…

    301 GitHub stars~1.5k tokensUpdated 12 days ago
    Auto-check passed

  • Commit

    open-thoughts/OpenThoughts-Agent

    Lint, run the pre-PR checks, commit, push, and author or update the branch's pull request in the required plain-text format.

    301 GitHub stars~2.2k tokensUpdated 12 days ago
    Auto-check: notes

Questions about Analyze Training Run Iris

What does Analyze Training Run Iris do?

Detailed health check for a Levanter/executor TRAINING run on the marin Iris cluster (e.g. Analyze Training Run Iris is an agent skill from open-thoughts/OpenThoughts-Agent.g.

When should I use Analyze Training Run Iris?

Analyze Training Run Iris fits situations like: an executor coordinator (<run-coord) plus its nested <run-coord/checkpoints-<step-<hash training child; which the harbor analyzer (analyze-job-history-iris) does NOT cover (training has no harbor trial sidecars; same as GPU-RL).

How do I install Analyze Training Run Iris in Claude Code?

Run `npx skills add open-thoughts/OpenThoughts-Agent --skill analyze-training-run-iris -a claude-code`. Or copy the skill folder (.agents/skills/analyze-training-run-iris in open-thoughts/OpenThoughts-Agent) into .claude/skills/analyze-training-run-iris in your project. Claude Code loads it when a task matches its description.

How do I install Analyze Training Run Iris in Codex?

Run `npx skills add open-thoughts/OpenThoughts-Agent --skill analyze-training-run-iris -a codex`. Or copy the skill folder (.agents/skills/analyze-training-run-iris in open-thoughts/OpenThoughts-Agent) into .agents/skills/analyze-training-run-iris in your project. Codex loads it when a task matches its description.

Can I use Analyze Training Run Iris in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add open-thoughts/OpenThoughts-Agent --skill analyze-training-run-iris -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/analyze-training-run-iris, .gemini/skills/analyze-training-run-iris, .github/skills/analyze-training-run-iris and .opencode/skills/analyze-training-run-iris in your project.

What does Analyze Training Run Iris need to run?

Going by SKILL.md and its folder, Analyze Training Run Iris needs the command-line tools its instructions call (gsutil and python) and credentials named WANDB_API_KEY. Our summary lists: Python 3; A credential in WANDB_API_KEY.

Does Analyze Training Run Iris access the network?

SKILL.md contains no URLs. Any network use would come from the scripts or tools the agent runs. This is read from the text; nothing was executed.

Is Analyze Training Run Iris safe to install?

Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.

What licence does Analyze Training Run Iris use?

Analyze Training Run Iris is published under the Apache-2.0 licence (the repository's licence). It allows redistribution, so the full SKILL.md is shown on this page.

How many tokens does Analyze Training Run Iris use?

About 2k tokens (SKILL.md is roughly 8.1k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full.

What are the alternatives to Analyze Training Run Iris?

Skills that share tags, products or a category with Analyze Training Run Iris: Ito Training (affaan-m/ECC, 276k stars), Skin Health Analyzer (sickn33/agentic-awesome-skills, 47k stars), Ray Train Distributed Training (Orchestra-Research/AI-Research-SKILLs, 13k stars) and Codebase Health Dashboard (garrytan/gstack, 136k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains Analyze Training Run Iris?

open-thoughts (a GitHub organization) maintains it in open-thoughts/OpenThoughts-Agent, which has 301 GitHub stars. The repository holds 44 skills in this directory. The repository was last updated on September 28, 2026.

Source: open-thoughts/OpenThoughts-Agent on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.