---
name: capability-horizon-estimator
description: Estimate whether an AI model can complete a task and how long it will take, using METR-style time-horizon modeling. Use when scoping agent work, deciding if a task is within reach, planning retries/parallelism, estimating wall-clock time for SWE/MLE/math tasks, or answering "can you do this" / "how long will this take" for autonomous work.
---

# Capability & Time Horizon Estimator

Estimate what a model can do and how long it takes using the METR time-horizon framework: every task has a human-equivalent duration $t$, every model has a 50% time horizon $h_{50}$, and success probability follows a logistic curve in $\log(h/t)$.

## When to use

- "Can you do X autonomously?" — give a calibrated yes/maybe/no instead of vibes
- "How long will this agent run take?" — wall-clock estimate with retry overhead
- Scoping: should this be one task, or decomposed?
- Deciding attempt budget (best-of-N) for a stretch task

## The core model

**Step 1 — Estimate human-equivalent task time $t$ (minutes).**
Anchor against known reference points (full table in `references/benchmark-catalog.md`):

| Reference task | Human time |
|---|---|
| SWE-bench Verified issue | 7 min – 2 h |
| HCAST task bands | 15 min / 1 h / 4 h / 8 h |
| RE-Bench ML research task | 8 h |
| Small bug fix, clear repro | 15–60 min |
| Multi-file feature | 2–8 h |
| Cross-repo refactor | 4–16 h |
| Kaggle competition (MLE-bench) | days of human effort |
| FrontierMath T4 problem | days–weeks of expert time |

Estimate for a *low-context* professional (new hire, contractor), not the resident expert — that is what the horizons are calibrated against.

**Step 2 — Get the model's horizon $h_{50}$ (minutes).**
Look it up in `references/horizon-data.md` (METR TH v1.1, May 2026). If the model isn't listed, extrapolate from release date:

$$h_{50}(\text{date}) = h_{50}(\text{ref}) \times 2^{(\text{date} - \text{ref}) / D}, \quad D \approx 130\text{–}190 \text{ days}$$

Use $D = 150$ days as default; state the range. Post-2024 data supports faster ($\sim$90–130 days); all-time average is $\sim$190.

**Step 3 — Success probability.**

$$p = \sigma\!\big(\beta \cdot \ln(h_{50}/t)\big), \quad \beta \approx 0.8 \text{ (range 0.6–0.9)}, \quad \sigma(x) = \frac{1}{1+e^{-x}}$$

Sanity anchors with $\beta = 0.8$: $t = h_{50} \Rightarrow p = 50\%$ · $t = h_{50}/5.7 \Rightarrow p = 80\%$ · $t = 2h_{50} \Rightarrow p \approx 36\%$ · $t = 4h_{50} \Rightarrow p \approx 25\%$.

**Step 4 — Difficulty adjustments (multiply $t$ before Step 3).**

| Condition | Multiplier on $t$ |
|---|---|
| Well-specified, auto-verifiable (benchmark-like) | ×1 |
| Proprietary/large codebase, high prior context needed | ×2 |
| Messy, underspecified, human-judged success | ×4 (range ×2–8) |
| GUI/computer-use (no API/DOM) | ×50–100 — different capability cluster |
| Parallelizable/additive work (N independent items) | Not one task — split, apply model per piece |

Rationale: METR's suite is clean and self-contained; models do measurably worse on messy tasks and under holistic (vs algorithmic) scoring. GUI horizons measured ~40–100× shorter than SWE/reasoning.

**Step 5 — "Pushed itself" uplift.**
- Better scaffold/elicitation: ×1.5–2 on $h_{50}$ (scaffold choice moves measured horizons ~2×)
- Best-of-N independent attempts: $P(\geq 1 \text{ success}) = 1 - (1-p)^N$
- Both compose. A $p = 0.25$ task with a good scaffold ($\times 1.75 h$) and 6 attempts: $p \to \sigma(0.8 \ln(1.75 \cdot h/t)) \approx 0.37$ per attempt, $1-(1-0.37)^6 \approx 94\%$.

**Step 6 — Wall-clock time.**
Agents complete tasks they *can* do several times faster than the human baseline:

$$\text{wall-clock} \approx \frac{t_{\text{human}}}{S} \times \text{expected attempts}, \quad S \approx 3\text{–}8\times \text{ for SWE (default 5)}$$

Expected attempts $= 1/p$ for independent retries (cap at 3–5 in practice; if it fails 3×, the plan is wrong, not the luck). Report a range, not a point.

## Output format

```
TASK: <one line>
HUMAN-EQUIVALENT TIME (t): <estimate> (anchor: <reference task>)
MODEL HORIZON (h50): <value> (source: table / extrapolated from <date>, D=150d)
DIFFICULTY: ×<mult> (<reason>)  → effective t = <value>
SINGLE-ATTEMPT SUCCESS: <p> → <verdict>
PUSHED (scaffold ×1.75, N=<n>): <p_eff> per attempt, <P> overall
WALL-CLOCK ESTIMATE: <range> (t/S × attempts, S=5)

VERDICT: CONFIDENT (≥80%) | LIKELY (60–80%) | COIN-FLIP (35–60%) | STRETCH (15–35%) | OUT OF REACH (<15%)
CAVEATS: <extrapolation flags, messiness, jagged-domain warnings>
DECOMPOSITION ADVICE: <if stretch/out-of-reach: where to split>
```

## Calibration rules

- **Always state the anchor.** Every $t$ estimate names the reference task it was derived from. No anchor, no number.
- **Report ranges.** Horizons have wide CIs (Opus 4.6 p50: 317–3634 min). Give the interval, pick the geometric mean for the headline.
- **Above 16 h human-equivalent, say so.** METR's suite saturates at 16 h; estimates beyond it are extrapolation, not measurement.
- **Jaggedness is real.** A model strong in SWE may be 50–100× weaker on GUI work, and different on math research. Never transfer a horizon across domains without the multiplier.
- **Long ≠ hard, hard ≠ long.** The model applies to *coherent, non-parallelizable* tasks. 1000 independent 1-hour problems is a 1-hour task done 1000 times.
- **Failure mode asymmetry.** Near the horizon, models fail by derailing (repeating failed approaches, losing state), not by slow progress. If $p < 50\%$, recommend decomposition over retries.

## Reference files

- `references/horizon-data.md` — METR TH v1.1 per-model p50/p80 table (2019 → May 2026), doubling-time stats, methodology caveats
- `references/benchmark-catalog.md` — SWE/MLE/math benchmark inventory with task times, human budgets, and July-2026 SOTA
- `scripts/estimate.py` — calculator for Steps 3–6 (logistic p, best-of-N, wall-clock)