CI Speedup

CI Speedup

Make the wait from push to green shorter, measured, and written down.

• IS: measuring a pipeline's critical path from run timestamps, choosing the levers that shorten it, changing workflow and test configuration, and recording before and after with the trend that says when the next round is due.
• IS NOT: getting a red build green (pr-babysitter), writing a pipeline for a package that has none (scaffold-cli), deleting low-value tests (test-audit), or general repository hygiene.

What Changes the Number

Wall-clock is the slowest chain of dependent jobs, not their sum. The number is push to green for one pull request, median and p90 over recent runs. Around five minutes an agent loop stops waiting on checks and the bottleneck moves to deploy and review. A lever that saves runner minutes off the critical path is a cost saving, and the ledger records it as one.

Three ratios decide what to do next:

• Setup tax. Checkout, runtime, dependencies, and service boot, per job. 110 seconds of setup around 60 of work is a batching candidate; around 400 it is a sharding candidate. Each shard adds a setup, so cut setup before adding shards.
• Import versus test. When Vitest's import plus environment figures exceed its tests figure, the suite pays to boot the module graph per file. A shared registry or a lighter environment moves that; more shards only spread it.
• Gate versus work. Change detection, matrix computation, and cache saves sit on the chain. Anything they do beyond deciding is paid by every job behind them.

Reference Files

evals/evals.json holds scenarios and the with-versus-without comparison for changing this skill; it never loads during a task.

Workflow

Copy this checklist to track progress:

CI speedup progress:
- [ ] Step 1: Baseline (median and p90 over recent runs, per-step table for the longest job, critical path)
- [ ] Step 2: Choose levers by gain on the critical path, with a predicted chain
- [ ] Step 3: Change the pipeline; re-time locally what can be re-timed
- [ ] Step 4: Push, read the branch's own runs as the after-measurement, iterate
- [ ] Step 5: Ledger with trend, instruction-file lines, and the queue of what is left

Done means a mergeable pipeline change, a ledger whose after column comes from runs on the branch, and instruction-file lines for the layout and the test defaults. Reading runs, re-timing locally, and pushing to the working branch need no check-in; spending money (paid runners, more parallel jobs on a metered plan) and merging are the user's.

Step 1: Baseline from timestamps

Use the run and job API, never memory or the workflow file. scripts/ci-timings.sh --runs 20 --workflow ci.yml --branch main gives median and p90 per job; the single-run form on one default-branch run and one pull request gives the steps and the chain. Name the critical path in a sentence: which jobs, in what order, minutes from run start to the last job's end. Pull the slowest test step's log for the runner's own breakdown, and classify every minute on the path as toolchain, gating, setup, or execution before proposing anything.

Step 2: Choose levers

Pick from references/levers.md by gain on the chain from Step 1. Write the predicted new chain with its minutes; a plan without a number cannot be checked. Prefer levers the suite itself proves safe (a sharded run passes or does not) over ones that need judgement per file. Runner minutes may rise when wall-clock falls; say by roughly how much, and whether the repository pays for minutes.

Step 3: Change the pipeline and re-time locally

Edit the workflow and test configuration in one branch. Measure locally whatever can be measured without pushing, with the current and the new configuration. A change that makes a shard fail is a finding, not a reason to skip that shard.

Keep the guards. Change detection, merge-DAG checks, concurrency groups, and deploy gates exist because of a past incident; move them, do not drop them, and keep the comment that cites the incident next to the guard.

Step 4: Push and read the after-measurement

The branch's own runs are the after-measurement. Read them as in Step 1, put the numbers beside the baseline, and apply the verification rules in references/measuring.md before crediting a lever. When predicted and measured disagree, find out why before adding another lever. Repeat until the path stops moving or the remaining levers need a decision the user owns.

Step 5: Ledger and guardrails

Write the ledger from references/ledger.md into the repository's docs: before, changes, after with run ids, a trend row with when the next round is due, and the queue of levers not taken with what each waits on. Add the instruction-file lines for the layout and for how a new test lands in the fast path. Done when the after column comes from real runs, not the prediction.