Rust Build

rust-build

Cross-worktree cache reuse wants a shared target dir; correctness wants isolation. scripts/rust-build.sh resolves it — share by default, isolate automatically only when a worktree diverges a crate other crates link.

Use it

Drop-in for cargo, from any worktree (or the main tree):

scripts/rust-build.sh build -p nub-cli --profile fast
scripts/rust-build.sh test  -p nub-cli --test integration
NUB_ALLOW_INCOMPLETE_RUNTIME=1 scripts/rust-build.sh clippy --all-targets --all-features -- -D warnings

That third line needs a staged runtime/ first, and opts out of the vendored-deps half because a lint ships nothing. The "--all-features needs a staged addon AND the vendored runtime deps" section below has the placeholder recipe and the two failure modes.

It prints which target dir it chose and why, then execs cargo with CARGO_TARGET_DIR set. Two default-on contention controls ride along:

• QoS clamp (darwin only): cargo runs under taskpolicy -c utility, so interactive work preempts fleet builds; an uncontended build still gets all cores. NUB_BUILD_FG=1 opts out.
• Job cap on big hosts (>8 cores): CARGO_BUILD_JOBS = ncpu-4 unless the caller already chose (pre-set CARGO_BUILD_JOBS, NUB_BUILD_JOBS, or an explicit -j/--jobs — cargo's flag outranks the env var).

These cover builds going THROUGH the wrapper (or make). Direct cargo invocations are clamped by a machine-global control: make qos-global installs scripts/rustc-qos.sh as the cargo rustc-wrapper in ~/.cargo/config.toml, so every rustc on the host compiles at utility QoS, at most TWO builds compile at a time (the rest queue first-come-first-served; make build-status shows the holders and the queue), and the compiling builds share NUB_RUSTC_LIMIT (default 6) rustc tokens (install-dev re-runs it, so it self-heals). NUB_BUILD_FG=1 opts a HUMAN's foreground build out of the clamp and the queue (and, through this wrapper, out of the tokens too) — never set it from an agent; NUB_BUILD_SLOTS=0 in a build's environment disables only the slot layer for that build. Toggling any of these does not invalidate fingerprints. Details: the rust-build-hygiene skill.

Why one shared target dir

All worktrees default to ~/.cache/nub/shared-target. A fresh worktree reuses the crates.io dependency rlibs a sibling already compiled and recompiles only the ~3 workspace crates — ~5s instead of a ~3-min cold build.

Relocation does NOT defeat reuse. Cargo revalidates a relocated target dir in place: cloning a warm dir to a new path rebuilt 0 crates; an empty dir rebuilt all 13. (The "path is baked into fingerprints" claim is true of sccache, which keys on the rustc command line including absolute --out-dir / -L dependency= paths — not of cargo.) So sharing a live path buys convergence; a CoW clone buys a warm start, which is why the wrapper seeds a fresh private dir from the matching bucket instead of cold-starting.

The hazard sharing creates

Cargo names a crate's output by package id (name + version), not source content. Two worktrees whose source for the same depended-on crate differs — classically vendor/aube on divergent branches — write the same output slot and clobber each other. A dependent crate then links the stale rlib:

error[E0063]: missing field `lockfile_legacy_basenames` in initializer of `aube_util::Embedder`

— a field that exists nowhere in your checkout. Only bites crates that other crates link; a divergent leaf binary (nub-cli) just rebuilds cleanly.

The rule the wrapper enforces

Sharers are grouped by the content of their depended-on crates, hashed into the bucket name (shared-target-<hash>):

• Depended-on crate sources and runtime/ unmodified → share that content's bucket. Everyone in it agrees by construction. The common case: feature work in nub-cli, integration tests, docs.
• Diverged a depended-on crate or runtime/ → isolate to a private per-worktree target/ (removed with the worktree), CoW-seeded from the matching bucket so you rebuild only what differs. runtime/ is hashed for a different reason than the crates: a dev binary resolves runtime/*.cjs from the tree that compiled nub-core (baked CARGO_MANIFEST_DIR), so a shared-bucket binary loads whichever sharer compiled last — a worktree with edited runtime files would silently test a sibling's copy.

Content, not merge-base: a merge-base proves only that this worktree made no local changes against its own base, so two worktrees whose bases straddle a nub-core/aube commit both pass while disagreeing on content. The key hashes the git index, which only matters on the shared branch (no local changes by definition), so it moves on rebase, never mid-edit.

The hashed set = every workspace/vendored crate except leaf artifacts nothing links — crates/nub-cli (bin), crates/nub-native (cdylib, own workspace), crates/nub-phantom (bin, own workspace) — plus runtime/. So: crates/nub-core, crates/nub-cache-key, crates/nub-phantom-core, crates/nub-phantom-scan, all of vendor/aube, and runtime/. nub-phantom-core/nub-phantom-scan are not leaves — nub-cli depends on both — and the pathspec :(exclude)crates/nub-phantom matches only that directory, not those siblings.

Letting the wrapper choose IS the caching strategy

export CARGO_TARGET_DIR overrides the wrapper's resolution — it is how caching is LOST, not gained: you land in a cold dir, or a multi-tenant one you can clobber.

Never put an exported CARGO_TARGET_DIR in a dispatch prompt. A sub-agent cannot know whether the value is the warm bucket, a cold path, or a dir a sibling is mid-build in; the wrapper does, and it prints which it chose and why.

Pin one only when seeding cannot fire — a branch cut before seeding landed, or a filesystem without CoW:

grep -c seed scripts/rust-build.sh      # 0 → this branch has no seeding; pinning may pay

When you must pin for a serial multi-phase epic, point the whole chain at ONE dedicated private target so deps compile cold once:

export CARGO_TARGET_DIR=~/.cache/nub/<epic>-target

• Serial only — cargo's target-dir lock serializes builds; never point two concurrently-building worktrees at it.
• Dedicated, NOT shared-target — the shared dir is multi-tenant.
• A review/verify sub-agent reuses the implementer's already-warm target, never a fresh one.

For a orchestrator run, pinning buys nothing: every lane edits ONE worktree and the orchestrator builds that same worktree.

Trade-offs and edges

• A worktree that edits aube pays a cold build even with no sibling diverging aube concurrently. The invariant is "match origin/main," which doesn't depend on volatile sibling state — that's what makes it robust.
• Concurrent builds in two sharing worktrees serialize on cargo's target-dir lock. A latency cost, never a correctness one. Need two at once? Isolate one.
• NUB_SHARED_TARGET relocates the target dir and the path is used exactly as given — not content-keyed, because a caller naming a path is asking for that path (make verify relies on this to reach $(CURDIR)/target). The value must be private to one checkout; two worktrees on one relocated path recreates the phantom-E0063 clobber. To relocate a cache several worktrees share, leave it unset and move ~/.cache/nub itself.
• Cleanup: git worktree remove <path> --force drops the worktree and its private target/; the shared dir is intentionally left in place.
• A build script can pin an ABSOLUTE PATH into the shared dir — a second contamination shape. A build script resolving inputs through compile-time env!("CARGO_MANIFEST_DIR") bakes the compiling worktree's path into the cached build-script binary, which is cached per package id and survives into every sharing worktree. Symptom: failed to read /…/worktrees/<some-other-tree>/… naming a directory absent from your checkout. Fixed at the source; for a stale one, rm -rf <target-dir>/*/build/<crate>-* and rebuild. Note cargo clean -p <crate> from the repo ROOT is a silent no-op for the aube crates — not root workspace members, so it reports Removed 0 files and exits 0.
• Running the vendored aube test suite from the repo root drops its serial-execution pin. vendor/aube/.cargo/config.toml sets RUST_TEST_THREADS = "1" because some aube-util tests mutate process env, and cargo discovers config from the CWD, not --manifest-path — so cargo test --manifest-path vendor/aube/Cargo.toml … from the root runs them in parallel and produces failures CI never sees. Run it as CI does: (cd vendor/aube && cargo test …).

The gates run on TWO profiles — budget for two dependency builds

fast and dev are different target subdirectories, so clippy artifacts do not serve cargo test.

Do not unify them: moving cargo test onto fast diverges from CI, and dropping fast from clippy drives a second full dependency build under dev (~26 GB of duplicated target/debug + target/fast). fast inherits dev — identical debug-assertions, overflow checks and opt-level, differing only in debuginfo, which no lint reads.