Probe Platforms

Bringing a probe up on a new platform

A harness that works on one OS fails on another for reasons that are almost never the thing the error names. Expect a CHAIN of faults, each hidden by the one in front of it.

⛔ Debug with the SMALLEST payload — the single most expensive mistake here

Measured, bringing the build-jail probe up on Windows: six sequential faults, found using puppeteer as the probe, at ~25 minutes per attempt. Every one was answerable by a question as small as "does the binary spawn?" — seconds with is-odd. About two hours bought six booleans.

Climb this ladder in order. Do not skip to the end.

Each step is seconds. Each isolates one layer. When one passes, expect the next to fail — do not read a green step as "the platform works."

The top fault usually accuses the wrong subsystem. On Windows the probe insisted the binary lacked a cargo feature; the truth four layers down was an unspawnable path, with every per-cell log at 0 bytes. When a diagnostic names a subsystem, verify that subsystem independently before acting on the accusation — and when you fix a misleading diagnostic, it pays for itself immediately.

Windows — all measured

• for /f runs its command through cmd /c, which STRIPS the outer quote pair when the string both starts and ends with a quote. So the natural spelling of a capture, for /f "usebackq delims=" %%i in (`"%EXE%" arg "%DIR%"`), degrades to C:\...\x.exe" arg "C:\...\dir and dies with "The filename, directory name, or volume label syntax is incorrect" — with or without spaces in the path. Fix: wrap the whole command in one MORE quote pair (`""%EXE%" arg "%DIR%""`). Measured against a temp-file-redirect alternative, which did not work.

• An undefined %VAR% expands to its own literal text, so a missing variable is passed onward as the string %VAR% rather than being empty — silently handing a bogus path downstream. Guard with if not defined VAR set "VAR=%CD%", under setlocal so the default does not leak to the caller.

• A .cmd on PATH is invisible to whole classes of CI. In this repo native-deps.yml is the only workflow touching node-gyp and it is ubuntu-only, so the Windows .cmd shim shipped broken and unnoticed for as long as another code path kept it from ever running. Green Windows jobs are not evidence for a Windows file nothing executes — check that some job actually runs it.

• Node aborts at startup inside nub's build jail on Windows: Assertion failed: ncrypto::CSPRNG(nullptr, 0). Any jailed scenario whose script is node therefore cannot report on what it meant to test; skip it with the reason printed rather than recording a pass or a fail.

• spawnSync cannot run npm/npx/pnpm. They are .cmd shims: the bare name gives ENOENT, the .cmd spelling gives EINVAL (Node has refused to CreateProcess a batch file since CVE-2024-27980). shell: true works but is DEP0190 — args are concatenated rather than escaped, so cmd.exe re-parses a spec like @scope/pkg@1.0.0. Run the bundled JS instead: node <path>/node_modules/npm/bin/npm-cli.js.

• Git Bash paths are not spawnable by native tools. cd $(dirname) && pwd under Git Bash yields /c/Users/…, handed to CreateProcess verbatim → ENOENT. Convert to C:\…; for any path passed to a Windows binary use cygpath -w. /tmp/x is NOT where a Windows node will look.

• A copied binary must keep its .exe. Windows decides executability from the SUFFIX, so a content-addressed copy named by bare hash is ENOENT even at 1.2 GB and mode 0755.

• Removing a node_modules tree fails with EPERM routinely — a lifecycle child still holds a handle, or an indexer/AV opens files behind you. Use maxRetries/retryDelay, and never let a cleanup failure discard a measurement that already succeeded (a non-zero exit from tidy-up reads upstream as "no result").

• Disk is a first-class hazard, and a full disk is SILENT AND EVIL. A debug Rust binary can be ~1.2 GB with a target/ of tens of GB; add leaked fixture trees and 80 GB vanishes. When it fills, git fetch fails and the probe runs STALE CODE while you believe you are testing the fix. Size 200 GB+, build --profile fast not debug, and check free space before trusting any run. Growing the GCE disk is not enough — the in-OS partition must be extended too, and Resize-Partition can simply fail.

• Prefer a CI-built binary to provisioning MSVC. A windows-latest workflow that uploads target/debug/nub.exe gives a real MSVC binary for one dispatch, versus installing the VS VCTools workload, cmake, and switching the rustup host triple by hand. A windows-gnu cross-compile is NOT a substitute when the thing under test is OS confinement behaviour.

• If you do build on the VM: rustup may default to windows-gnu (nub needs -msvc), VS Build Tools can be present with no C++ workload so there is no linker at all, and /STACK:8388608 is an MSVC-only flag that breaks a GNU link — it is required under MSVC because Windows gives the main thread 1 MB against Linux's 8 MB.