Benchmarking

What it does

Example prompts

• “/benchmarking”

Requirements

• Docker

What it can do on your machine

Read from SKILL.md and the folder at commit 568e73a. It shows what the files ask for, not the result of running them.

• Tool permissions

  Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

  From allowed-tools in the SKILL.md frontmatter.

• Runs code

  Shell commands in SKILL.md call:

  • bun
  • pnpm
  • docker

  From the folder's file list and the shell code blocks in SKILL.md.

• Network

  No URLs in SKILL.md. Its commands use pnpm and docker, which can reach the network depending on how they are called.

  From URLs in SKILL.md, links to its own repository left out.

• Credentials

  Names no API keys, tokens, secrets or passwords.

  From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Benchmarking loads about 1.8k tokens when it runs. Until then it costs about 215 tokens; SKILL.md has 783 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it

~215

When it runs · the whole SKILL.md, loaded when a task matches

~1.8k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from nubjs/nub at commit 568e73a, republished under its MIT licence (© nubjs). 783 words, ~1,844 tokens.

name

benchmarking

description

Comparative install-benchmarking methodology for nub vs npm/pnpm/bun — cold/warm protocol, genuine-cold cache isolation, load-robust measurement, and the anti-juicing honesty bar. Invoke (via the Skill tool) whenever you need to benchmark `nub install` against another package manager, produce or update the homepage/blog install numbers, or verify a perf claim before it ships. Encodes the hard-won gotchas: time setup OUTSIDE the measurement (hyperfine `--prepare`), the cache lives on DISK so env-var isolation is NOT trustworthy (bun ignores `BUN_INSTALL_CACHE_DIR`/`$HOME` — wipe the real path), VERIFY every cold is genuine via an offline-fails check, and only measure wall-clock on a quiet machine (gate on low load) with file counts as a load-independent cross-check. Pairs with `pm-perf-tracing` for the internal Rust phase decomposition.

metadata

internal: true

# COLD: empty the tool's REAL cache + wipe node_modules before each run (untimed), then time the install.
hyperfine --warmup 0 --runs 5 \
  --prepare 'rm -rf node_modules && d="$(bun pm cache)" && rm -rf "${d:?}"' \
  'bun install --ignore-scripts'

# WARM-RELINK: cache populated, wipe ONLY node_modules before each run.
hyperfine --warmup 1 --runs 5 \
  --prepare 'rm -rf node_modules' \
  'nub install --ignore-scripts'

# WARM-SAT: node_modules already present (idempotency path) — no prepare wipe.
hyperfine --warmup 1 --runs 5 'nub install --ignore-scripts'

find node_modules -type f -o -type l | wc -l          # total materialized entries
ls -d node_modules/**/core-js node_modules/core-js    # physical copies of a duplicated dep (dedup story)

© nubjs, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

Just SKILL.md in .claude/skills/benchmarking of nubjs/nub.

Open the folder on GitHubat commit 568e73a