Light Experiment Coding

What it does

When your agent uses it

• Implementing a preregistered training or evaluation experiment from a frozen plan
• Reproducing a published training or preprocessing pipeline with fixed seeds
• Auditing experiment code for train/test or cross-validation leakage
• Preparing raw run outputs that a later analysis stage can recompute

Example prompts

• “Implement experiment matrix rows 1 to 4 from our frozen plan, writing the tests first.”
• “Review this cross-validation script for preprocessing that was fit on the test fold.”
• “Set up fixed-seed reproducibility for our sklearn baseline and record the environment hashes for each run.”
• “Add gold and metamorphic tests for the feature scaling step in src/preprocess.py.”

Requirements

• Python with `uv` for the bundled project scaffold
• Frozen upstream plan, matrix and data lineage files

Workflow steps

5 steps, taken from the step headings in SKILL.md.

1. 1建立最小可运行项目
2. 2测试先行
3. 3防泄漏实现
4. 4控制随机性与数值边界
5. 5每个 run 保留 raw bundle

What it can do on your machine

Read from SKILL.md and the folder at commit 6b44f57. It shows what the files ask for, not the result of running them.

• Tool permissions

  Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

  From allowed-tools in the SKILL.md frontmatter.

• Runs code

  Ships 1 file in scripts/ (Python and Shell, from the files we listed), which the agent can run.

  Shell commands in SKILL.md call:

  • python
  • uv

  From the folder's file list and the shell code blocks in SKILL.md.

• Network

  No URLs in SKILL.md. Its commands use uv, which can reach the network depending on how they are called.

  From URLs in SKILL.md, links to its own repository left out.

• Credentials

  Names no API keys, tokens, secrets or passwords.

  From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Light Experiment Coding loads about 2.3k tokens when it runs, and up to ~8.5k if it reads all its reference files. Until then it costs about 116 tokens; SKILL.md has 588 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it

~116

When it runs · the whole SKILL.md, loaded when a task matches

~2.3k

With references · SKILL.md plus every file in references/, read only if the agent opens them

~8.5k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); the scripts in this folder are not scanned.

SKILL.md

The full file from Light0305/Light-skills at commit 6b44f57, republished under its MIT licence (© Light0305). 588 words, ~2,296 tokens.

name

light-experiment-coding

description

Light 科研主线 stage 6：把冻结的 question/estimand、experiment matrix、pre-registration 与 data lineage 落成最小可运行、测试先行、无泄漏、可复现且能交给 result-analysis 的实验代码。用于实现或复现训练/预处理/评测， 设计 gold/property/metamorphic 测试，控制 Python/NumPy/框架/CUDA/DataLoader 随机性，审查 train/test 或 CV fit 穿越，记录 config/code/environment/input hashes、stdout/stderr、raw metrics、patient/entity predictions 与 failure artifacts，以及运行 stage-6 checkpoint。数据泄漏或不可复现是 critical；静态扫描和同 seed 两次一致都不证明跨硬件 绝对复现。

metadata.version

2.2.0-round3

metadata.truth_source

../../docs/competitors/experiment-coding.md

metadata.resource_map

references/experiment-coding-resource-map.md

metadata.engine

scripts/experiment_execution_contract.py · scripts/repro_gate.py · scripts/seed_audit.py · scripts/review_gate.py · scripts/run_artifact_check.py

metadata.emits

light.findings.v1 · light.run_manifest.v3 · raw run bundles

metadata.consumes

research-plan frozen plan/preregistration/failure-tree report · data-engineering lineage/split_leakage · result-analysis raw-run contract

metadata.stage

6

python scripts/experiment_execution_contract.py --spec experiment_execution_contract.json \
  --report execution_contract_findings.json --json-out execution_contract_report.json
python scripts/review_gate.py src/ --json
python scripts/seed_audit.py src/train.py src/reproducibility.py
python scripts/repro_gate.py --spec repro_spec.json --report repro_findings.json
python scripts/run_artifact_check.py --manifest runs/EXP-01/run-a/manifest.json
python scripts/run_artifact_check.py --compare \
  runs/EXP-01/run-a/manifest.json runs/EXP-01/run-b/manifest.json
python ../light-orchestrator/scripts/run_checkpoint.py \
  --file .light/passport.yaml --stage 6 --findings repro_findings.json --write --ts <ISO-8601>

python ../light-orchestrator/scripts/reroute.py \
  --findings result_findings.json --stage 7 --passport .light/passport.yaml

© Light0305, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

SKILL.md and 37 other files (scripts, references, assets) in skills/light-experiment-coding of Light0305/Light-skills.

• SKILL.md
• assets/project-scaffold/.github/workflows/ci.yml
• assets/project-scaffold/.pre-commit-config.yaml
• assets/project-scaffold/CODE_REVIEW_CHECKLIST.md
• assets/project-scaffold/README.md
• assets/project-scaffold/configs/experiment.example.json
• assets/project-scaffold/configs/experiment.schema.json
• assets/project-scaffold/pyproject.toml
• assets/project-scaffold/scripts/boundary_trace.py
• assets/project-scaffold/scripts/debug_instrument.sh
• assets/project-scaffold/src/example/__init__.py
• assets/project-scaffold/src/example/experiment_contracts.py
• assets/project-scaffold/src/example/reproducibility.py
• … and 25 more

Open the folder on GitHubat commit 6b44f57