Model Scaffold

Model-Scaffold Skill

Purpose

This skill stamps out a runnable PyTorch training repo for a medical-imaging task — --task segmentation (U-Net), classification (CNN / timm backbone), detection (torchvision Faster R-CNN / FPN), synthesis (Pix2Pix generator + PatchGAN), ssl (SimCLR encoder), or finetune (transfer-learning a pretrained backbone with a frozen→unfrozen schedule + a provenance record) — with the reproducibility guarantees baked in by construction — so the build is leakage-safe and reproducible before a single epoch runs. It is the imaging analogue of how /analyze-stats generates runnable statistical code: the generator produces the repo, you run the training on your GPU / Colab, and the lane's deterministic gates verify the network-free parts.

It is the missing middle link in the lane: /model-selection (choose) → model-scaffold (build) → /model-assessment (validate the split / design, compute metrics) → /analyze-stats → /write-paper + /check-reporting (publish). It integrates MONAI / nnU-Net / TorchIO (referenced in the generated requirements.txt); it does not reimplement them.

When to use

• You have a data manifest (one row per image, with a patient/subject ID) and want a reproducible, leakage-safe starting repo for a segmentation, classification, detection, synthesis or self-supervised model.
• You want to fine-tune a pretrained backbone (transfer learning — the common clinician workflow: a timm / MONAI / MedSAM checkpoint adapted to your collected clinical data) with the freeze schedule, discriminative learning rates, and pretrained-weight provenance recorded (--task finetune).

When NOT to use

• Auditing an already-trained model's validation design → /model-assessment.
• Held-out metrics / calibration / bootstrap CIs → /model-assessment then /analyze-stats.
• Choosing the architecture for the research question → /model-selection.
• Reimplementing MONAI / nnU-Net → out of scope (the scaffold integrates them).
• LLM / MLLM evaluation → /mllm-eval.

Workflow

Phase 1 — Prepare the manifest

A CSV with one row per image and a patient/subject ID column (patient_id / subject_id / case_id), plus image and label path columns. The ID column is load-bearing: the split is done at the patient level off this column. IDs are compared after stripping surrounding whitespace (P01 and P01 are one patient), in the split and in the generated dataset.py alike.

Phase 2 — Generate the repo

python3 ${CLAUDE_SKILL_DIR}/scripts/scaffold.py \
  --manifest <manifest.csv> --task segmentation --out model_repo --seed 42 \
  --in-channels 1 --out-channels 1
# --task = segmentation | classification | detection | synthesis | ssl | finetune
#   (out-channels = num classes for classification/finetune, target channels for synthesis;
#    finetune uses a softmax CrossEntropy head, so it refuses --out-channels < 2 — binary = 2)
# fine-tuning a pretrained backbone (transfer learning) on collected clinical data:
python3 ${CLAUDE_SKILL_DIR}/scripts/scaffold.py \
  --manifest <manifest.csv> --task finetune --out model_repo --seed 42 \
  --out-channels <num_classes> --from-pretrained timm:resnet50.a1_in1k
#   emits PRETRAINED.md (provenance) + a frozen→unfrozen train.py with discriminative LRs;
#   record the exact pretrained source so the fine-tune is reproducible. build_model(pretrained=True)
#   raises if timm is missing (never a silent random-init stand-in); best.pt records backbone_class.
# reuse the split /imaging-data's preprocessing gate checked (do not draw a new one):
python3 ${CLAUDE_SKILL_DIR}/scripts/scaffold.py \
  --manifest <manifest.csv> --preprocessing-manifest preprocessing_manifest.json --out model_repo
#   copies its split_assignment + split_seed into splits/, reading rows with the gate's own rules
#   (patient key patient_id/subject_id/patient/id; split synonyms such as training/validation/holdout);
#   exits 2 if a manifest patient has no split there, a patient sits in two splits, a split does not
#   map to train/val/test, or split_seed is absent.

The imaging-data QC handoff is enforced, not advisory. With --preprocessing-manifest the scaffold also reads /imaging-data's gate reports (check_dataset_profile, check_preprocessing_leakage, check_normalizer_domain JSON) from <manifest dir>/qc/ and <manifest dir>/../qc/ (the manifest path is resolved first). --imaging-qc <file|dir> (repeatable) replaces that search: pointed at an empty directory, every gate is recorded NOT ASSESSED. A leakage report is skipped only when its recorded manifest has a different file name, resolves to an existing file, and no candidate path holds a byte-identical copy of the scaffolded manifest (the reason is shown); anything else — a copied project, a same-named manifest elsewhere — is read.

• A Major claim — or any severity that is not plainly Minor/Flag, or a report whose summary.n_major exceeds its listed Majors — refuses: exit 1, nothing written, each code + report listed. Resolve it upstream and re-run the gate, or pass --ack-qc CODE='reason' once per code.
• Minor / Flag claims never block; they are carried forward as warnings.
• A gate with no readable report is NOT ASSESSED; an unparseable or off-shape file in qc/ is listed as UNREADABLE (stderr + record), never dropped.

All of it lands in model_repo/IMAGING_QC.md, referenced from config.yaml (imaging_qc:) and REPRODUCIBILITY.md, so training, evaluation and the Methods read what /imaging-data found. Never write an --ack-qc reason the user has not given. Without either flag the output is unchanged. This writes model_repo/ with config.yaml, model.py (the task's model — U-Net / CNN / Faster R-CNN / Pix2Pix / SimCLR encoder), dataset.py (reads the frozen split), losses.py (task-appropriate), train.py, evaluate.py, requirements.txt, REPRODUCIBILITY.md, methods_stub.md (+ IMAGING_QC.md when imaging-data outputs are given), and — the key artifact — splits/split_assignment.csv + splits/split_seed.txt. The split is patient-disjoint by construction (a deterministic group split) and the emitted code seeds every RNG, sets cuDNN deterministic, builds the training loader from the train split only, and infers under model.eval() + torch.no_grad().

Phase 3 — Verify the build (network-free)

# this skill's own training-hygiene gate
python3 ${CLAUDE_SKILL_DIR}/scripts/check_training_hygiene.py --repo model_repo --strict
# the split-leakage gate (proves patient disjointness) — owned by /model-assessment

Route the emitted splits/split_assignment.csv to /model-assessment (check_split_leakage.py --splits model_repo/splits/split_assignment.csv --strict) for the patient-disjointness proof, and (optionally, locally with torch installed) bash ${CLAUDE_SKILL_DIR}/scripts/scaffold_challenge/verify.sh to smoke the forward pass.

Phase 4 — Plug in your data and train

Implement dataset.py's _load_image / _load_label for your modality (DICOM / NIfTI / TIFF via nibabel / pydicom / tifffile / TorchIO / MONAI transforms). For production, swap model.py for MONAI UNet / SegResNet or an nnU-Net plan (see ${CLAUDE_SKILL_DIR}/references/training_guide.md). For a fine-tuning repo (--task finetune), fill PRETRAINED.md and set the freeze schedule / discriminative learning rates (see ${CLAUDE_SKILL_DIR}/references/finetuning_guide.md, which also covers MedSAM/SAM adaptation and train-only diffusion augmentation). Run python train.py (best model selected on the val split), then python evaluate.py (predictions on the test split, touched once).