Nv Generate Vae Finetune

NV-Generate-VAE-Finetune

Purpose

• Used for finetuning the NV-Generate-CTMR MAISI VAE/autoencoder from user-supplied CT or MRI NIfTI training volumes.
• Not for clinical interpretation, regulatory use, or approving synthetic data for production training.
• Upstream currently documents VAE training in train_vae_tutorial.ipynb and provides configs/helpers, but not a scripts.train_vae CLI. This skill does not execute the notebook; it stages the required config/datalist glue locally and uses upstream helper APIs.
• Manifest I/O: inputs are datalist and data_base_dir; outputs are autoencoder_checkpoint, discriminator_checkpoint, and result_json.
• The underlying training contract is the upstream config/env JSON (config_maisi_vae_train.json + environment_maisi_vae_train.json, as used in train_vae_tutorial.ipynb). The wrapper stages those JSON files for you and exposes the most-tuned fields as CLI flags; the sections below document the fields, their defaults, and how to monitor/tune a run.

Instructions

• Read skill_manifest.yaml before changing arguments, side effects, or validation gates.
• Run scripts/run_vae_finetune.py from the Medical AI Skills repo root.
• If a host agent exposes run_script, use run_script("scripts/run_vae_finetune.py", args=[...]); otherwise run the Bash/Python command below.
• Use --preflight first when checking a new datalist; remove --preflight only when the user explicitly wants to launch GPU finetuning.
• For a staged preflight input bundle directory, use BUNDLE/preflight_datalist.json as the datalist and BUNDLE/preflight_dataset as --data-base-dir when those files are present.

Examples

Validate and stage a preflight finetune check from an input bundle (the recommended first step — no GPU, no training). This is the single canonical command; replace INPUT_BUNDLE and OUT_DIR with your paths:

export NV_GENERATE_ROOT="${NV_GENERATE_ROOT:-$HOME/.cache/nvidia-skills/upstreams/NV-Generate-CTMR-61c4ec7}" && \
python skills/nv-generate-vae-finetune/scripts/run_vae_finetune.py \
  INPUT_BUNDLE/preflight_datalist.json \
  --data-base-dir INPUT_BUNDLE/preflight_dataset \
  --output-dir OUT_DIR \
  --modality mri \
  --preflight

For real GPU finetuning and other variations, see Usage below.

Available Scripts

Prerequisites

• An explicit NV_GENERATE_ROOT may point to the caller's local checkout and must contain configs/config_maisi_vae_train.json, scripts/transforms.py, and scripts/utils.py. The result records its current commit.
• If NV_GENERATE_ROOT is unset, the wrapper searches .workbench_data/upstreams/NV-Generate-CTMR.
• CUDA_VISIBLE_DEVICES is optional and can be used to select the GPU for real training.
• Runtime requirements: NVIDIA CUDA GPU for real training, Python packages from the upstream requirements.txt, lpips, and downloaded VAE weights unless using --train-from-scratch.
• Side effects: writes staged configs, checkpoints, TensorBoard logs, and run summaries under the caller-provided --output-dir; may write model caches under the upstream checkout, ~/.cache/huggingface/, and ~/.cache/torch/; may contact https://huggingface.co, https://github.com, and https://download.pytorch.org.
• The datalist is a MONAI-style JSON object with non-empty training[] and validation[] or testing[]. Each entry has an image path relative to --data-base-dir and optional class or modality of ct or mri.

When no local checkout is supplied, create the recommended pinned default checkout once:

if [ -z "${NV_GENERATE_ROOT:-}" ]; then
  export NV_GENERATE_COMMIT=61c4ec709b84cad468852243c48e250bec732074
  export NV_GENERATE_ROOT="$HOME/.cache/nvidia-skills/upstreams/NV-Generate-CTMR-61c4ec7"
  if [ ! -d "$NV_GENERATE_ROOT/.git" ]; then
    git clone https://github.com/NVIDIA-Medtech/NV-Generate-CTMR.git "$NV_GENERATE_ROOT"
    git -C "$NV_GENERATE_ROOT" checkout --detach "$NV_GENERATE_COMMIT"
  fi
fi

1. Config and environment JSON (adapt to your data)

The wrapper copies the upstream VAE config/env JSON from $NV_GENERATE_ROOT/configs, rewrites the fields below, and writes the staged copies under OUT_DIR/workflow/configs/. You normally only set your datalist and data root; the listed CLI flags override individual fields when you need to.

Environment JSON (environment_maisi_vae_train.json):

Training fields (config_maisi_vae_train.json):

--modality (ct or mri, default mri) fills the per-entry class for datalist items missing one. Validation/testing entries are required because the training loop runs a validation pass.

For an end-to-end reference including example data download, see the upstream tutorial train_vae_tutorial.ipynb.