Train Lora

Train LoRA Skill

Use this workflow to validate Draw Things LoRA training end to end with draw-things-cli.

Goal

Train on a tiny local dataset, watch loss and scaler health, then verify visually with a reference/base/LoRA comparison.

Build Once

Build the optimized CLI first:

bazel build --compilation_mode=opt //Apps:DrawThingsCLI

Use the built binary for every run:

bazel-bin/Apps/DrawThingsCLI

Do not switch between bazel run and bazel-bin/... during one validation cycle unless you need to. Reuse the same binary so compile/runtime behavior stays comparable and permission prompts stay predictable.

Runtime Notes

• Graph compile can be quiet for a long time. With --compilation_mode=opt, 5 to 30 minutes is possible on heavy trainers. Do not assume a hang too early.
• Training checkpoints are written into the models directory.
• If the environment requires command approvals, ask once for a stable command shape and stable output/log names, then rename artifacts afterward.
• For local unregistered LoRAs, prefer passing explicit loras[].version during generation instead of depending on custom_lora.json.

Dataset Setup

For a single-image reconstruction check:

• Create a local dataset directory.
• Put the image in that directory.
• Add a matching .txt caption file beside it.
• Keep the caption minimal for trigger-only tests, for example:

zimgdogref

Validation Ladder

Use this order:

1. Run a 1-step smoke test to confirm the graph compiles, the loss is finite, and a checkpoint is written.
2. Run a 20-step probe to confirm scale stays healthy and loss is not obviously blowing up.
3. Run a 100-step check to see whether like-for-like timestep bands decline.
4. Run a 500-step run before declaring the trainer healthy.
5. If you are validating a new attention backend, run the 500-step check on that intended backend, not only on a fallback path.
6. Generate a base image and a LoRA image with the same prompt, seed, and settings.
7. Compose reference/base/LoRA into one image for visual review.

Baseline Train Command

Use this as the generic 512x512 single-image baseline:

bazel-bin/Apps/DrawThingsCLI train lora \
  --models-dir /Users/liu/Library/Containers/com.liuliu.draw-things/Data/Documents/Models \
  --model MODEL.ckpt \
  --dataset /tmp/single_image_dataset \
  --output RUN_NAME \
  --name RUN_NAME \
  --steps 500 \
  --rank 32 \
  --scale 1 \
  --learning-rate 0:4e-4 \
  --gradient-accumulation 4 \
  --warmup-steps 20 \
  --save-every 100 \
  --width 512 \
  --height 512 \
  --seed 7 \
  --config-json '{"steps_between_restarts":200}' \
  --no-download-missing \
  --offline

Model Baselines

Scaler Rules

• Healthy scale is architecture-dependent; choose it from the model's numeric contract.
• Do not set scale lower than 1 to make a run stable. That hides overflow and can prevent useful learning.
• If the model does not apply internal scaling that shrinks gradients, start from 32768.0.
• If the model has explicit internal downscaling or projection compensation, use the validated lower scale for that model family and record why.

FLUX.1

• Base model: flux_1_dev_q8p.ckpt
• Validated guidance settings:
  • guidanceScale = 3.5
  • guidanceEmbed = 3.5
  • shift = 2
  • resolutionDependentShift = false
• Healthy scale is typically 32768.0

Z Image Turbo

• Base model: z_image_turbo_1.0_i8x.ckpt
• The validated training baseline is the generic command above.
• The validated trainer scale is 1024.0
• For generation validation, use cfg = 1

Z Image Base

• Base model: z_image_1.0_q8p.ckpt
• The validated training baseline is the generic command above.
• The validated trainer scale is 1024.0
• For generation validation, keep the model’s recommended Base path:

{"sampler":17,"shift":1.8776105999999999,"resolutionDependentShift":true}

• A validated comparison used cfg = 4

Qwen Image BF16

• Base model: qwen_image_2512_bf16_i8x.ckpt
• The validated training baseline is the generic command above.
• Healthy scale is 32768.0
• Use the exact training caption first before trying richer prompts