Release Process

Cutting & publishing a release — HOT-Step CPP

HOT-Step CPP ships as portable archives for Windows (CUDA/Vulkan/CPU), Linux, and macOS. A release is produced entirely by CI: you push a git tag matching v*, the Release workflow (.github/workflows/release.yml) builds every platform variant and creates a draft GitHub Release on scragnog/HOT-Step-CPP. You then verify the draft and publish it with the gh CLI. There are no version numbers to edit in any file — the git tag is the sole version source.

All commands below are Windows PowerShell (use ; to chain, never && in older shells — this repo's convention). gh is already authenticated as scragnog.

When to use this skill

• The user asks to cut, tag, or publish a release (vX.Y.Z).
• The user wants a throwaway CI compile check of the full build matrix.
• A Release or Cache Warm workflow run failed and needs diagnosing.
• You are about to push ANY tag starting with v (read Golden rules first).
• Someone asks "where do I bump the version?" (answer: nowhere — see step 2).

Golden rules (hard constraints)

1. ANY pushed tag matching v* triggers the full multi-platform CI build. The trigger is on: push: tags: ['v*'] (release.yml:14-17) with no other filter. That is ~11 GitHub-hosted jobs including multiple CUDA toolkit installs. Never push a local marker/feature tag that starts with v. WHY: historical junk tags like v1.5-pre-ggml-migration exist locally; pushing one fires the whole pipeline and can pollute the release list.
2. Real releases are plain semver vX.Y.Z — no hyphen. Throwaway builds MUST be hyphenated, conventionally vX.Y.Z-CI-Test. WHY: release-notes anchoring finds the previous release with git describe --tags --abbrev=0 --exclude '*-*' "${TAG}^" (release.yml:1138), so hyphenated tags are skipped by the changelog. A stray non-hyphenated tag between releases would silently truncate the next release's notes.
3. Pushing a tag requires explicit user approval (repo git rule: every push needs approval). Ask before git push origin vX.Y.Z.
4. Do not bump any package.json version. server/package.json and ui/package.json both sit at "version": "1.0.2" while v1.1.2 has shipped — these fields are stale and read by nothing in the release pipeline. There is no root package.json. Editing them achieves nothing.
5. Everything must be committed AND pushed to origin/master before tagging. CI checks out the tag's commit, not your working tree — UNCOMMITTED work will not be in the release. (Committed-but-unpushed work would technically ship via the tag push itself, but push master first anyway so origin/master is never behind a published release tag.) Stage explicit paths only — NEVER git add -A or git add -f (untracked dirs like data/, models/, toinstall/ sit in the tree right now and a blanket add would sweep them in).
6. The release is created as a DRAFT (gh release create --draft, release.yml:1220-1228). Nothing auto-publishes. Publishing (--draft=false) requires explicit user approval, same as a push — present the asset count and notes to the user and wait for their go-ahead. Shipped binaries deserve at least the scrutiny of a push.
7. Never change the Windows runner from windows-2022 (release.yml:34). windows-latest is windows-2025 whose MSVC (_MSC_VER >= 1950) is rejected by CUDA 12.8/13.1 nvcc (host_config.h: "Only the versions between 2017 and 2022 are supported"). The runner image is also baked into the build cache key on purpose.
8. Never tag until check-release-prereqs.mjs passes. A packaged build ships code only; weights and data files reach the user by download or by being inside the archive, and both are easy to forget because the failure is invisible on the dev machine. node server/scripts/check-release-prereqs.mjs (exit 0 required) verifies every catalogue entry exists on Hugging Face at the claimed size in a public repo, that packs reference real file ids, and that every server/src/data/ file is packaged. WHY: v1.3 shipped MM3 training gated on two GGUFs nobody could download (#137) and an MM3 caption corpus CI never copied into the archives (#139). Green CI proves neither.
9. Clean up hyphenated test tags after use (release draft + remote tag + local tag). WHY: leftover drafts clutter the release page, and tag hygiene protects the changelog logic.

Procedure

0. Preconditions

• On master (the only branch used in this repo), working tree clean.
• git status clean; git push of master already done (with user approval).
• If master recently absorbed an upstream acestep.cpp sync, run engine/verify-hooks.ps1 before tagging — the sampler hook can be lost silently (compiles, but all solvers/schedulers/guidance go dead) and nothing in CI or the asset check catches it.
• node server/scripts/check-release-prereqs.mjs exits 0 (golden rule 8). If this cycle added a model, its weights must already be uploaded and in server/src/data/model-registry.json — uploading after the release is live does not help anyone who already downloaded it.
• Pick the version: look at the latest release (gh release list --limit 3) and bump semver appropriately.

1. Optional: throwaway compile test first

Use this to verify CI compiles across all platforms without cutting a real release. Hyphenated tags build everything but are excluded from changelog anchoring.

git tag -a vX.Y.Z-CI-Test -m "compile test"
git push origin vX.Y.Z-CI-Test        # ask the user first — this is a push
# ... monitor (step 3) ...
# cleanup when done:
gh release delete vX.Y.Z-CI-Test --cleanup-tag --yes   # deletes draft + remote tag
git tag -d vX.Y.Z-CI-Test

If the compile test FAILED, no draft exists (the release job needs all three build jobs green), so gh release delete errors with "release not found". Delete the tag directly instead:

git push origin --delete vX.Y.Z-CI-Test; git tag -d vX.Y.Z-CI-Test

Re-pushing the same -CI-Test name after deletion is fine. Tags cannot be renamed — delete and recreate.

2. Version bump — there is none

This is the biggest doc/reality trap. "Cutting a release" is: commit + push master, create tag, push tag. No file edits.

• The tag is the version. release.yml derives the version string from ${{ github.ref_name }} (e.g. release.yml:335, 515) purely for archive naming: HOT-Step-CPP-vX.Y.Z-win-x64-<variant>.zip.
• Engine binaries embed the git short hash + commit date, not semver: engine/tools/version.cmake:29 writes #define ACE_VERSION "<short-hash> (<date>)" into a generated version.h, wired as an always-run CMake target in engine/CMakeLists.txt:9-15. Automatic at build time — never edit it.
• Nothing in server/src checks GitHub for updates, so no in-app version string needs touching either.

3. Cut the release

git tag -a vX.Y.Z -m "vX.Y.Z — <one-line summary>"
git push origin vX.Y.Z                # requires explicit user approval

To rebuild against a different commit: delete the tag remotely and locally, re-create it on the new commit, re-push (the workflow re-runs).

4. Monitor the build

gh run list --limit 5                 # find the Release run for your tag
gh run view <run-id>                  # per-job status + timings

Expected jobs: build-windows × 5 variants (cuda13.1, cuda12.8, cuda12-volta, vulkan, cpu), build-linux × 6 (the same 5 plus rocm), build-macos (Apple Silicon Metal, macos-15), then a final release job that collects artifacts, generates notes, and creates the draft. Warm-cache CUDA jobs run roughly 7–13 min; a cold cache means ~1.5 h per CUDA job (timings from workflow comments, not re-measured — see Failure signatures for the fix).

To read a failed/cancelled job's log while the run is still in progress (gh run view --log won't show it yet), pull it from the API:

gh api repos/scragnog/HOT-Step-CPP/actions/jobs/<job-id>/logs > log.txt

(If you happen to be in Git-Bash instead of PowerShell, prefix with MSYS_NO_PATHCONV=1 so the leading-slash API path isn't mangled.)

Note fail-fast: true on the release build matrices: one variant failing cancels its siblings. Cancelled siblings are not the root cause — find the one that failed.

5. Verify the draft, then publish

gh release view vX.Y.Z --json assets --jq '.assets | length'   # expect 24
gh release view vX.Y.Z --json body --jq .body                  # eyeball notes + download table
gh release edit vX.Y.Z --draft=false --latest                  # publish — ONLY after explicit user approval (Golden rule 6)

Expect 24 assets (12 archives + 12 .sha256). It was 22 until Linux gained a rocm variant; docs/dev/releasing.md still says 18, which predates cuda12-volta as well. Asset names:

• HOT-Step-CPP-vX.Y.Z-win-x64-{cuda13.1,cuda12.8,cuda12-volta,vulkan,cpu}.zip
• HOT-Step-CPP-vX.Y.Z-linux-x64-{same 5, plus rocm}.tar.gz
• HOT-Step-CPP-vX.Y.Z-macOS-arm64.tar.gz
• one .sha256 per archive

If the count is short, do NOT publish — a build job failed or an artifact upload was missed; go back to step 4.

6. Cleanup

Delete any leftover hyphenated test tags and their draft releases (step 1 cleanup commands). Leave nothing matching v*-* on the remote.

Release notes — shaped by commit messages

The release job (release.yml:1132-1218) buckets commits since the previous non-hyphenated tag by conventional-commit prefix on the first line: feat* → "🎵 Features", fix* → "🔧 Fixes", everything else → "📝 Other", then appends a per-file Downloads table and a SHA256 note. Commit message discipline on master directly becomes the release notes — write feat(...)/fix(...) first lines that read well in a changelog.

Key files