Generation Request Flow

Generation Request Lifecycle & Param Plumbing

This skill maps the end-to-end path of a generation request in HOT-Step CPP and catalogs every point where a parameter can be silently lost. Most historical bugs in this area are param-plumbing bugs: a value set in the UI never reaches the C++ sampler, and nothing errors.

Terms used below:

• DiT — the diffusion transformer that turns audio codes + text embeddings into audio latents (the "synth" phase). Pipeline is LM → DiT → VAE.
• LM — the language model phase that generates metadata (bpm, duration, key), a chain-of-thought (CoT) caption, lyrics, and audio_codes for the DiT.
• aceReq — the Node-side request object (interface AceRequest, server/src/services/aceClient.ts:38) sent to the C++ engine. Snake_case, produced from camelCase UI params by translateParams().
• Sideband — HOT-Step-only fields (solver, scheduler, guidance, adapter knobs, …) that travel alongside the upstream C++ AceRequest struct in the same JSON body but are parsed separately into ServerFields (engine) because they don't exist in the upstream struct.

When to use this skill

• Adding a new generation parameter (UI knob → engine behavior).
• Debugging: a param is set in the UI but the engine logs/behavior show the default.
• Modifying server/src/routes/generate.ts, translateParams.ts, aceClient.ts, or engine/tools/hot-step-server.cpp request handling.
• Understanding generation modes (text2music, cover, repaint, lego, …) or the job queue/watchdogs.

Golden rules

1. NEVER rebuild the post-LM synth request by copying the LM echo or by whitelisting fields to carry across. The engine's /lm response only contains fields in the C++ AceRequest struct — every sideband field vanishes. Always rebuild as { ...aceReq, <the 7 LM output fields> }. WHY: every field-copy whitelist ever written here went stale and silently dropped params; commits 8ea519b and 168dcb5 fixed real bugs of exactly this class. Full detail in the next section.
2. When adding a float field to engine JSON parsing, use the int-safe get_num lambda, not bare yyjson_get_real(). WHY: yyjson_get_real() returns 0 for JSON integers, and JavaScript serializes 2.0 as 2 — so a whole-number value silently becomes 0.0. See hot-step-server.cpp:700-705.
3. C++ changes rebuild ONLY via .\dev-rebuild.bat at repo root — never engine\build.cmd directly, under any circumstances (you cannot reliably tell whether the app is running; Node auto-respawns ace-server on crash → infinite respawn + file-lock loop). Never cmake --build . --clean-first (20+ min CUDA recompile).
4. Type-check TypeScript with npx tsc --noEmit; do not npm run build during dev.
5. After any upstream sync, run powershell -File engine\verify-hooks.ps1. WHY: pipeline-synth-ops.cpp losing its hot-step-sampler.h include is SILENT — everything compiles but all solvers/schedulers/guidance/sideband go dead (they read g_hotstep_params, which nothing consumes anymore).
6. Git: all work on master, stage explicit paths (never git add -A), push only with explicit user approval. Any pushed v* tag triggers a full multi-platform CI release build.
7. Params consumed as GPU tensors inside hot-step-sampler.h (masks, encoder states, constants) must be re-uploaded EVERY step — the GGML scheduler aliases input tensor buffers as scratch space; uploading once leaves them garbage from step 2 onward (hot-step-sampler.h:591-601, re-upload sites :936, :1091). Never hoist an upload out of the step loop as an "optimization".
8. Never delete generated audio artifacts (server/data/audio/*, test outputs) based on your own judgment of quality — the user verifies results by ear.
9. Node 20 to 24 LTS, 24 recommended — engines enforces <25.

THE LM ECHO SIDEBAND GOTCHA (read this before touching generate.ts)

Ground truth from the departing lead engineer, verified line-by-line in code:

Server-side params (ServerFields) do not survive the /lm round trip — the engine echoes back only what it knows. When building the synth request after the LM step, rebuild it from the original aceReq plus the LM-returned fields ONLY. Never maintain a whitelist of fields to copy across — every whitelist has gone stale and silently dropped params. Commits 8ea519b and 168dcb5 fixed bugs of exactly this class.

Mechanics of why:

• /lm's result body is request_to_json() serialized from the C++ AceRequest struct (hot-step-server.cpp:869, struct at engine/src/request.h:29).
• Fields that exist only in ServerFields (hot-step-server.cpp:557-604) — vae_model, emb_model, scheduler, guidance_mode, adapter_group_scales, adapter_mode, adapter_runtime_quant, rebase_source/rebase_beta, apg_*, stork_substeps, denoise_*, plugin_params, seed_strength, evict_lm, vae_chunk, batch_cfg, and more — are not in the C++ struct, so the LM echo cannot contain them. Copy the echo forward and they all silently drop.

The correct pattern — it exists in exactly two places, keep them identical:

// generate.ts:319-328 (fresh LM) and generate.ts:237-246 (LM-cache hit)
lmResults = lmResults.map(lmOut => ({
  ...aceReq,                       // CURRENT request = every sideband field survives
  audio_codes: lmOut.audio_codes,  // then ONLY the 7 LM-generated fields:
  caption: lmOut.caption,
  lyrics: lmOut.lyrics,
  bpm: lmOut.bpm,
  duration: lmOut.duration,
  keyscale: lmOut.keyscale,
  timesignature: lmOut.timesignature,
}));

The only whitelist allowed is the LM-output list (those 7 fields). The base must always be a spread of the live aceReq. If you add a new server-side param you do not touch this rebuild code — that is the whole point of the pattern.

The pipeline at a glance

UI form (CreatePanel + Zustand globalParamsStore)
  → GenerationParams JSON, camelCase          [ui/src/types.ts:80]
  → POST /api/generate (Node :3001)           [server/src/routes/generate.ts:1355]
  → in-memory job Map + strictly serial queue [generate.ts:44 statuses]
  → translateParams(): camelCase → snake_case [server/src/services/generation/translateParams.ts:11]
  → POST http://:8085/lm (unless skipped)     [aceClient.ts:319; engine handle_lm hot-step-server.cpp:888]
  → REBUILD synth req: {...aceReq, 7 LM fields} [generate.ts:319-328]  ← THE SIDEBAND GOTCHA
  → POST http://:8085/synth (JSON or multipart) [aceClient.ts:332 / :347; engine handle_synth :1442]
  → engine: parse_server_fields() + request_parse_json() on the SAME body [hot-step-server.cpp:606]
  → synth_worker: ServerFields → g_hotstep_params global [hot-step-server.cpp:949, copy at :1132-1194]
  → synth_batch_run: TextEnc → DiT (hot-step-sampler.h reads g_hotstep_params) → VAE → WAV/MP3
  → Node saves data/audio/<uuid>.wav|mp3      [generate.ts:840]
  → auto-trim → post-processing chain → INSERT INTO songs [generate.ts:1057-1094, :1152]

Engine port 8085 is set in server/src/config.ts:102 (ACESTEPCPP_PORT). Node job statuses: pending | lm_running | synth_running | saving | succeeded | failed | cancelled (generate.ts:44).

Procedure: adding a new generation parameter end-to-end

A new param must be threaded through five layers. Missing any one = silent default.

1. UI state: add to ui/src/stores/globalParamsStore.ts (persisted per-field to localStorage under an hs-* key) and include it in getGlobalParams() (line 324). Watch the conditional gating there — many params are sent only when their mode is active (e.g. apgMomentum only when guidanceMode === 'apg', line 396). Add the field to GenerationParams in ui/src/types.ts:80. The visible controls for engine knobs live in ui/src/components/global-bar/ (GlobalParamBar.tsx, GenerationDropdown.tsx) — grep an existing knob's setter to find the right spot.
2. Node translate: map camelCase → snake_case in server/src/services/generation/translateParams.ts. Use !== undefined guards for fields where 0/false is meaningful — a truthy guard (if (params.foo)) silently drops legitimate zeros.
3. Node wire type: add the snake_case field to the AceRequest interface in server/src/services/aceClient.ts:38 (this interface is the Node-side canonical superset: C++ struct fields + sideband).
4. Engine parse: add to ServerFields (hot-step-server.cpp:557) and parse_server_fields() (:606). For floats, use the get_num lambda pattern — Golden rule 2. Note: get_num is declared mid-function at :703 — add your parse AFTER that line (e.g. next to the denoise fields at :763), or inline the is_real ? get_real : get_int ternary like rebase_beta does (:663); a parse placed above :703 fails to compile (use before declaration).
5. Engine global: copy into g_hotstep_params in the block at hot-step-server.cpp:1132-1194 (note: this copy happens before ace_synth_load() because adapter merge reads group scales at merge time), and add the field to HotStepParams in engine/src/hot-step-params.h. Consume it in hot-step-sampler.h (or wherever).
6. Do NOT touch the LM-echo rebuild in generate.ts — the ...aceReq spread carries your field automatically.
7. Rebuild: .\dev-rebuild.bat (C++), npx tsc --noEmit (TS). Verify arrival — but beware the [DIAG] trap: the [DIAG] /synth body (first 300 chars) line (hot-step-server.cpp:1568) truncates at 300 chars, and the body starts with caption then lyrics, so with any realistic caption your sideband field will NOT appear in the visible prefix. A field absent from the DIAG line is NOT evidence it was dropped. Reliable checks: add a temporary fprintf of your parsed value in parse_server_fields (or next to the [Server] HOT-Step params: summary at hot-step-server.cpp:1197), or log the full aceReq server-side before submitSynth. For any new UI control, ask the user to visually confirm it — do not use the browser agent for visual verification (API checks are fine).

Key files