Scenario Kinetic Music Video

Kinetic Music Video

A song goes in. A 1080p60 music video comes out, landscape for YouTube and X or vertical for TikTok, Reels and Shorts: a motion designer's showreel cut to the track. Kinetic type, 2D and 3D graphics and transitions land on the beats and sung syllables. Generated footage of the subject is woven through it where the concept wants a performer. The approved master is the only soundtrack. Preserve the original; an authorized audio edit produces a new, versioned master.

The work splits by what each tool does best:

• Scenario supplies what code can't: a consistent subject with real physics, choreography, hair and lip-sync, in whatever art style each section needs.
• The JS engine supplies what video models can't: exact typography, frame-exact timing to onsets and beats, infographics, compositing around the subject, and endless re-renders at zero generation cost.
• Sub-agents give each section a dedicated designer. A shared style bible and a verification loop keep them coherent.

Connection and the core generation loop: see the scenario skill in this repo. For a song-to-video cut without the engine, see scenario-seedance-music-video. If a sibling skill named here is missing from your available skills, ask the user to install it (npx skills add scenario-labs/skills --skill <name>); unattended, proceed from tool schemas and flag the gap.

Read references/lessons.md before starting. Every item in it cost real time or money once. references/motion_library.md is the design menu.

Before you start (ask once, then run)

• Workspace: the Scenario team and project (teams_list; ask if there is more than one).
• Subject:
  • Who or what performs: a character from reference images, a mascot, a product, the artist's own likeness (their own photos and consent only), or nobody (a pure motion-graphics or lyric video).
  • Footage share: performer-led, a mix (often about 60/40), or graphics only.
• Look:
  • Ask for references: images, videos or links for the mood, the motion and the color. Vague color asks without references drift to safe palettes.
  • Decide the 2-4 looks and which sections own which.
  • Ask what must never appear.
  • Defaults: generated footage shown clean (no filters on it), graphics layered on and around it, a fast and upbeat showreel energy.
• Brand: any logo, wordmark or credit. Use official files in their official colors, only where the director wants them.
• Budget: lean (6-10 video generations), moderate (12-20) or big. Price each paid step with dry_run (one run per distinct payload, then total) before spending. Relative anchors at 720p: a 6 s clip costs about twenty style frames, a 10 s clip about thirty-five, and a full mixed video about five hundred. A 1080p clip measured about 2.5 times its 720p price, so settle the footage resolution before quoting.
• Format: one canvas per project, asked once: portrait 9:16 for TikTok, Reels or Shorts, landscape 16:9 for YouTube or X (unattended with no destination: landscape). It is CANVAS in engine/timeline.js, and style frames and footage are generated at that ratio, never cropped from the other. On portrait, check the destination's current length cap and plan the teaser as the platform cut when the song runs longer, or when you cannot check. Both ratios only on request: the code re-renders free with --canvas and most shots are windowed, but each shot that must fill the second frame (hook, lip-sync close-ups, finale) is paid again (references/pipeline.md §9). X caps standard accounts at 2:20, so plan a teaser.
• Lyrics: use them if supplied (they beat any transcript). Otherwise transcribe and flag the uncertain words at the end.

With no one around to answer, choose sensible defaults for the creative choices and write them into TREATMENT.md. Do not default the money: with no stated budget or no explicit team and project, stop before the first paid step and say what is missing.

Workflow

1. Scaffold and analyze the song

Run bash <skill>/scripts/scaffold.sh <song> inside a new project folder, then follow references/pipeline.md §2:

• audio_analysis.py writes stems and engine/data/audio.json: beats, downbeats, kick/snare/vocal onsets, and 60 fps envelopes. Measure the tempo; labels lie (a "130 BPM" song was 98 with double-time drums).
• lyrics_align.py transcribe, then hand-correct analysis/lyrics.txt, then lyrics_align.py align writes engine/data/lyrics.json with word onsets.
• Check timing by eye with plot_lyrics.py. Audition when playback is available; a spectrogram locates onsets but cannot prove pronunciation or musical continuity.

1a. Edit the soundtrack when requested

For pronunciation repairs, lyric changes, section replacements or structural cuts, read references/audio_editing.md. Use scenario-ace-step for its Cover/Repaint contracts. Keep the current cut and master until a candidate is selected; preserve audio outside the agreed window and rebuild affected timing data after applying it. A picture-only revision reuses the approved soundtrack.

2. Research (parallel, background)

Launch two sub-agents with web access while you work on other things:

• Culture: for the song's subject, find recognizable events, memes and visual tropes, each with a verified fact, its iconic visual form, an insert idea and a short on-screen string.
• Craft: study the user's motion references (download them, contact-sheet them, read the frames), plus genre direction techniques. Extract timing numbers, color hex values and implementable JS moves.

Save the condensed results as analysis/research_*.md. Scene agents read them alongside references/motion_library.md.

3. Concept, style bible and treatment

• Give the video a spine. The strongest concepts make the edit itself carry the song's meaning: a device that progresses through the whole video, for example a counter, clock or countdown that moves with the song while the cut rate or the looks escalate with it. The spine usually lives in the HUD counters.
• STYLE.md (the law):
  • a one-line concept and the spine
  • the footage rules (clean plate and the back/mid/front layers, or another policy)
  • the looks, each with ground, ink, accent, rendering rule and matching footage style
  • the palette with roles
  • 4-5 type roles with named fonts
  • lyric display rules
  • a motion grammar mapping beats to effects, with the cut density per section
  • the subject (identity, design details usable as graphic motifs)
  • brand rules, references and bans
• TREATMENT.md:
  • song facts and the energy map
  • the arc
  • the hook for the first 3 seconds (the strongest idea in the video, and it must work as a muted autoplay)
  • one row per section: time range on downbeats, lyrics, look, must-have ideas
  • a footage map (filled in as clips land)
• PLAN.md: a one-page version for the director to approve before you spend on footage (unattended, this is the stop point if the budget is unstated).

4. Subject, style frames and footage on Scenario

Skip this for a graphics-only video, apart from any stills or 3D props. Otherwise follow references/pipeline.md §3-5:

• Subject sheet: a turnaround, expressions and callouts from the user's references. Look at it, then crop identity references from it.
• Style frames: one still per shot at the canvas ratio (an image model that takes reference images), in its section's look. Compose with negative space for type (on portrait, the face and the type space inside the safe band), on a set that composites (black void with rim light, flat color cyc, or white high-key). No text. In a lip-sync frame nothing covers the mouth (no mic, hand or prop): a covered mouth fails the sync check and is paid again. Contact-sheet them and get the look approved cheaply (unattended: proceed only inside a stated budget).
• Footage routes:
  • Image-to-video from the frame for actions.
  • A video model with reference images and reference audio, given a synthesized beat track for dances that lock to the song, then beat-warp.
  • Audio-to-video from a frame plus a vocal-stem slice for lip-sync. The alternative is reference-audio video plus a lip-sync correction model.
  • Code instead of generation for networks, charts, UI and particles.
• Never send the song to a video model. Turn the model's native audio off when its schema offers it (the render takes sound only from the master either way), and launch everything in parallel (jobs_wait re-called with pending_job_ids, never job_get, never a relaunch).
• Prep every clip with tools/prep_clip.sh: frames, a subject matte (Apple Vision, with Scenario background removal as the fallback; see references/pipeline.md §5) and tracking (MediaPipe pose plus the matte bbox). Inspect the mattes. Write each clip's content, key moments, slot and matte quality into the footage map.