Talking Head Recut

Plugin installs: Before setup or freshness commands, follow plugin execution rules when this skill is inside a HyperFrames plugin. Standalone installs keep the update instructions below.

First, keep this skill fresh — confirm with the user before running: npx hyperframes skills update talking-head-recut. A fast no-op when everything is current; otherwise it refreshes this skill plus the core domain skills it depends on before you rely on them.

Talking Head Recut

Talking Head Recut takes a local video that plays in full and layers a sequence of timed, designed graphic cards onto it — titles, lower-thirds, data callouts, quotes, side panels, picture-in-picture — synced to what's being said. The agent designs the cards (timing + content) and writes each card's HTML directly in the conversation, then assembles a single composition HTML and renders it to MP4 via hyperframes. There is no fixed archetype list and no prescribed card structure — the overlays emerge from what the transcript actually says.

The front door is /hyperframes. This skill packages an existing talking-head clip with designed graphic cards (titles, lower-thirds, data callouts, quotes, side panels, PiP) — not plain captions (the spoken words as text). The clip plays untouched. Any other intent — plain subtitles, a standalone graphic, a from-scratch video — or any uncertainty → read /hyperframes first: the intent layer owns every route decision.

Graphic-packaging sibling of embedded-captions. Captions add the spoken words as a readable subtitle; this adds designed graphics on top of the playing video. Plain subtitles → embedded-captions. Build a video from scratch → the creation workflows (product-launch-video / faceless-explainer / …).

Routed through /hyperframes, the intent layer confirms only the input (which clip) and announces the render-strategy questions as deferred asks — aspect, layout, style group, and card count stay at Step 7, where the probed footage and transcript ground the recommendations; the layer's run-shape questions don't apply. A BRIEF.md, when present, carries the confirmed input and any user notes — read it first.

Inspectable intermediate files in the work directory:

• metadata.json — duration / width / height / fps
• audio.mp3 — extracted audio
• transcript.json — a flat word array [{ text, start, end }, …] (Whisper; no segments, no words wrapper)
• storyboard.json — lightweight card outline (the agent's plan)
• public/cards/card-XX.html — one HTML fragment per card
• public/index.html — final assembled composition
• output.mp4 — rendered video

CLI Resolution

# hyperframes — transcription (local Whisper) + rendering the assembled HTML to MP4
npx hyperframes --help

This skill runs entirely on the hyperframes CLI plus system ffmpeg / ffprobe. Transcription is local Whisper via hyperframes transcribe — no third-party service, API key, or rate-limited proxy.

Workflow

1. Check Environment

npx hyperframes doctor          # ffmpeg, headless browser, render deps
# confirm bundled assets:
ls "<SKILL_DIR>/assets/fonts" "<SKILL_DIR>/assets/vendor/gsap.min.js"

Required:

• ffmpeg / ffprobe (system)
• <SKILL_DIR>/assets/fonts/*.woff2, <SKILL_DIR>/assets/vendor/gsap.min.js (bundled inside this skill, staged to work dir in Step 9)

Transcription needs no key — hyperframes transcribe runs Whisper locally (Step 4).

Strongly recommended on macOS for hyperframes render:

export PRODUCER_BROWSER_GPU_MODE=hardware

2. Create a Work Directory

All artifacts live under videos/<project-name>/ — the same convention as the other video workflows (product-launch-video / faceless-explainer / pr-to-video). Keep the cwd at the workspace root; everything below writes under this one subdirectory.

VIDEO_PATH="/absolute/path/input.mp4"
WORK_DIR="videos/$(basename "$VIDEO_PATH" | sed 's/\.[^.]*$//')"
mkdir -p "$WORK_DIR"

3. Extract Audio and Metadata

# metadata — duration / width / height / fps
ffprobe -v error -select_streams v:0 \
  -show_entries stream=width,height,r_frame_rate \
  -show_entries format=duration -of json "$VIDEO_PATH" > "$WORK_DIR/metadata.json"
# audio
ffmpeg -y -i "$VIDEO_PATH" -vn -acodec libmp3lame -q:a 2 "$WORK_DIR/audio.mp3"

Outputs: metadata.json (read width/height/duration; fps = the r_frame_rate fraction evaluated, e.g. 30000/1001 → 29.97) + audio.mp3.

4. Transcribe

npx hyperframes transcribe "$WORK_DIR/audio.mp3" -d "$WORK_DIR" --json --model small.en

Local Whisper — no API key, no proxy, no rate limit. Writes a word-level transcript.json into the work dir (word text + start / end timestamps). Read it for the word / sentence timings that drive card timing in Step 6; group words into sentences yourself at punctuation / pauses if you need segment-level chunks.

Clamp to media duration. Whisper can return the final word's end a hair past the actual clip length — clamp every card endSec and composition.durationSeconds to the metadata.json duration, or the render will show a black tail past the video.

5. Correct Transcript

transcript.json is a flat array of word objects — [{ "text": "...", "start": s, "end": s }, …] (no segments array, no words wrapper; the per-word key is text). Read it and fix obvious ASR errors:

• Homophones, product names, technical terms, punctuation
• Edit a word's text in place; preserve its start / end timestamps
• There is no pre-grouped segments array — group words into sentences yourself (split at terminal punctuation / pauses) when you need segment-level chunks for card timing

6. Draft a Lightweight Storyboard (in chat)

No CLI involved. Read transcript.json + metadata.json and design cards directly. storyboard.json is an agent-internal planning artifact — no CLI command consumes it; it exists so you can think clearly about timing and content before writing each card's HTML. Keep the shape consistent with the example below so the same outline can drive the composition you author in Step 9:

{
  "schemaVersion": 3,
  "composition": {
    "fps": 30,
    "width": 1080,
    "height": 1920,
    "durationSeconds": 121.2,
    "layout": "portrait",
    "themeId": "noir",
    "seed": 42
  },
  "videoTrack": {
    "sourcePath": "input-video.mp4",
    "startSec": 0,
    "endSec": 121.2,
    "bounds": { "x": 0, "y": 0, "width": 1080, "height": 1920 }
  },
  "subtitles": { "enabled": false },
  "cards": [
    {
      "id": "card-01",
      "intent": "Hook with the speaker's anxious midnight question",
      "startSec": 0.5,
      "endSec": 13.0,
      "accentIndex": 0,
      "zone": "fullscreen",
      "contentHints": {
        "kicker": "AN HONEST QUESTION",
        "title": "The soul-searching question at 11 PM",
        "detail": "Client's 60-second voice message: 'If the RMB appreciates, does that mean my USD policy is a terrible loss?'"
      }
    }
  ]
}

Required Card fields:

Five zone values:

When you assemble the composition in Step 9, resolve each card's zone into pixel bounds on the card-host wrapper following the table above. Video bounds are set once at composition level (videoTrack.bounds); to make video appear to "move between cards", author GSAP tweens against #video-wrap in the composition's <script> (see Step 9).

No prescribed card roles, no prescribed narrative arc. Cards emerge from what the video actually says — could be all quotes or all data, could open with a number or with a story. Let the transcript drive the rhythm.

How many takeaways? — auto-infer from duration + density. No fixed upper limit. Pick a base pace from the video duration, then adjust by information density. Only floor is fixed: minimum 5 cards so even short videos have rhythm.

Step 1 — base pace by duration (the natural sec/card for medium density):

Step 2 — density multiplier (multiplies the base pace):

Step 3 — compute:

secPerCard = basePace × densityMultiplier
cardCount  = max(5, round(videoDurationSec / secPerCard))

Examples (notice — no upper clamp; long videos naturally produce more cards):

• 30s reel, single punchline (low density) → 7 × 1.5 = 10.5s/card → round(30/10.5)=3 → floor to 5 cards
• 60s reflective monologue (low density) → 10 × 1.5 = 15s/card → 4 → floor to 5 cards
• 121s talking-head with rich data (high density) → 10 × 0.7 = 7s/card → 17 cards
• 5 min interview, mixed density → 16 × 1.0 = 16s/card → 19 cards
• 10 min deep-dive, high density → 16 × 0.7 = 11s/card → 55 cards
• 30 min lecture, medium density → 28 × 1.0 = 28s/card → 64 cards
• 1 hr podcast, low density → 45 × 1.5 = 67.5s/card → 53 cards

When a card holds longer than ~15s, plan for a richer card (data block, multi-step reveal, several sub-points unfolding with staggered animations) — a static one-liner gets boring past 8s. For long pieces where many cards exceed 30s, consider chunking the timeline into sub-compositions (one .html per chapter, mounted with data-composition-src) so the GSAP timeline per file stays manageable — see the timeline_track_too_dense HyperFrames lint warning.

contentHints is a free-form object; its keys and nesting can take whatever shape captures the data. The agent decides the shape per card.

Optional outro. This skill ships no fixed brand outro. If the user wants a closing card, design a neutral one yourself (wordmark + one-line tagline, ~1.5-2s, fade in -> short hold -> fade out), append it to cards[], and extend composition.durationSeconds to its endSec. Otherwise end on the last content card.

7. Decide Render Strategy

Confirm Visual Direction with User (DO THIS FIRST)

Before you start designing cards or deciding bounds, ask the user to pick the output ratio, the layout, the style, and the card-density preset. Frames are auto-selected from the chosen layout × style combination (see "Auto-pick frame" table below). Before sending the question, precompute two things:

1. recommendedRatio from the source video's aspect ratio (metadata.json width / height):

   • sourceAspect = width / height
   • sourceAspect ≥ 1.5 (≥ ~3:2 wide) → recommend 16:9
   • sourceAspect ≤ 0.7 (≤ ~9:13 tall) → recommend 9:16
   • 0.7 < sourceAspect < 1.5 (near-square) → recommend 4:5

   Mark the recommended option's label with " (recommended · matches source video X:Y)" so the user sees why it's recommended.

2. autoCount from Step 6 (max(5, round(videoSec / (basePace × densityMultiplier)))) so the "auto" option's label can show the concrete number.

Environment compatibility — pick the best available question channel. Not every runtime exposes the same structured-question tool. Apply this order:

1. Native clarification tool — use the structured 4-question call below.
2. Other native clarification tool (e.g. ask_question, request_user_input, IDE-specific prompt) — use that tool with the same 4 question texts and option lists. Preserve the recommendation markers and the precomputed values.
3. No native tool (Codex CLI, plain text-only runtimes) — ask directly in normal conversation. Use the plain-text template at the end of this section. Keep it to one message, 4 numbered questions (the global cap is 2–5 questions per round; we stay inside it).

Rules that apply to every channel:

• Ask at most 2–5 questions per round. Our 4 here fits.
• Even if missing info doesn't block rendering, ask once to confirm the parameters that materially affect the final output (ratio, layout, style, cardCount).
• If the user has already pre-approved defaults ("just use defaults", "no need to ask", "auto-pick everything"), asked you not to ask, or the run carries an ongoing autonomous signal ("surprise me" / "decide for me" — ../hyperframes/references/brief-contract.md § 1) — skip the question entirely and use: recommendedRatio, layout="stack" (safest cross-ratio default), style chosen from transcript tone in the most neutral group (editorial/data), autoCount. Tell the user what you picked in one sentence and continue.

Channel A — native AskUserQuestion:

// Precompute before the call:
//   recommendedRatio = "16:9" | "9:16" | "4:5"
//   autoCount        = integer (from Step 6)

AskUserQuestion({
  questions: [
    {
      question: "Output video aspect ratio (canvas):",
      header: "Aspect ratio",
      multiSelect: false,
      // Reorder so the recommended option appears FIRST (per AskUserQuestion convention).
      // Append " (recommended · matches source video W×H)" to the recommended option's label.
      options: [
        { label: "16:9 (1920×1080) landscape", description: "TV / YouTube / desktop playback. Most natural when the source video is already landscape; widest canvas." },
        { label: "9:16 (1080×1920) portrait", description: "TikTok / Reels / short-form mobile. Most natural for portrait source; native mobile experience." },
        { label: "4:5 (1080×1350) near-portrait", description: "Instagram feed / WeChat Moments. Best when source is near-square or you want to cover both platforms." }
      ]
    },
    {
      question: "Choose the overall layout: how should the video and cards coexist on the canvas?",
      header: "Layout",
      multiSelect: false,
      options: [
        { label: "side-by-side (split)",  description: "Video and card each take half the canvas. Most stable for interview / data side-by-side; clear visual separation." },
        { label: "top-bottom (stack)",    description: "Video on top (~52%), card below. Classic combo of speaker face + summary card; works well in portrait too." },
        { label: "picture-in-picture (pip)", description: "Card fills the canvas, video shrinks to a rounded corner window. Use when content is primary and speaker is secondary." },
        { label: "full-screen overlay (overlay)", description: "Video plays full-bleed, card floats as a glass layer on top. Strong cinematic / emotional feel." }
      ]
    },
    {
      question: "Choose the card visual style (style):",
      header: "Style group",
      multiSelect: false,
      // NOTE: these 3 groups intentionally match the frame auto-pick matrix
      // rows below, so picking a group resolves both `style` group AND the
      // frame matrix column in one step. Memberships are mutually exclusive.
      options: [
        { label: "warm paper (warm-paper)", description: "academic notebook · editorial big-type · whiteboard hand-drawn · xhs social. Best for interview reflections, product launches, lifestyle, emotional stories." },
        { label: "clinical / cold (clinical)",   description: "audit magazine · swiss grid · terminal CLI · minimal modern. Best for financial analysis, investigative reports, technical tutorials, serious presentations." },
        { label: "experimental / avant-garde (experimental)", description: "geom color-clash geometry · spotlight dark-background. Best for short-form highlights, product launches, strong emotion, cinematic feel." }
      ]
    },
    {
      question: "Card count (takeaway pacing): how many cards to cut?",
      header: "Card count",
      multiSelect: false,
      options: [
        { label: "Auto (recommended) · approx N cards", description: "Inferred automatically from video duration and information density (see Step 6 rules). This run estimates approx N cards. Substitute the real N (your autoCount) into the label." },
        { label: "Fewer · approx round(N × 0.6) cards", description: "Sparser cuts, each card holds longer — suits reflective / slow-paced content." },
        { label: "More · approx round(N × 1.5) cards", description: "Tighter cuts, faster rhythm — suits staccato / data-dense / short-form highlight content." }
      ]
    }
  ]
})

About "Other" — AskUserQuestion automatically adds an "Other" option to the card count question. The user can type a number directly (e.g. "8", "20") as the cardCount target. Parse the input as an integer: if parsing succeeds → use that value (minimum 5 as a floor); if parsing fails → fall back to "auto".

Channel B — plain-text fallback (Codex CLI, runtimes without a native question tool). Post this as one normal message, then wait for the reply. Bullet-style 1/2/3/4 keeps the reply parseable:

I need to confirm four visual decisions with you before I start cutting cards:

1) Output aspect ratio (canvas):
   A. 16:9 landscape (1920×1080) — TV / YouTube / desktop playback
   B. 9:16 portrait (1080×1920) — TikTok / Reels / short-form mobile
   C. 4:5 near-portrait (1080×1350) — Instagram feed / works for both platforms
   ▸ My recommendation:  <recommendedRatio>  (matches source video W×H = <sourceW>×<sourceH>)

2) Overall layout (how video & card coexist):
   A. split   side-by-side (50/50)
   B. stack   top-bottom (video top, card bottom)
   C. pip     picture-in-picture (card full canvas, video rounded corner window)
   D. overlay full-screen glass overlay (video full-bleed, card glass layer)

3) Card style group (maps to frame auto-pick matrix, pick 1 of 3):
   A. warm paper (warm-paper)      (academic / editorial / whiteboard / xhs)
   B. clinical / cold (clinical)   (audit / swiss / terminal / minimal)
   C. experimental (experimental)  (geom / spotlight)

4) Card count (takeaway pacing):
   A. Auto (recommended) — approx <autoCount> cards
   B. Fewer — approx round(<autoCount> × 0.6) cards
   C. More — approx round(<autoCount> × 1.5) cards
   D. Give me a specific number (e.g. "8", "20")

Reply format: "1A 2C 3B 4A" or natural language is fine.
If you want all recommended defaults, reply "default" / "auto" / "use all recommendations".

Parsing the plain-text reply:

• Accept loose formats: "1A 2C 3B 4A", "A C B A", "16:9 / pip / data / auto", full sentences, or default.
• If any answer is ambiguous → re-ask only the ambiguous ones (still inside the 2–5 cap).
• If the user says "default / auto / use all recommendations" → skip without re-asking.

After the user answers (any channel):

1. Resolve the output canvas from the ratio answer — these are the exact storyboard.composition.width / height values to write:

   user choice	composition.width × height	storyboard.layout field
   16:9	1920 × 1080	"landscape"
   9:16	1080 × 1920	"portrait"
   4:5	1080 × 1350	"portrait" (schema treats 4:5 as portrait — height > width)

   For 4:5 bounds inside references/layouts/*.html — those files only document landscape (1920×1080) and portrait (1080×1920). For 4:5 (1080×1350) derive bounds by proportional scaling from portrait: keep horizontal values, scale vertical values by 1350/1920 ≈ 0.703. Example: overlay portrait card = { x: 24, y: 1280, w: 1032, h: 564 } → 4:5 card = { x: 24, y: round(1280 × 0.703), w: 1032, h: round(564 × 0.703) } = { x: 24, y: 900, w: 1032, h: 397 }.

2. Map the style group to a specific style by looking at the transcript tone — pick the one that best fits, but stay inside the user's chosen group. If you're unsure between two specific styles inside the group, send a second AskUserQuestion with those 2–4 specific style options.

3. Resolve final cardCount from the density answer:

   user choice	final cardCount
   Auto (recommended)	the autoCount you already computed
   Fewer	max(5, round(autoCount × 0.6))
   More	round(autoCount × 1.5) (no upper clamp)
   Other = "<n>" (integer)	max(5, parseInt(n))
   Other = anything else	fall back to autoCount

4. Auto-pick the video frame from this table (frames don't ask the user — they follow from layout × style):

   layout	warm-paper styles (academic / whiteboard / editorial / xhs)	clinical styles (audit / swiss / terminal / minimal)	experimental styles (geom / spotlight)
   split	polaroid	hairline	clean
   stack	polaroid	hairline	clean
   pip	clean (pip pill already has chrome)	clean	clean
   overlay	clean (full-bleed forbids deco frames)	clean	clean

5. Tell the user what you chose in one sentence — ratio (+ canvas size), layout, specific style, frame, and final cardCount — then proceed with the rest of Step 7 (per-card layouts, motion patterns).

6. Record the five values (ratio / layout / style / frame / cardCount) in working memory (no schema field needed); you'll reference them while writing each card's HTML in Step 8 and while reading the matching references/<dim>/<key>.html for tokens and structure.

If the user picks an answer via "Other" with a free-text style name not in the 10-style library, treat it as a hint to design a fresh card visual yourself, but still anchor on the chosen layout's bounds.

Render Strategy Inputs

With ratio / layout / style / cardCount / frame locked from Step 7.0, the remaining per-card decisions are:

• Source-video fit inside the GSAP target: video element has object-fit: cover and is clipped to #video-wrap's tween bounds. If you want NO cropping (e.g. portrait source on landscape canvas shouldn't get its top/bottom chopped), aim the tween at a rect that matches the source's aspect ratio and let surrounding canvas show through (or fill with the card / a backdrop).
• card.zone per card: derive from your chosen composition layout (split → side-panel, stack → lower-third, pip → fullscreen, overlay → video-overlay), OR pick a different zone for one-off variants (fullscreen for hero / quote, whiteboard-area for dense data).
• accentIndex per card: each card pulls one of the 5 theme accent colors. Vary across cards for rhythm; reuse the same index when two cards belong to the same narrative beat.
• Motion vocabulary: pick 2–3 repeatable patterns from data-anim kinds (see the table later) and stick to them so the composition feels coherent.

Pick from these themeId palettes (use them as --accent-N / --bg / --text CSS variables in your composition <style> block):

Available fonts (woff2 in <SKILL_DIR>/assets/fonts/, staged to work dir in Step 9): Caveat (handwriting), LXGW WenKai TC (Chinese hand-script), Inter (modern sans), Virgil (geometric hand). Reference via @font-face or font-family directly.

For inspiration on visual patterns, <SKILL_DIR>/references/styles/ ships 10 self-contained reference cards (academic / editorial / minimal / spotlight / geom / whiteboard / audit / terminal / swiss / xhs) that you can copy as starting points — but do not feel constrained to match any of these. Each card is your own design.

Visual Design Library (<SKILL_DIR>/references/)

Beyond the composition-level themeId, the skill ships a richer reference library at <SKILL_DIR>/references/ covering three orthogonal visual dimensions you can freely mix:

Style  ×  Layout  ×  VideoFrame
 (10)      (4)         (3)

Read <SKILL_DIR>/references/DESIGN_INDEX.md for the full matrix and a loose decision guide (interview / product launch / data analysis / social clip / technical tutorial / emotional story …). When you decide to use a specific style / layout / frame, Read the corresponding file:

• references/styles/<key>.html — self-contained card fragment with that style's CSS tokens (colors, fonts, padding, ornament) and a placeholder takeaway. Copy the .card[data-card-id="ref-<key>"] style block, rename the data-card-id to your card's id, swap the placeholder content for the real takeaway, and you're done.
• references/layouts/<key>.html — exact videoBounds + cardBounds for both landscape and portrait. The storyboard records only card.zone.
• references/frames/<key>.html — decorative HTML to add as a sibling of #video-wrap, plus placement instructions for the composition CSS.

Within the user's chosen style group you may vary the style per card, and pick layout × frame per card, as long as the transitions read smoothly.

The 10 styles are skill-side design tokens, not composition-level themes — they don't need to be declared in storyboard.composition; they live inside each card's HTML. The themeId field can still pick a composition-level palette (table above) that controls page-body background and video border chrome.