Synthetic Screen Recording

Synthetic Screen Recording (Remotion TerminalScene)

Decision this skill answers: When the user wants a screen-recording-looking demo of a terminal, CLI tool, or coding workflow — do I capture the real desktop (OS screen recording via screen_recorder, Windows-MCP, Cap, or Playwright), or do I synthesize it in Remotion with the TerminalScene component?

Heuristic: If the agent can author the exact command/output sequence in advance, synthesize. Only capture live when the real behavior is unpredictable, needs a real app UI, or the user explicitly asked for a real recording.

Why this exists

v3 of the OpenMontage showcase tried to use Windows-MCP + screen_recorder to drive a Git-Bash window for the install walkthrough. It stalled on window positioning, focus races, and taskbar privacy concerns. We pivoted to pure Remotion rendering — a React component named TerminalScene that draws a fake terminal and types commands character-by-character. The output is visually indistinguishable from a real screen recording (same traffic-light window chrome, blinking cursor, scrolling output) but deterministic, privacy-safe, pixel-perfect at 1080p, and pace-controllable to the frame.

That component + pattern is the capability this skill makes discoverable.

When to use synthetic (TerminalScene)

YES, synthesize when:

• The demo is a terminal / CLI / coding session where commands and outputs are predictable
• The user wants a polished tutorial feel (clean typography, floating pills, cursor blink)
• Install walkthroughs, setup demos, API key config, make targets, git clone flows
• You need tight sync with narration — every command must land on a specific beat
• You want the result reproducible (re-render gets identical pixels)
• The user's actual desktop has private apps/windows visible you'd otherwise have to crop

NO, capture a real screen when:

• The demo is a real app UI that can't be faked (Figma, Photoshop, a web app with live state, a browser flow)
• The user explicitly asked for a recording of their actual screen
• The behavior depends on timing you can't script (streaming LLM output, real network latency)
• There's a visual quirk (a cursor effect, a plugin pop-up) that only appears in the live environment

For a browser demo → playwright-recording skill, not this one. For a real desktop → screen_recorder tool or Cap via cap_recorder.

The component — TerminalScene

Located at: remotion-composer/src/components/TerminalScene.tsx Exported from: remotion-composer/src/components/index.ts Wired in dispatch: remotion-composer/src/Explainer.tsx (if (cut.type === "terminal_scene"))

Props:

interface TerminalSceneProps {
  title?: string;           // shown in the window title bar
  steps: TerminalStep[];    // the timeline
  prompt?: string;          // "$", ">", etc.
  accentColor?: string;     // pill + prompt glow
  backgroundColor?: string;
}

Step kinds:

{ kind: "cmd",   text: string, typeSpeed?: number, holdSeconds?: number }
{ kind: "out",   text: string, holdSeconds?: number }
{ kind: "pause", seconds: number }
{ kind: "pill",  text: string, color?: string, durationSeconds?: number }

• cmd — prints the prompt, types the text character-by-character (typeSpeed is seconds per character, default 0.035), then holds for holdSeconds (default 0.3)
• out — a line of program output, reveals instantly with a short fade-in
• pause — dead time. Terminal holds on last visible state. USE THIS TO SYNC WITH NARRATION.
• pill — non-blocking floating badge (top-right). Spring-in, hold, spring-out. Does NOT advance the cursor — the next step runs in parallel.

Authoring pattern

Author a new scene by adding a cut to build_composition.py (or your equivalent props builder):

install_steps = [
    {"kind": "pause", "seconds": 7.0},                 # wait for intro narration
    {"kind": "cmd", "text": "git clone https://github.com/calesthio/OpenMontage.git",
     "typeSpeed": 0.045, "holdSeconds": 0.3},
    {"kind": "out", "text": "Cloning into 'OpenMontage'..."},
    {"kind": "out", "text": "remote: Enumerating objects: 2847, done."},
    {"kind": "pill", "text": "repo cloned", "color": "#34D399", "durationSeconds": 2.6},
    {"kind": "pause", "seconds": 3.8},                 # bridge to next narration cue
    # ...
]

cuts.append({
    "id": "install-terminal",
    "type": "terminal_scene",
    "terminalTitle": "bash — OpenMontage setup",
    "prompt": "$",
    "accentColor": "#22D3EE",
    "steps": install_steps,
    "in_seconds": 50.0,
    "out_seconds": 110.0,
})

THE RULE: pace with narration, never ahead

The #1 failure mode: steps run continuously and burn through all content in the first 40% of the scene, leaving the terminal frozen for the remaining 60%. This is what killed the v3 first pass — the capability menu rendered at t=80s but narration didn't announce it until t=92s.

Do this instead:

1. Know your narration cues — for each scene, write down the exact video-time each narration segment starts.
2. Start with a pause that reaches the first narration cue before any command types.
3. Time each command to land with its narration line — cmd should start typing the moment narration says its line, not before.
4. Put pauses between command groups that bridge to the next narration cue.
5. End with a closer hold — a pause long enough that the final state is readable after narration ends.

Sanity-check your steps before rendering — every minute of Remotion render is precious. Sum the step durations and verify they equal scene duration:

import math
def trace(steps, scene_start, fps=30):
    t = 0.0
    for s in steps:
        k = s["kind"]
        if k == "cmd":
            tf = math.ceil(len(s["text"]) * s.get("typeSpeed", 0.035) * fps)
            t += tf / fps + s.get("holdSeconds", 0.3)
        elif k == "out":
            t += max(2, math.ceil(0.08 * fps)) / fps + s.get("holdSeconds", 0.15)
        elif k == "pause":
            t += s["seconds"]
        # "pill" is non-blocking — does NOT advance cursor
        print(f"  {t + scene_start:6.2f}s  {k}: {s.get('text', '')[:40]}")
trace(install_steps, 50)

Look at the output column. Each narration cue's video-time must appear adjacent to the command/output it announces. If a command lands 10s before or after its cue, adjust pauses.

See lib/verify_scene_pacing.py for a reusable version of this script.