Stage Plan

stage-plan

How to turn "here is my material + here's the video I want" into a single, inspectable plan spanning reference images/videos, deterministic or semantic editing, generation, and composition. The output is project/plan.json—a cross-modal EDL that the assembler walks. Ingest with the public ovs analysis commands, validate with ovs plan, and execute through the compose/generate/edit lines.

Use this line when the deliverable is NOT cleanly one axis — e.g. "trim my clip, add a title card and captions, and a voiceover", or "my footage for the middle, generate an opener, compose the stats". For a pure single-axis job, route to that single line instead (see video-router).

Step 1 — Ingest from evidence, never from assumption

You cannot plan against material you have not looked at. For EVERY supplied clip, before writing any segment:

1. Probe it (ovs edit probe) for real duration / resolution / fps / audio presence. A plan that cuts past the real duration breaks.
2. Read its content the cheapest way that fits:
   • spoken audio → ovs transcribe raw/clip.mp4 --out project/transcripts/clip.json (pass --model large-v3 for non-English) → you now have timecoded words to cut on and a reusable transcript file for edit ops.
   • silent / screen-recording / slideshow → extract representative frames with ovs edit extract-frame and inspect them with the current multimodal model. The audio being empty does NOT mean the screen is. If you cannot inspect images, ask the user for the on-screen beats. Never infer slide/screen content from the topic alone.
   • need to judge what a moment LOOKS like (is the hero shot usable? is the product right-side up?) → read frames: ovs edit extract-frame then look at them. If you cannot see images, say so and plan on probe/transcript evidence alone — mark those judgments unverified, do not invent them.
3. Record what each input is good for in project/ingest.json: {input_id, duration, has_audio, content_summary, quality_risks:[...], usable_for:[...], planning_implications:[...]}. This is the factual basis the plan cites — segments reference input_ids from here. Rules:
   • content_summary is specific and from observation: "45 s of interview, no b-roll, mono audio" — never "user provided footage". An entry is only "reviewed" if a real probe, transcript, or inspected extracted frame supports it; never claim you looked at a clip you did not.
   • Usability heuristics: video > 10 s → hero footage; > 3 s → b-roll; has speech → dialogue source; audio-only → narration/music source, production must supply the visuals; image-only → motion must come from animation or generation.
   • Quality risks to flag: width < 720 / height < 480 (will look soft), clip < 3 s (limited use), mono audio, a still where the brief wants motion. A flagged risk the plan ignores is a planning bug — resolve it during direction confirmation.

For every supplied image/video that constrains the result, lock its requested relationship as reproduce, edit, or guide. Record roles, protected attributes, allowed changes, and target segments. Video reproduction/editing or motion/timing guidance requires source-time-to-target-segment anchors.

Step 2 — Choose the delivery promise

Pick ONE delivery_promise.type and make the whole plan keep it:

• source_led — the user's footage is the hero (repurpose / highlight / localize). source_required: true.
• motion_led — real motion (footage or generated video) dominates; composed cards are accents.
• compose_led — designed HTML is the spine (explainer / data); footage/generation are accents.
• hybrid — a deliberate mix (e.g. source hero + composed framing + generated opener).

Set motion_min_ratio to the minimum share of runtime that must be real footage/generated motion rather than composed HTML. For compose_led, use exactly 0; HTML animation quality is enforced by composition motion QA instead of this source-mix ratio. If you cannot hit a nonzero promise from the available material, say so during direction confirmation instead of quietly shipping a slideshow. If source_required is true, at least one PRIMARY segment must be real footage (source: edit | provided) or a semantic video edit that consumes a required top-level original video reference with intent edit and matching target segment and the supplied footage must play in the rendered timeline, not merely appear as a still reference frame.

Step 3 — Decompose into a cross-modal EDL

Write project/plan.json. Every segment declares HOW it is produced (source) and WHERE it sits (layer):

• source: edit (trim a real clip — needs input_id + in_sec/out_sec), generate (needs prompt + explicit media_kind; billable; video signs supported provider settings such as generation_duration_sec, ratio, resolution, generate_audio, and reference_image_urls; omit optional settings the provider should choose; for recurring subjects also set characters, refs, and variation_type), compose (designed HTML — needs a kind), provided (needs asset_id and explicit kind: video|image; still images never count as real motion/source footage).
• layer: primary (the main timeline), overlay (sits over a primary via over: <segment id> — captions, lower-thirds, title cards), bg (behind).
• role: MUST be exactly one of hook / body / proof / cta / transition — the schema rejects any other value (E_SEG_ROLE) and the plan fails validation. Narrative BEAT names from the arc ("payoff", "establishing", "climax", ...) are NOT roles: map a payoff / closing / CTA beat to cta, an establishing / evidence beat to proof. Front-load the hook.

Top-level references is mandatory for every edit/provided source and generation reference. Each entry uses {id,media_type,source,intent,intent_basis,roles,required,preserve,may_change,target_segment_ids,temporal_anchors?}. Omit the field when no references exist; never emit an empty array. Roles are exactly content|identity|composition|structure|style|motion|timing|audio. User requirements win; otherwise guide/inferred is safe. Preserve and may-change must not overlap.

When OVS intelligently selects or transforms content, add edit_strategy with {mode,objectives,decision_signals,preserve,may_change}. Use deterministic for evidence-driven cuts, semantic for AI pixel edits, and mixed for both. A semantic video edit stays an EDIT/AUTO workflow but is a billable source:"generate" video segment with operation:"edit" and at least one declared original in reference_video_paths or reference_video_urls. It requires semantic_model, a matching top-level edit reference and temporal anchor, and inclusion in the paid-generation count.

tracks is required even when the project has no audio or captions; use {} for the empty case. Tracks are separate from the visual timeline. For narration, run ovs speech-capabilities and sign its executable route/model/voice/format together with the BCP-47 video language and speed under tracks.narration.synthesis; raw voice is legacy recovery only. Timed lines are {text,start_sec,target_sec}; target_sec is duration, never end time, and windows must not overlap and each receives a produced_path. Music is path + ducking. Captions stay editable DATA under tracks.captions.lines. Put the exact number of generate segments in cost_estimate.billable_generations; a mismatch is a validation error because paid-generation confirmation reviews that count.

Fit narration in the plan before any TTS call: use natural cadence (about 2.2-2.7 English words/sec or 4-5 Chinese chars/sec), shorten over-budget lines here, and do not rely on repeated synthesis to discover timing.

Author plan.json in EXACTLY this shape (copy the field names — ovs plan validate rejects any other shape):

{
  "aspect": "9:16",
  "total_target_sec": 30,
  "language": "en",
  "delivery_promise": { "type": "hybrid", "source_required": true, "motion_min_ratio": 0.6 },
  "segments": [
    { "id": "s1_hook", "order": 1, "role": "hook", "layer": "primary", "source": "edit",
      "target_sec": 6, "spec": { "input_id": "clipA", "in_sec": 12, "out_sec": 18 } },
    { "id": "s2_body", "order": 2, "role": "body", "layer": "primary", "source": "compose",
      "target_sec": 8, "spec": { "kind": "stat-card" } },
    { "id": "s2_cap", "order": 3, "role": "body", "layer": "overlay", "over": "s2_body",
      "source": "compose", "target_sec": 3, "spec": { "kind": "lower-third" } }
  ],
  "references": [
    {
      "id": "clip-a-source", "media_type": "video", "source": "raw/clipA.mp4",
      "intent": "edit", "intent_basis": "user", "roles": ["content", "timing", "audio"],
      "required": true, "preserve": ["approved content", "audio sync"],
      "may_change": ["signed timeline cuts"], "target_segment_ids": ["s1_hook"],
      "temporal_anchors": [{ "source_start_sec": 12, "source_end_sec": 18, "target_segment_id": "s1_hook" }]
    }
  ],
  "tracks": {
    "narration": { "synthesis": { "route_ref": "openai-compatible", "voice": "nova", "model": "tts-1", "format": "mp3", "language": "en-US", "speed": 1 },
      "segments": [ { "text": "one line of narration", "start_sec": 0, "target_sec": 6 } ] },
    "music": { "path": "assets/bed.mp3", "duck": true },
    "captions": { "style": "bold-bottom", "lines": [ { "text": "one caption line", "start_sec": 0, "target_sec": 3 } ] }
  },
  "cost_estimate": { "billable_generations": 0 }
}

Field gotchas the validator enforces (these are the common breakers):

• source is the production-method enum edit | generate | compose | provided — NOT a file path. The actual clip/asset goes in spec.input_id (edit) or spec.asset_id (provided).
• Every segment needs order + layer + spec; use target_sec (not target_duration_sec/duration). At least one segment must be layer:"primary".
• Every edit/provided source and generation reference needs a matching top-level reference declaration; spec.input_id does not express intent or preservation boundaries.
• operation:"edit" requires reference video input plus edit_strategy.mode:"semantic"|"mixed" and may change only the declared axes.
• tracks is a required object {narration, music, captions} — NOT an array or null. Use {} when no tracks are needed.
• delivery_promise must MATCH this deliverable (Step 2) — do NOT copy the example's hybrid/source_required:true/0.6. A designed-HTML explainer is type:"compose_led", source_required:false, motion_min_ratio:0; set source_required:true ONLY when the user's real footage must star; for other promise types, motion_min_ratio is the real-motion floor you are actually committing to.

Plan to the craft bar (video-craft): a hook in the first seconds, one idea per beat, readable type in safe zones, ducked audio, the right aspect.