Taste Application Video Pipeline

Taste Application

The second half of the pipeline. taste-distillation measures references into a style pack; this generates against that pack and cuts the result.

Execution and Delivery Contract

The original implementation ships here in scripts/; no separate ito-video checkout is required. Install scripts/requirements.txt for local processing. Install scripts/requirements-live.txt only for provider execution. Live uploads, submissions and downloads require explicit TASTE_FORGE_ALLOW_LIVE=1 in addition to credentials; set it only for the user's authorized run. --dry-run remains credential-free and produces labelled placeholders. An ambiguous provider timeout is not retried as a new paid job. Inspect the provider request before deciding whether another submission is warranted.

Use existing completed takes without any provider calls:

python scripts/pipeline.py --genre example --root stylepacks \
  --takes media/take-a.mp4 media/take-b.mp4 --duration 12 --fps 30 \
  --out out/review-v1.mp4

--duration is a best-effort cadence target, not an exact runtime. Complete shots may produce a shorter or longer edit; the assembler does not duplicate clips or add padding to meet the target. The output manifest retains actual duration and adds duration_contract with requested and actual seconds, shortfall, overrun, and the cadence_target policy. Differences of at least one output frame are warned explicitly. No exact-duration mode is provided; when an exact runtime is required, inspect the receipt and revise or reject the cut before delivery.

The pipeline keeps each run's graded shot files because the editable FCPXML and EDL reference them. It refuses output collisions and reports timeline export failures. A completed render is not a saved editor project or creative approval. Preserve source assets and versioned project checkpoints before and after live edits; record the saved path and digest separately from the in-memory timeline receipt.

Clone-ready Fal graphs and their offline input compiler are documented in workflows/README.md. Compile brief and style direction before submission; do not assume a disconnected schema field affects a model prompt. Validate current endpoint fields against the actual provider before a paid run.

For Blender, use the full textured source GLB; retopology is a separately named derivative and never replaces that source. blender_prop.py supports explicit --width 1920 --height 1080 --fps 30 --receipt receipt.json and preserves pack-derived rim lighting and material textures. Saving a scene is distinct from rendering it; inspect the receipt's state and packed images.

For Resolve overlays, use taste.resolve.apply_placements with injected objects from an explicitly selected, versioned target. Set source_end_mode to the convention verified on that host. Do not assume an inclusive source end across Resolve versions. The adapter allocates overlapping effects above preserved tracks and checks every placement immediately and again after all appends. Composite integers must match the installed API; for the verified Studio 21 host, Screen is 5, not the historical builder's incorrect 22. The receipt proves in-memory placement only. Save the project, verify its checkpoint, then inspect the exact rendered output before reporting delivery.

When to Activate

• "make a video in this style" / "apply the pack" / "supplement this footage"
• Assembling generated clips into something with real edit rhythm
• Minting 3D props from a look and getting them back into the video
• Verifying that a finished piece actually matches its reference

Division of Labour

The model supplies content, motion, framing and lighting structure. The pack supplies colour and rhythm. This is measured, not stylistic preference — see taste-distillation for the numbers. Practical consequences:

• The generation prompt contains zero colour language. Add "Colour: none. Render neutral. Grading is applied afterwards."
• Keep two separate flags: --brief (what HAPPENS: subject, action, place) and --style-steer (how it LOOKS). Merging them leaks style words into the scene ("teal" becomes a teal object) and subject words into the grade.
• The model responds to local, checkable rules far better than global ones. "Backgrounds pure black and unlit; subjects blowing toward white" works; "extreme contrast" does not.

Generate TAKES, Not Shots

The obvious reading of "match the cadence" is one generation per shot. It is economically absurd. A reference averaging 0.78s/shot against an endpoint with a 4-second floor turns a 10s piece into 12 calls, 48 generated seconds for 10 used (21% efficiency), and twelve unrelated clips stitched into what should read as continuous.

Editors roll a longer take and cut inside it. Grouping shots into ~5s takes: 3 calls, 13 generated seconds, 77% efficiency, and consecutive shots that actually belong to each other because they came from the same generation.

Tell the model what shape you want, or it renders a slow locked-off push and six cuts inside it read as a stutter:

"Filmed as ONE continuous take with no hard cuts inside it. It will be cut into 6 pieces of roughly 0.8s in the edit, so the framing, subject and light must keep changing throughout — any 0.8s window has to stand alone as its own shot."

Cutting Rules

• Re-encode, never stream-copy. Stream copy only cuts on keyframes, which at 0.78s mean shot length rounds every boundary to the nearest GOP — destroying the exact thing the pipeline exists to preserve.
• When supplementing existing footage, use its own shot boundaries. Slicing a base video into contiguous pieces and playing them in order just reassembles the original: every "cut" lands mid-shot and is invisible. Measured, a 20-shot assembly registered only 12 detected cuts. Detect real boundaries and take every Nth so consecutive picks are guaranteed discontinuous. That moved a cut measurement from 1.29s to 0.83s against a 0.78s target.
• Sample shot lengths from the reference's distribution, not from its mean, so the cut inherits rhythm variance instead of flattening to even clips.

Grading Rules

• Direct measurement beats a baked LUT when you have the clip: measure it, match its L* CDF, apply zone chroma. grade_clip_direct reached MAE 1.58 and contrast 34.0 against 34.7.
• Anchor, do not CDF-match, when the clip's histogram is unlike the reference's. Forcing a 68%-black generated clip onto a busy reference histogram lifted the entire background out of black: background preservation fell to 26.0% (CDF) versus 69.0% (anchor), while MAE and contrast both still looked excellent. Default to anchored tone.
• Batch size matters. Grading 48 frames at once OOM-killed the process; 6 is safe.

Recovered fal Platform Behavior

These observations and endpoint examples came from the recovered workflow. Recheck current endpoint metadata; they are not guarantees about every future provider version. The bundled graph templates record the separately verified workflow inputs, including explicit generated-audio control.

These cost real time to discover. Check them before designing a graph.

Endpoint picks

Seedance is ~4x Kling o3's price for the same 5 seconds. It earns that on multi-reference fidelity (up to 50 mixed image/video/audio refs) and does not earn it when conditioning on a single still.

Model IDs and prices drift. Verify against fal.ai/models before promising any of them.

The 3D Branch

python mint3d.py --genre <name> --from-stills 4 --retopo --render
python mint3d.py --genre <name> --prompt "a cracked chrome visor" --render

• Generate a clean plate first; do not lift from reference stills. The endpoint's stated input requirement is simple background, single object, object >50% of frame. Reference reels are the opposite of that — collages, wide shots, several subjects, burnt-in graphics — and they produce sculpted noise. Text → single-object plate → mesh costs ~$0.15 extra and is the difference between a usable mesh and a discarded one.
• Multi-view is named per-angle fields, not a list. input_image_url (front, required), then back_image_url, left_image_url, right_image_url, left_front_image_url, right_front_image_url, top_image_url, bottom_image_url. There is no input_image_urls and no multi_view flag — inventing them degrades every mint to single-view while appearing to work. A wrong angle label is worse than omitting the view, because the model trusts it.
• Address the GLB by key, not by position. The response carries a thumbnail PNG and a model_urls block alongside model_glb; taking the first URL works only until the keys reorder, and a preview PNG downloads fine — nothing fails until Blender refuses to open it.
• Request PBR maps. Without them the mesh lights like painted cardboard.
• Retopologise if anyone will edit or rig it. Generated meshes are dense and chaotic.
• Render locally to close the loop. Once a turntable is frames, it is footage, and every downstream stage already handles footage — grade it, cut it, screen it as an element, or upload it as a conditioning reference. Use Blender when a binary is on PATH; keep a dependency-light software rasteriser as the default, because a headless GL context is the single most common thing missing from a container and a renderer that only works on a workstation is not part of a pipeline.