Shortform Editing

Shortform Editing — Cut the Footage Into a Posted-Ready Short

You are the edit-bay technician. Someone hands you real footage — a talking-head take, a screen recording, a voiceover, some B-roll — and you turn it into a vertical short that is ready to post: silences gone, cuts fast, captions burned in word-by-word, motion snapped to the beat, exported to spec. You encode pixels and timing. You do not write the words or pick the hook.

The rigor is a checkable file: a 1080x1920 H.264 MP4 with burned captions that ffprobe confirms, plus the intermediate artifacts (a cut list, an .ass/.srt, a beat grid). If you cannot ffprobe the result and see the specs, you are not done.

Scope: three skills touch "a vertical video" — know your lane

One-line router: footage in → cut, captioned, exported → this skill. Words or a plan → video-shorts. A React codebase → remotion-video. You are the only one of the three that is tool-portable.

Pick your route first

Do not start cutting before you have chosen the tool. The choice is about who runs it and whether it must reproduce.

The rest of this skill is the ffmpeg + WhisperX pipeline (the reproducible path), with the Remotion and CapCut routes called out where they diverge. Long command blocks live in references/ffmpeg-pipeline.md; the caption detail lives in references/captions.md.

The reproducible pipeline, in order

This is the spine. Run it top to bottom; each step feeds the next.

1. Extract a clean mono 16 kHz audio track for transcription.

ffmpeg -i in.mp4 -vn -ac 1 -ar 16000 a.wav

Why mono 16 kHz: that is what Whisper models expect; feeding stereo 48 kHz wastes time and changes nothing about accuracy.

2. Transcribe with word-level timestamps. Segment timestamps are not enough — word-by-word captions need per-word timing.

whisperx a.wav --model large-v3 --output_format json --highlight_words True

WhisperX (v3.8.5) runs faster-whisper plus wav2vec2 forced alignment, giving word timing accurate to under 100 ms — the floor for karaoke highlight. faster-whisper alone with word_timestamps=True works (~10% overhead) but drifts more on word boundaries. Why forced alignment: Whisper's native word stamps are interpolated and jitter; forced alignment pins each word to the audio. See references/captions.md for the install and the JSON shape.

3. Detect silence, then cut video AND audio together. This is the step people get wrong.

# Pass 1: find the silent ranges (speech threshold)
ffmpeg -i in.mp4 -af silencedetect=noise=-25dB:d=0.3 -f null - 2> silence.log

silenceremove alone is an audio-only filter and desyncs the video — never use it as a one-shot. Parse the silence_start / silence_end lines from silence.log, invert them into keep-ranges, and build a select + aselect filter (or a trim+concat graph) that drops the same ranges from both streams. Thresholds: speech noise=-25dB:d=0.3; music beds noise=-40dB:d=1. The filter-generation snippet is in references/ffmpeg-pipeline.md.

4. Build the beat grid (only if you are cutting to music).

python -c "import librosa; y,sr=librosa.load('music.wav'); \
_,b=librosa.beat.beat_track(y=y,sr=sr,units='time'); print(*b,sep='\n')" > beats.txt

beats.txt is a list of beat times in seconds. CapCut drops these markers on the waveform automatically; librosa (or aubio) gives you the same grid in a script.

5. Snap cut points and zoom keyframes to the nearest beat. Round each cut time to its closest value in beats.txt. A zoom-punch on the beat reads as energy; see Beat sync below for when NOT to.

6. Burn the karaoke .ass captions.

ffmpeg -i tightened.mp4 -vf "ass=captions.ass" -c:a copy captioned.mp4

The .ass carries the per-word \k timing built from step 2's word timestamps and the safe-zone margins. Generation detail and the template are in Karaoke captions below.

7. Export the platform master (see Export specs).

Karaoke captions

Word-by-word captions are the signature look. Build them from the word timestamps, not from segment timestamps.

• Timing. Each word's on-screen highlight comes from its forced-alignment start/end. In .ass, the \k<centiseconds> tag advances the karaoke fill per word. Word-by-word from segment timing jitters and lands the highlight on the wrong word — always use the per-word stamps from step 2.
• Legibility for sound-off. Big bold sans, heavy outline/stroke, optional drop shadow. People watch muted: if it is not readable at a glance on a phone, it failed. Keep it to ≤ 2 lines / ~6 words on screen at once.
• Safe zone — the non-obvious one. Place captions in the vertical middle band, not the bottom. Platform UI (right-rail icons, bottom caption/CTA/profile) covers the bottom ~15–20% and the right edge. Keep captions and key text inside the central ~80% of the 1080x1920 frame. Captions burned at the bottom get hidden behind the TikTok buttons — this is the #1 "my captions are broken" cause.

Minimal .ass style block (full template in references/captions.md):

[V4+ Styles]
Style: Karaoke,Montserrat,64,&H00FFFFFF,&H0000FFFF,&H00000000,&H64000000,-1,0,0,0,100,100,0,0,1,5,2,5,80,80,640,1

MarginV (the trailing 640) lifts the line into the middle band; the heavy Outline (5) keeps it readable on any background.

Remotion route: if you are inside a Remotion project, use @remotion/captions — createTikTokStyleCaptions({ captions, combineTokensWithinMilliseconds }) pages the tokens. Low (200–500 ms) = word-by-word pop; high (1200–2000 ms) = phrase pages. Page durationMs was added in v4.0.261. CapCut route: Text → Auto captions → Generate, then restyle.