Ffmpeg Media Finishing

FFmpeg media finishing

Use this skill at the finishing stage: after creative generation or editing has produced source media and before the user receives deliverables. Your job is to turn fragile, inconsistent media outputs into reproducible, spec-aware files with an audit trail.

This skill is provider-independent. It assumes local ffmpeg and ffprobe are available, but not that every codec, filter, hardware encoder, or subtitle renderer is compiled into the installed build. FFmpeg build capabilities, encoder names, platform upload rules, and social specs are volatile; re-check them at production time with the commands below and with the current delivery spec.

Operating rule

Do not guess the state of media. Probe first, decide, run deterministic commands, then probe outputs.

Every finishing handoff should include:

• input audit summary from ffprobe;
• chosen delivery spec and any unresolved assumptions;
• exact commands run, including FFmpeg version/build;
• output audit summary;
• checksums and, when useful, frame hashes;
• known risks such as re-encoded captions, color metadata ambiguity, clipping, lossy transcodes, or platform-specific uncertainty.

Bundled deterministic assistance

This skill includes a deterministic Python 3.11+ standard-library helper at scripts/media_probe.py. Use it when you need a repeatable technical audit or a quick delivery-spec gate before or after finishing. It is assistance, not a replacement for perceptual review, color-managed approval, audio listening, caption/accessibility review, rights review, or platform/client signoff.

Documented fact, verified 2026-07-11: ffprobe is designed to gather multimedia stream information and can print JSON with -show_format, -show_streams, and -print_format json; if an input cannot be opened or recognized it returns a positive exit code. The helper invokes ffprobe with a subprocess argv list, never through a shell, and never runs ffmpeg or mutates media.

Invocation:

python skills/production/runtime-assembly/ffmpeg-media-finishing/scripts/media_probe.py input.mp4 --pretty
python skills/production/runtime-assembly/ffmpeg-media-finishing/scripts/media_probe.py input.mp4 --expect expected.json --pretty

Options:

• media: required media path to inspect; the file is opened only by ffprobe.
• --expect expected.json: optional expectation spec. Exit code 0 means the audit ran and every expectation passed; exit code 1 means the audit ran but one or more expectations failed.
• --ffprobe PATH_OR_NAME: override the ffprobe executable. Default is ffprobe on PATH.
• --timeout SECONDS: ffprobe timeout. Default is 30.
• --pretty: emit two-space formatted JSON instead of compact JSON.

Exit codes:

0  audit completed and expectations passed, or no expectations were provided
1  audit completed but at least one expectation failed
2  usage or input-file error
3  ffprobe executable unavailable
4  ffprobe returned a non-zero exit code
5  ffprobe timed out
6  ffprobe output was not valid or usable JSON
7  expectation JSON could not be read, parsed, or validated

Normalized audit schema, version 1.0:

{
  "schema_version": "1.0",
  "status": "pass",
  "tool": "media_probe.py",
  "ffprobe": {
    "executable": "ffprobe",
    "argv": ["ffprobe", "-v", "error", "-show_format", "-show_streams", "-print_format", "json", "input.mp4"]
  },
  "media": {
    "path": "input.mp4",
    "format_name": "mov,mp4,m4a,3gp,3g2,mj2",
    "format_long_name": "QuickTime / MOV",
    "duration_seconds": 1.0,
    "size_bytes": 12345,
    "bit_rate": 98765
  },
  "streams": {
    "counts": {"video": 1, "audio": 1, "subtitle": 0, "other": 0},
    "video": {
      "index": 0,
      "codec_name": "h264",
      "profile": "High",
      "width": 1920,
      "height": 1080,
      "pix_fmt": "yuv420p",
      "r_frame_rate": {"raw": "30/1", "value": 30.0},
      "avg_frame_rate": {"raw": "30/1", "value": 30.0},
      "duration_seconds": 1.0
    },
    "audio": {
      "index": 1,
      "codec_name": "aac",
      "sample_rate": 48000,
      "channels": 2,
      "channel_layout": "stereo",
      "sample_fmt": "fltp",
      "duration_seconds": 1.0
    }
  },
  "expectations": null
}

If a video or audio stream is absent, its normalized value is null and the corresponding count is 0. Unknown numeric values are emitted as null rather than invented.

Expectation schema:

{
  "checks": [
    {"field": "media.duration_seconds", "op": "tolerance", "value": {"value": 10.0, "tolerance": 0.05}},
    {"field": "streams.video.codec_name", "op": "equals", "value": "h264"},
    {"field": "streams.video.width", "op": "equals", "value": 1920},
    {"field": "streams.video.height", "op": "equals", "value": 1080},
    {"field": "streams.video.avg_frame_rate.value", "op": "tolerance", "value": {"value": 30.0, "tolerance": 0.01}},
    {"field": "streams.video.pix_fmt", "op": "equals", "value": "yuv420p"},
    {"field": "streams.audio.codec_name", "op": "equals", "value": "aac"},
    {"field": "streams.audio.sample_rate", "op": "equals", "value": 48000},
    {"field": "streams.audio.channels", "op": "equals", "value": 2}
  ]
}

Supported op values:

• equals: exact JSON equality.
• in: actual value must be one of the values in the expected array.
• range: numeric actual value must be within {"min": number, "max": number}; either boundary may be omitted.
• tolerance: numeric actual value must be within value +/- tolerance.

Malformed expectation numeric values, non-finite numbers, and negative tolerance values are usage errors in the expectation file. The helper reports them as exit code 7 with structured error JSON instead of treating them as failed media checks.

Fields are dotted paths into the normalized report. Supported expectation targets intentionally cover common finishing gates: duration tolerance, video codec, width, height, frame rate, pixel format, audio codec, sample rate, and channel count. The schema can also address any stable field shown above, but do not treat this helper as proof of perceptual quality, color correctness, loudness compliance, or rights status.

1. Preflight the toolchain and inputs

Record the FFmpeg build and encoder/filter availability because behavior depends on the installed binary, compile flags, and external libraries.

ffmpeg -hide_banner -version
ffmpeg -hide_banner -encoders
ffmpeg -hide_banner -muxers
ffmpeg -hide_banner -filters
ffmpeg -hide_banner -h encoder=libx264
ffmpeg -hide_banner -h filter=loudnorm

Run ffprobe on every source and save machine-readable JSON when the job has more than one input, a client spec, or a risk of mismatch:

ffprobe -v error \
  -show_format -show_streams -show_chapters \
  -print_format json input.mp4 > input.ffprobe.json

For a compact human audit:

ffprobe -v error \
  -select_streams v:0 \
  -show_entries stream=codec_name,profile,width,height,pix_fmt,r_frame_rate,avg_frame_rate,time_base,field_order,color_range,color_space,color_transfer,color_primaries,sample_aspect_ratio,display_aspect_ratio,duration,nb_frames \
  -of default=noprint_wrappers=1 input.mp4

ffprobe -v error \
  -select_streams a:0 \
  -show_entries stream=codec_name,sample_rate,channels,channel_layout,sample_fmt,bits_per_sample,duration,bit_rate \
  -of default=noprint_wrappers=1 input.mp4

Audit at least:

• container, duration, file size, timecode/chapter needs;
• video codec, resolution, display aspect ratio, sample aspect ratio, pixel format, bit depth, alpha, interlacing, nominal and average frame rate, time base, start time;
• color metadata: range, primaries, transfer, matrix/colorspace;
• audio codec, sample rate, channel count/layout, duration, bit depth/sample format, loudness target if required;
• subtitle streams and external captions;
• attachments such as fonts in MKV/ASS workflows;
• whether inputs have identical parameters before concat or stream copy.

If ffprobe reports missing or inconsistent metadata, treat it as a warning, not as permission to invent values. Ask the user or infer cautiously from the creative pipeline only when necessary, and document the inference.

2. Choose stream copy, transcode, or filter

Use stream copy when the encoded streams are already acceptable and only container-level operations are needed:

ffmpeg -y -i input.mov -map 0 -c copy -movflags +faststart output.mp4

Stream copy is appropriate for remuxing, trimming on keyframe boundaries, adding compatible metadata, or extracting tracks. It is not appropriate when changing resolution, codec, pixel format, frame rate by resampling, loudness, captions burn-in, overlays, color conversion, or any filtergraph output. FFmpeg documents -codec copy as copying streams without re-encoding; filters require decoded frames and therefore a re-encode.

Transcode when the deliverable requires a different codec/container, resolution, frame rate, pixel format, audio layout, loudness, burned captions, overlays, or compatibility profile.

When quality matters, prefer a constant-quality setting over arbitrary bitrates unless a platform or client spec requires a bitrate cap. For H.264 with libx264, a common mezzanine-to-delivery pattern is:

ffmpeg -y -i input.mov \
  -map 0:v:0 -map 0:a? \
  -c:v libx264 -preset slow -crf 18 -pix_fmt yuv420p \
  -c:a aac -b:a 192k -ar 48000 \
  -movflags +faststart output.mp4

Production heuristic: use lower CRF for higher quality/larger files and higher CRF for smaller files. Verify visually; CRF values are not delivery guarantees. If a client gives a target bitrate, max bitrate, GOP length, profile, chroma, or level, follow the spec instead of this heuristic.

3. Containers, codecs, and compatibility

Distinguish container from codec:

• MP4/MOV/MKV/WebM are containers.
• H.264/H.265/AV1/ProRes/DNxHR/AAC/Opus/PCM are codecs.
• A stream can be compatible with one container and incompatible with another.

Common delivery choices:

• Broad web/social compatibility: MP4 with H.264 video and AAC audio, usually yuv420p.
• Transparent or alpha-preserving intermediate: ProRes 4444 in MOV, PNG sequence, WebM VP9/AV1 with alpha only if the target supports it and the installed build can encode it.
• Archival or review mezzanine: ProRes/DNxHR/PCM or another client-specified mezzanine; do not substitute H.264 unless approved.
• WebM: typically VP9/AV1/Opus or VP8/Vorbis depending on target compatibility.
• Audio-only: WAV/PCM for interchange; AAC/MP3/Opus for compact delivery according to platform spec.

Before using a codec, check the local build:

ffmpeg -hide_banner -encoders | findstr /i "264 265 av1 prores aac opus"

On non-Windows shells, use grep -Ei instead of findstr.

Hardware encoders such as h264_nvenc, hevc_videotoolbox, h264_qsv, or h264_amf are machine- and driver-dependent. They can be fast but may differ in rate-control behavior and quality. Do not assume availability; test a short sample.

4. Scaling, cropping, padding, and aspect ratio

Pick the geometry behavior intentionally:

• Scale to fit inside a canvas and pad when preserving the whole frame matters.
• Crop to fill when edge content may be sacrificed.
• Scale exactly only when distortion is acceptable or source already matches the target aspect ratio.
• Normalize sample aspect ratio to square pixels for most web/social deliverables.

Fit inside 1920x1080 with black padding:

ffmpeg -y -i input.mp4 \
  -vf "scale=1920:1080:force_original_aspect_ratio=decrease,pad=1920:1080:(ow-iw)/2:(oh-ih)/2,setsar=1" \
  -c:v libx264 -crf 18 -preset slow -pix_fmt yuv420p \
  -c:a copy output_1080p_letterbox.mp4

Fill 1080x1920 vertical by cropping:

ffmpeg -y -i input.mp4 \
  -vf "scale=1080:1920:force_original_aspect_ratio=increase,crop=1080:1920,setsar=1" \
  -c:v libx264 -crf 18 -preset slow -pix_fmt yuv420p \
  -c:a aac -b:a 192k output_9x16.mp4

For generated stills, use even dimensions for codecs that require chroma subsampling:

ffmpeg -y -loop 1 -framerate 30 -t 5 -i still.png \
  -vf "scale=1920:1080:force_original_aspect_ratio=decrease,pad=1920:1080:(ow-iw)/2:(oh-ih)/2,setsar=1,format=yuv420p" \
  -c:v libx264 -crf 18 -preset slow -an still_card.mp4

5. Frame rate, timestamps, time base, and sync

Frame rate changes are a common source of judder, drift, duplicate frames, and broken captions. Inspect both r_frame_rate and avg_frame_rate; variable-frame-rate sources often have a confusing nominal rate.

Rules:

• Preserve source frame rate unless a delivery spec requires conversion.
• Avoid converting 23.976/24/25/30/50/60 fps casually; conversions can introduce cadence artifacts.
• For social variants, choose a single constant frame rate only when the platform/client spec, editing timeline, or concat requirements demand it.
• When cutting with stream copy, expect keyframe-aligned trimming unless you re-encode.
• When using filters such as fps, trim, setpts, atrim, or asetpts, re-encode the filtered stream.

Force constant 30 fps for a platform-specific version:

ffmpeg -y -i input.mp4 \
  -vf "fps=30,scale=1080:1920:force_original_aspect_ratio=increase,crop=1080:1920,setsar=1" \
  -c:v libx264 -crf 18 -preset slow -pix_fmt yuv420p \
  -c:a aac -b:a 192k -ar 48000 output_30fps_vertical.mp4

Trim accurately by re-encoding:

ffmpeg -y -ss 12.345 -to 27.890 -i input.mp4 \
  -map 0:v:0 -map 0:a? \
  -c:v libx264 -crf 18 -preset slow \
  -c:a aac -b:a 192k clip_exact.mp4

Fast keyframe-aligned trim by stream copy:

ffmpeg -y -ss 12 -to 28 -i input.mp4 -map 0 -c copy clip_fast.mp4

If audio and video durations differ after finishing, inspect start times, PTS/DTS warnings, and whether a filtergraph dropped or duplicated frames. Do not hide sync defects with arbitrary audio padding unless the spec calls for it.

6. Color range and color space caution

FFmpeg can set or transform color metadata, but metadata is not the same as a correct color conversion. Generated assets often lack reliable color tags. Mismatched full/limited range, Rec.709/Rec.2020 primaries, transfer functions, or HDR/SDR assumptions can produce washed-out or crushed exports.

Minimum practice:

1. Probe source color tags.
2. If the source was generated or composited in a known working space, carry that information into finishing notes.
3. For normal SDR web delivery, commonly target Rec.709 metadata and yuv420p unless the client spec says otherwise.
4. If converting HDR, wide gamut, log, or client-critical brand colors, escalate to a color-managed workflow or human color approval.
5. After setting tags, re-probe the output and visually review on a known-good player.

Example of tagging an SDR H.264 delivery without claiming a color transform:

ffmpeg -y -i input.mp4 \
  -c:v libx264 -crf 18 -preset slow -pix_fmt yuv420p \
  -color_primaries bt709 -color_trc bt709 -colorspace bt709 -color_range tv \
  -c:a aac -b:a 192k output_sdr_tagged.mp4

If you must transform between matrices/ranges, use a color filter such as zscale or colorspace only after checking the installed filter options and validating with test frames. Do not use metadata flags alone as a conversion.

7. Audio loudness, sample rate, and channel layout

Finish audio to the delivery spec. If there is no spec:

• For web/social video, 48 kHz AAC stereo is broadly compatible.
• For broadcast, client, or advertising delivery, follow the exact loudness standard and tolerance.
• For archival/interchange, preserve PCM/WAV or deliver separate stems when requested.

Use standards language precisely:

• ITU-R BS.1770 defines loudness and true-peak measurement algorithms.
• EBU R 128 is a broadcast recommendation based on loudness normalization and true-peak limits.
• Platform loudness targets vary and are volatile; re-check platform documentation rather than relying on memory.

Measure first:

ffmpeg -hide_banner -i input.mp4 \
  -af loudnorm=I=-16:TP=-1.5:LRA=11:print_format=json \
  -f null -

For repeatable loudness normalization, use two-pass loudnorm: first measure, then feed measured values into the second pass. Example target shown is a common online-video heuristic, not a standard:

ffmpeg -hide_banner -i input.mp4 \
  -af loudnorm=I=-16:TP=-1.5:LRA=11:print_format=json \
  -f null -

ffmpeg -y -i input.mp4 \
  -af "loudnorm=I=-16:TP=-1.5:LRA=11:measured_I=-18.7:measured_TP=-3.2:measured_LRA=8.4:measured_thresh=-29.1:offset=0.2:linear=true:print_format=summary,aformat=sample_rates=48000:channel_layouts=stereo" \
  -c:v copy -c:a aac -b:a 192k output_loudnorm.mp4

Do not normalize blindly when preserving a theatrical mix, music dynamics, stems, spatial audio, or client-approved master. Ask before downmixing 5.1/7.1/Atmos/spatial audio to stereo.

Downmix example. Inspect the probed channel layout first and adapt channel names; some 5.1 sources expose side channels as SL/SR rather than back channels BL/BR.

ffmpeg -y -i input.mov \
  -map 0:v:0 -map 0:a:0 \
  -c:v copy \
  -af "pan=stereo|FL=0.707*FC+FL+0.707*BL+0.5*LFE|FR=0.707*FC+FR+0.707*BR+0.5*LFE,aformat=sample_rates=48000:channel_layouts=stereo" \
  -c:a aac -b:a 192k output_stereo.mp4

Review downmixes by ear. Formulaic downmix coefficients are a starting point, not a substitute for listening.