Auto Cut

Auto-Cut - measured silence & dead-weight trimming

Principles

1. Cut BEFORE building scenes/captions. Cutting afterwards throws off every timestamp (captions, zooms, SFX). Output of this step: assets/<source>.cut.mp4 + assets/transcript.cut.json - every later step uses the cut version.
2. Cutting is MANDATORY when the brief enables autoCut - it is not a suggestion. If nothing can be cut, you must state a concrete reason (the video was already tight) in the report.
3. The thresholds live in the server, not in your head. Do NOT run silencedetect yourself and do NOT pick a dB threshold by feel. Two measured facts killed that workflow:
   • The right threshold is a property of the FILE, not a constant. On one real file, -40dB found 0 silences, -30dB found 13, -25dB found 21. A hardcoded number is either useless or eats speech.
   • Loudness alone cannot tell "pausing" from "speaking quietly". On the source file measured, silencedetect at -30dB reported 48.6s of "silence" but only 18.2s of it sat in a real gap between words; the other 30.4s was inside words (syllable breaks, unvoiced Vietnamese finals c/t/p/ch). Cutting on sound level alone swallows speech. The server does the measuring (autoTrim.ts) and the candidate generation (deadWeight.ts). You do the reviewing. That split is the whole point.
4. Two kinds of dead weight get cut: silences (machine-measured, guarded by the transcript) and content dead weight (fillers, stutters, restated takes - proposed by the server, approved by you; plus repeated POINTS, which only you can find - see Step 3).

Step 0 - Prerequisite: a transcript with word timestamps

The whole guard depends on it. The server looks for assets/transcript.raw.json, then assets/transcript.json. Without one, analysis still runs but comes back guarded: false, the dead-weight list is empty, and the numbers are guesswork in both directions (it can pass a bad cut and fail a good one). Transcribe first - never trim a Vietnamese talking head unguarded.

Step 1 - Analyze (free, repeatable, no encoding)

POST http://localhost:6869/api/projects/<id>/auto-trim/analyze
body: {}                                   # or { "source": "assets/face.mp4", "level": "tight" }

level is natural | default | tight and defaults to brief.autoCutLevel. Response:

• silence - durationSec, the chosen thresholdDb + thresholdNote (why that threshold won), measured noiseFloorDb, the silences that will be cut, keepRanges, removedSec, the full sweep table, and wordGuard (how many seconds the transcript vetoed).
• deadWeight - candidates[] with kind (filler | stutter | repeat-take | hesitation), start/end, text, confidence, reason, context; plus totalSec and byKind.
• guarded - true only when a transcript was found. If false, fix that before cutting.

Nothing is encoded, so call it as often as you like (a 217s source analyzes in about a second).

Step 2 - REVIEW every candidate (your job, not the machine's)

The list is deterministic: same transcript, same candidates, same confidence. What it cannot do is understand meaning. Go through them one by one and keep only the ones you would defend:

• Low confidence means "read the context first", not "probably fine". Vietnamese fillers almost always collide with real words. đó, ấy, thế, mà, là are both filler particles and real demonstratives/conjunctions.
• Connector phrases (hoặc là, tức là, bởi vì là, với lại là) are the trap. Measured on a real transcript: "…có thể là ok ứng dụng nó / Hoặc là / Tham khảo để tìm cách…" is a genuine alternative - cutting it destroys one branch of the sentence. But "Còn trường hợp mà mọi người có thể nghe / Hoặc là / Còn trường hợp mà người AI không ứng dụng được…" is an abandoned sentence and should go. The surface form is identical; only the meaning separates them. That is exactly why these come back with a low base confidence and a reason that says READ THE CONTEXT.
• repeat-take candidates: check that the later take really is the fuller one before approving.
• hesitation candidates are pure silence between words - usually safe, but check you are not removing a deliberate beat before a punchline.

Reject freely. A filler left in costs 0.4s; a real word cut makes the sentence nonsense and the viewer hears it immediately.

Step 3 - REPEATED POINTS (only you can find these - mandatory)

No detector catches this, and it is the dead weight users complain about most. Read the transcript as a piece of speech and analyze it semantically:

1. Group sentences that make the SAME POINT - the words need not match, only the content. A speaker often restates one point 2-3 times: short/stumbling first, more complete later. Consider NON-adjacent sentences too (makes point A, rambles, comes back to point A in more detail).
2. Keep exactly ONE version per group - the MOST COMPLETE one:
   • Earlier short version, later fuller restatement -> keep the later, cut the earlier.
   • Later sentence is only a short echo ("đúng vậy, như tôi nói…") -> keep the earlier, cut the echo.
   • Two equivalent versions -> keep the smoother delivery (fewer stumbles, no fillers).
3. Check the flow after cutting: re-read the kept text end to end. If a kept sentence refers back ("như vừa nói") to one you cut, either keep both or pick the version without the back-reference.
4. Turn each decision into a {start, end} range and send it with the approved candidates in Step 4.
5. Build a table before cutting (put it in the report): timestamp | sentence cut | reason | sentence kept - so the user can review every decision.

Step 4 - Apply (goes through the render queue)

POST http://localhost:6869/api/projects/<id>/auto-trim/apply
body: { "cutCandidates": [{ "start": 51.81, "end": 52.25 }, …] }   # ONLY what you approved
-> 202 { job }

Send an empty list if you approved nothing - the measured silences still get cut. Then poll GET /api/jobs/<jobId> until it finishes. The job:

1. Re-analyzes with the transcript guard, merges the silence ranges with your approved ranges (overlaps merged, so nothing is double counted).
2. Drops any approved range that would swallow a word midpoint - a last safety net over your review, and it logs every range it rejects. Rejected ranges in the log mean you approved something that sits on real speech; go back and re-read that spot.
3. Cuts in ONE ffmpeg pass (trim/atrim + setpts/asetpts + concat), writing assets/<stem>.cut.mp4.
4. Remaps every timestamp into the cut timeline -> assets/transcript.cut.json.
5. Verifies the OUTPUT with the remapped words and writes assets/auto-trim-report.json.

If nothing is worth cutting, the job deliberately does NOT produce a .cut.mp4 (a re-encoded identical copy only loses quality) - output in the report is null and you keep using the source.

Step 5 - Read the report (the job is not done until you do)

assets/auto-trim-report.json holds: before/after duration, removed.silenceSec vs removed.approvedSec, the chosen threshold and why, rejected candidates, and verification.

• verdict: "pass" - the result meets the profile for that level.
• verdict: "fail" - the job still finished and the file is usable, but it did NOT meet the profile. The log says so explicitly. Approve more dead weight (Step 2/3) and run apply again, or state in the final report why you are accepting it. Never report the cut as done while ignoring a fail.

The pass criteria are not the old "no silence > 0.8s" rule - measurement showed that rule is far too lax. A real file passed it while carrying 13 residual silences of 0.45-0.67s totalling 6.7s (4.2% of the runtime), which sounds obviously draggy. The profiles now cap BOTH the longest single silence and the total ratio (default: 0.5s / 3%).