Kling Advanced Lip Sync

Kling Advanced Lip-Sync

Use this skill for Kling AI Open Platform's documented Advanced Lip-Sync workflow on an existing video. It is distinct from Avatar, which generates a new talking performance from a still image and audio.

Verification and evidence

• Documented fact: behavior stated in first-party Kling Open Platform pages.
• Production heuristic: a practical sync/repair recommendation not guaranteed by Kling.
• Empirical observation: a result measured from an actual task/output.

Facts were verified 2026-07-12. Endpoint visibility, entitlement, pricing, callback behavior, retention, limits, and policy can change. Re-check official documentation, authenticated console, account package, and applicable agreement before paid use.

Official workflow

Base URL in current examples:

https://api-singapore.klingai.com

Operations:

1. POST /v1/videos/identify-face
2. POST /v1/videos/advanced-lip-sync
3. GET /v1/videos/advanced-lip-sync/{id}
4. GET /v1/videos/advanced-lip-sync

Although the navigation may call a tab Face Recognition, the operation is Identify Face. Describe it as face identification/detection for selecting a visible face; do not claim general identity matching.

Authentication

Official docs use JWT Bearer authentication derived from Kling Open Platform access/secret credentials:

Authorization: Bearer ${KLING_JWT}
Content-Type: application/json

Follow the current authentication page for token construction and lifetime. Generate server-side, use least privilege, and never place keys/secrets/JWTs in browser code, logs, callbacks, or fixtures.

Validate the video source

Identify Face accepts exactly one:

• video_id: Kling-generated video from the documented recent window;
• video_url: accessible external video.

They are mutually exclusive.

Current external-video constraints include .mp4 or .mov, 2-60 seconds, at most 100 MB, 720p or 1080p, and width/height each 512-2160 pixels. Current docs also limit Kling video_id recency to 30 days. Kling performs content validation.

Preflight codec/container, duration, dimensions, file size, URL reachability, audio presence, face visibility, cuts, occlusion, and consent. Do not upload a private URL that Kling cannot access or a mutable URL that can change between operations.

Identify and select a face

Example:

POST /v1/videos/identify-face

{"video_url":"https://example.com/source.mp4"}

The response includes session_id and face_data[]; each face can include face_id, face_image, start_time, and end_time in milliseconds.

Present detected face crops/intervals to the user or authorized operator. If several faces exist, require an explicit face_id unless a documented, approved selection rule exists. Record selected face, reason, and interval.

face_choose is an array, but current docs explicitly say only one-person lip-sync is supported. Do not claim simultaneous multi-speaker replacement. Re-identify after any source-video edit; do not reuse a stale session against changed media.

Kling does not publicly establish session expiry/reuse semantics in the verified docs. Keep identify/create close together and handle invalid sessions as a new identification job.

Prepare audio and timing

For the selected face, provide exactly one:

• audio_id: recent Kling TTS audio;
• sound_file: accessible URL or Base64 audio.

They are mutually exclusive.

Current external-audio constraints include .mp3, .wav, .m4a, or .aac, 2-60 seconds, and at most 5 MB. Current docs limit audio_id recency to 30 days.

Timing fields are milliseconds:

• sound_start_time: crop start in source audio;
• sound_end_time: crop end;
• sound_insert_time: insertion point in source video.

Validate:

$$ cropDuration = soundEnd - soundStart \ge 2000\text{ ms} $$

The crop must fit inside the source audio, the inserted range must fit inside the video, and the inserted range must overlap the selected face's detected appearance interval by at least two seconds.

Use explicit millisecond arithmetic. Never pass seconds by accident.

Mix controls:

• sound_volume: current documented range 0-2, default 1;
• original_audio_volume: range 0-2, default 1; no effect for silent source.

Preserve ambience when it matters, but ensure source dialogue does not create double speech. Review the final mix separately from lip motion.

Create the task

Illustrative request:

{
  "session_id": "<identify-face-session>",
  "face_choose": [
    {
      "face_id": "<approved-face-id>",
      "sound_file": "https://example.com/dialogue.wav",
      "sound_start_time": 0,
      "sound_end_time": 3000,
      "sound_insert_time": 1000,
      "sound_volume": 1,
      "original_audio_volume": 0.25
    }
  ],
  "external_task_id": "scene-014-take-03",
  "callback_url": "https://example.com/hooks/kling",
  "watermark_info": {"enabled": false}
}

This assumes a three-second crop, sufficient video duration, and at least two seconds of overlap with the selected face interval.

external_task_id uniqueness is caller-owned. Store it with the system task_id, source/audio hashes, session/face IDs, timings, volumes, attempt, consent record, and cost preflight.

Monitor and reconcile

Task statuses:

• submitted
• processing
• succeed
• failed

Use the single-task endpoint as source of truth. Callbacks can reduce polling latency, but production handling should be idempotent and accept duplicates/out-of-order events. Unless the current callback page documents a signature mechanism, do not invent one. Retrieve the task before trusting or publishing callback data.

List queries use current pagination ranges; do not scan the list when a task ID is known.

Retry transient rate/concurrency/server errors with bounded jitter. Do not retry auth, permission, parameter, policy, or invalid-media failures unchanged. Preserve provider code/message/request ID.

Cost, access, and result custody

The public pricing page did not clearly establish a dedicated Advanced Lip-Sync list price during verification. Do not assume Avatar pricing applies. Check authenticated console/package entitlement and create a dated budget approval.

Successful task responses expose final deduction fields; reconcile actual units/balance against the estimate.

Generated media is documented as deleted after 30 days. Download promptly into controlled storage, checksum, probe, and preserve provenance. Do not use provider URLs as permanent delivery storage.

Endpoint entitlement and concurrency are account/package dependent. Errors such as rate, concurrency, package exhaustion, or permission denial should be surfaced precisely.