Repair one dialogue beat before rebuilding a scene
A short drama close-up may already have the right framing, identity and reaction while its speech needs a new take. Save that accepted visual source and approve the exact replacement recording. The useful test is whether the returned close-up serves that line without introducing distracting facial or scene changes.
The official ByteDance LatentSync repository documents an audio-conditioned lip-sync project and separate releases. It identifies version 1.6 as trained at 512 by 512 resolution and lists different inference memory requirements for 1.5 and 1.6. Record the version rather than treating every workflow named LatentSync as the same setup. We did not run either checkpoint for this guide.
Keep the source performance and audio traceable
For the first take, use a permitted single-person face video with a readable mouth, restrained head movement and a stable camera. Watch the original for blur, existing facial changes and crop problems. A defect present before the lip-sync stage belongs in the source record.
Leave the envelope by the blue door. I will return before sunrise.
This original dialogue exercise is included in the recording and narration brief. Record it in a voice you are permitted to use, listen to both complete sentences and save the approved audio. The text is a recording prompt, not supplied speech or a prediction of mouth timings.
Keep a quiet lead and tail in the approved recording and measure its actual duration. Do not trim the first word merely to fit an assumed shot length. If you change the performance and the spoken delivery together, save them as a new input pair so the earlier test remains reproducible.
Match the official checkpoint and command configuration
The upstream inference example identifies a UNet configuration, checkpoint, visual input, audio input and output path. Its inference source defines the required file arguments. This illustrative command uses placeholder media paths inside an already configured project environment:
python -m scripts.inference \
--unet_config_path configs/unet/stage2_512.yaml \
--inference_ckpt_path checkpoints/latentsync_unet.pt \
--video_path inputs/face.mp4 \
--audio_path inputs/speech.wav \
--video_out_path outputs/take.mp4It was not executed for this article. Follow the official setup for the version and device you will actually use. Save the repository revision, checkpoint identity, configuration, seed and any changed settings with the returned file. A successful command exit and an acceptable dialogue shot are separate checks.
Inspect the ComfyUI integration before loading a graph
A concrete community path is ShmuelRonen's ComfyUI LatentSync wrapper, which identifies itself as an unofficial version 1.6 implementation. Its example workflow folder is a starting point to inspect, not a graph we executed or a compatibility guarantee.
- Read the maintainer's current prerequisites and supported platform.
- Check the example graph's video loader, audio loader and required custom nodes.
- Map the permitted face video and approved audio into the LatentSync node.
- Record the node version, checkpoint, seed and settings before running.
- Locate the returned file in the actual output directory and review the full export.
The wrapper documents video and audio loaders feeding its LatentSync node and notes its own face and frame-rate constraints. Those are integration-specific instructions. Do not transfer an old graph's dependencies or a different wrapper's controls without checking them. Preserve an existing working setup and read the maintainer's upgrade notes before changing versions. For the broader local environment choice, use the local video workflow guide.
Use the original flash and click timing exercise
Before diagnosing speech, confirm that your review path can play known picture and sound events. These original four-second clips contain three white flashes and short synthesized tones. They contain no face or speech and are not LatentSync inputs, model outputs or a phoneme-alignment benchmark. Start playback deliberately and choose a comfortable listening level.
Aligned reference
Watch this clip and view its details
Open the aligned reference watch page or download the calibration clip.
Audio delayed reference
Watch this clip and view its details
Open the audio delayed reference watch page or download the calibration clip.
The picture streams decode identically: flashes begin at 1, 2 and 3 seconds. In the aligned fixture, clicks are authored at those times. In the delayed fixture, audio is authored 160 milliseconds later. Each file has 120 frames at 30 fps with mono audio. The separate aligned WAV and delayed WAV are calibration tones, not voice references.
Our scoped verification report checks the encoded files. A declared amplitude-threshold detector finds the aligned clicks about 0.7 milliseconds after the visual onset and the delayed clicks about 160.7 milliseconds after it. The small extra interval reflects the defined threshold on a faded tone, not a measured device-latency or speech-quality claim. Hardware playback latency and perceptual mouth alignment remain unmeasured.
Keep one offset convention in the manual log
The free log above uses visual event time minus audio event time. A negative result means the visual event occurs first. Using the delayed fixture's nominal authored times, enter audio 1.16 and visual 1 for the opening, audio 2.16 and visual 2 for the middle, and audio 3.16 and visual 3 for the ending. Each pair returns minus 160 milliseconds and the entered offset range is zero.
Those inputs are a declared calibration example, not manually measured mouth events. For your own speech clip, identify corresponding events carefully and use the same timeline origin. A mouth shape is not a perfect impulse marker, so preserve the uncertainty in your notes. Similar offsets can motivate a track-shift investigation; changing offsets need a closer timing review. Neither pattern proves its cause.
Leave an unobserved pair blank. Unknown should remain unknown rather than becoming zero. The log performs local arithmetic, makes no upload and does not alter your media. Save its report with the blank dialogue take ledger after entering actual observations.
Review mouth quality separately from file timing
Listen to the whole line without watching, then watch at normal speed. Confirm complete words, pauses and the return to rest. Inspect lips, teeth and chin around the important phrases, and compare identity and background with the source. A consistent whole-track offset does not explain every changing mouth boundary; an expressive mouth does not prove the audio track starts at the right time.
Record the first visible defect and whether it persists. Keep one accepted baseline while testing a single setting change. Do not replace an unclear recording, change the visual crop and switch checkpoints in the same comparison. That may produce a different take without explaining the improvement.
Return the accepted close-up to the drama
Watch the previous shot, the repaired close-up and the next reaction together. The character's line should create the intended response and fit the available pause. Use the AI short drama workflow and multi-shot guide for scene structure and coverage.
For generating an approved speech track from text and a permitted voice reference, see the IndexTTS narration guide. For another existing-visual-source model path, use the MuseTalk dialogue workflow. Explore Clout AI Cinema for its available project and scene workflow; this article does not establish LatentSync hosting or local-checkpoint import in Clout.
