What is podcast_dub?

This project takes a video podcast in one language and produces a version dubbed into another language, using only open-weight models that you can run yourself. The pipeline figures out who is speaking and when, translates the conversation, clones each speaker's voice, and generates new spoken audio that lines up with the original timing. The original audio stays underneath the new voices but is lowered in volume so the dub is clear. The pipeline works in several stages. It extracts audio from the video, uses a speech recognition model to get precise word and phrase timing, then separates up to four speakers. It mines clean reference audio for each speaker, translates the text with an LLM, synthesizes new speech with cloned voices, and places everything back into a final video. The placement stage includes checks for coverage and dead air, and it only publishes the finished video after verification passes. To run it, you need Python 3.12, a package manager called uv, and FFmpeg. You clone the repository, run a sync command, and then point the tool at a video file with a source and target language. The first run downloads model weights, which can take a while. You can also use a TOML job file for repeatable runs, where you specify the video, languages, output path, speaker names, proper nouns, and translation context. Rerunning a job reuses intermediate results from previous stages. Translation defaults to a model called Kimi K3 through Moonshot's API, which requires an API key. You can swap in other OpenAI-compatible services by setting environment variables for the endpoint, model, and key. Anthropic's native API is not supported directly, you need a compatible gateway. The tool runs on macOS with Apple Silicon or Linux with NVIDIA GPUs, and it picks the right accelerator for each stage automatically. Supported target languages include Chinese, English, Japanese, Korean, German, French, Russian, Portuguese, Spanish, and Italian. The full README is longer than what was shown.