Connect the World, Frame by Frame

English｜简体中文｜繁體中文｜日本語｜Español｜Русский｜Français

🌟 Overview (Try VL Now!)

VideoLingo combines speech recognition, subtitle translation, segmentation and dubbing in a Streamlit interface. It produces subtitle files and optionally subtitled or dubbed videos. Translation quality depends on the source audio, language and chosen models.

Key features:

• 🎥 YouTube video download via yt-dlp

• Word-level speech recognition and alignment with WhisperX

• 📝 NLP and AI-powered subtitle segmentation

• 📚 Custom + AI-generated terminology for coherent translation

• Direct translation with optional reflection and natural rewriting

• Subtitle segmentation with configurable length limits

• 🗣️ Dubbing with GPT-SoVITS, Azure, OpenAI, and more

• 🚀 One-click startup and processing in Streamlit

• 🌍 Multi-language support in Streamlit UI

• 📝 Detailed logging with progress resumption

• 🔍 Model searchbox with API auto-fetch — search and filter from your provider's full model list

• ⏯️ Task control — pause, resume, or stop processing at any step

The workflow combines transcription, translation, subtitle layout and dubbing in one project.

🎥 Demo

Dual Subtitles trans.mp4

Cosy2 Voice Clone dubbing.mp4

GPT-SoVITS with my voice sovits.mp4

Language Support

Input Language Support(more to come):

🇺🇸 English 🤩 | 🇷🇺 Russian 😊 | 🇫🇷 French 🤩 | 🇩🇪 German 🤩 | 🇮🇹 Italian 🤩 | 🇪🇸 Spanish 🤩 | 🇯🇵 Japanese 😐 | 🇨🇳 Chinese* 😊

*For local Chinese recognition, explicitly select Chinese to use the punctuation-enhanced Belle Whisper model.

Translation languages depend on the selected LLM; dubbing languages depend on the selected TTS method.

Installation

Meet any problem? Chat with our free online AI agent here to help you.

Install Git, uv and FFmpeg first. Reopen your terminal after installation and check git --version, uv --version and ffmpeg -version.

For NVIDIA acceleration, install a driver compatible with your GPU. The host installer selects PyTorch cu128 when nvidia-smi reports CUDA >=12.8, otherwise cu126; without NVIDIA it selects CPU packages. This selects Python packages, not a system CUDA Toolkit. Local WhisperX GPU recognition also needs CUDA 12 cuBLAS and cuDNN 9 libraries available to the process; see GPU prerequisites.

Note: FFmpeg is required. Please install it via package managers:

• Windows: choose a shared-library build from the Windows builds linked on the FFmpeg download page, then add its bin directory to PATH.
• macOS: brew install ffmpeg (via Homebrew)
• Linux: sudo apt install ffmpeg (Debian/Ubuntu)

Install with uv

uv downloads Python 3.13 and creates an isolated .venv. No preinstalled Python is needed for the command below. The application supports Python 3.10–3.13. Use FFmpeg 7 shared libraries for the pinned TorchCodec 0.7; FFmpeg 8/9 alone is not compatible. See the verified Windows build.

1. Clone the repository

git clone https://github.com/Huanshere/VideoLingo.git
cd VideoLingo

2. Create the environment and install dependencies

uv run --no-project --python 3.13 setup_env.py

3. Start the application

.venv\Scripts\streamlit run st.py        # Windows
.venv/bin/streamlit run st.py            # macOS / Linux

Or double-click OneKeyStart.bat on Windows. It prefers ~/.venvs/videolingo when present, then the project .venv. Open http://localhost:8501 and enter your API URL, key and model in the sidebar.

Docker

For a Linux NVIDIA container deployment, install Docker, a compatible GPU driver and the NVIDIA Container Toolkit. The image uses the same Python 3.13 setup and application dependencies, with CUDA 12.8.1/cu128 by default. See Docker docs for the matched CUDA 12.6 alternative and persistence settings.

docker build -t videolingo .
docker run -d -p 8501:8501 --gpus all videolingo

APIs

VideoLingo supports OpenAI-Like API format and various TTS interfaces:

• LLM: choose an OpenAI-compatible Chat Completions provider and model that can return the structured JSON required by the workflow. Configure the API URL, key and model in the sidebar.
• Speech recognition: run WhisperX locally or use the ElevenLabs API.
• TTS: Azure, OpenAI, Fish TTS, SiliconFlow Fish/CosyVoice2, GPT-SoVITS, Edge TTS, F5-TTS and a custom adapter in core/tts_backend/custom_tts.py.

For detailed installation, API configuration, and batch mode instructions, please refer to the documentation: English | 中文

Current Limitations

1. Background noise and language-specific alignment models affect recognition and word timestamps. Vocal separation may help. Numbers and symbols may lack reliable word timings; inspect the resulting subtitles.

2. LLM output must satisfy the workflow's JSON structure. For failures, inspect output/gpt_log/error.json. Existing successful response caches and completed outputs can be reused on retry; changing the model alone does not regenerate every completed stage. Do not delete all output as the first troubleshooting step.

3. Dubbing quality and timing depend on translation, the TTS service and speech rate. Speed adjustment does not guarantee natural delivery or perfect synchronization.

4. Local WhisperX uses one recognition/alignment language per segment. Mixed-language speech is not guaranteed to retain accurate text and timing in every language.

5. The dubbing workflow does not automatically assign a separate voice to each speaker.

📄 License

This project is licensed under the Apache 2.0 License. Special thanks to the following open source projects for their contributions:

whisperX, yt-dlp, json_repair, BELLE

📬 Contact Me

• Submit Issues or Pull Requests on GitHub
• DM me on Twitter: @Huanshere
• Email me at: team@videolingo.io

⭐ Star History

---

If you find VideoLingo helpful, please give me a ⭐️!