🎬 ClipCast AI - Turn Podcasts into Viral Shorts

The ultimate AI-powered tool to transform long-form podcasts into viral vertical clips for TikTok, Reels, and YouTube Shorts.

---

🚀 Overview

ClipCast AI is an end-to-end platform that automates the tedious process of podcast clipping. Instead of manually scrubbing through hours of footage, ClipCast AI uses state-of-the-art AI to identify clinical "viral" moments, transcribe them with frame-perfect accuracy, and re-frame the video for vertical platforms using intelligent face tracking.

✨ Key Features

• 🧠 AI-Driven Moment Detection: Uses Gemini 2.5 Flash to analyze transcripts and find the most engaging stories, questions, and answers.
• 🎙️ High-Precision Transcription: Powered by WhisperX for word-level timestamps and alignment.
• 👤 Smart Face Tracking: Automatically detects the active speaker and crops the video (9:16) to keep the focus where it belongs.
• ✍️ Dynamic Subtitles: Automatically generates stylized, readable captions with the Anton font.
• ⚡ Serverless GPU Processing: Heavy lifting is handled by Modal using NVIDIA L40S GPUs for blazing-fast rendering.
• 💳 Subscription Ready: Integrated with Stripe for tiered access and payments.
• ☁️ Cloud Native: Scalable storage using AWS S3 and database management with Prisma & Supabase.

🛠️ Tech Stack

Frontend

• Framework: Next.js 15 (App Router)
• Styling: Tailwind CSS + Shadcn UI
• State Management: React Hooks + Inngest for background flows
• Auth: NextAuth.js (v5 Beta)
• Database: Prisma with PostgreSQL (Supabase)

Backend

• Core: FastAPI + Python 3.12
• Infrastructure: Modal (Serverless GPUs)
• AI Models:
  • Transcription: OpenAI WhisperX
  • LLM: Google Gemini 2.5 Flash
• Processing: OpenCV, FFmpeg, NumPy, PySubs2

🧬 How it Works

1. Upload: User uploads a 16:9 podcast video to AWS S3.
2. Transcribe: WhisperX generates a word-level transcript with exact timestamps.
3. Analyze: Gemini analyzes the transcript to find engaging 30-60 second segments.
4. Reframing: For each segment, the backend tracks faces and crops the video to vertical.
5. Styling: Styled subtitles are burned into the video.
6. Delivery: The final .mp4 is uploaded back to S3 and served via the dashboard.

⚙️ Setup & Installation

Backend Setup

1. Navigate to the backend directory:

   cd ai-podcast-clipper-backend

2. Install Modal CLI:

   pip install modal
   modal token set

3. Deploy the backend:

   modal deploy main.py

4. Configuration: Ensure you have a Modal secret named ai-podcast-clipper-secret containing GEMINI_API_KEY and AUTH_TOKEN.

Frontend Setup

1. Navigate to the frontend directory:

   cd ai-podcast-clipper-frontend

2. Install dependencies:

   npm install

3. Setup environment variables:
   • Create a .env file based on .env.example.
   • Add your DATABASE_URL, AUTH_SECRET, AWS credentials, and STRIPE keys.
4. Push database schema:

   npx prisma db push

5. Run the development server:

   npm run dev

📜 License

This project is licensed under the MIT License.

---

Made with ❤️ by Aaditya