feat(transcription): Add Faster Whisper transcription worker foundation and job queue - #288
feat(transcription): Add Faster Whisper transcription worker foundation and job queue#288dhruvi-16-me wants to merge 3 commits into
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. 🗂️ Base branches to auto review (2)
Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
Summary
This PR introduces the foundation for replacing the existing Vexa-based transcription ingestion pipeline with an internally managed transcription workflow using Faster Whisper.
The implementation adds a dedicated transcription job queue, meeting recording storage infrastructure, and a Python worker capable of processing queued recordings and writing transcripts back to the existing meetings pipeline.
The goal of this phase is to validate that meeting recordings can be transcribed without Vexa while preserving compatibility with the current downstream processing flow (
final_transcription, summarization, embeddings, and RAG).Screen Recording
Google Drive Link
Background
The current meeting intelligence pipeline relies on Vexa to generate transcripts:
This dependency introduces external service requirements for transcription.
To enable an internally managed transcription stack, I am introducing a new transcription ingestion layer based on Faster Whisper while keeping all downstream consumers unchanged.
Changes
Database
Added
transcription_jobstableIntroduced a dedicated queue table for transcription processing.
Responsibilities:
Job lifecycle:
Added meeting recordings storage support
Created storage infrastructure for uploaded recordings.
Bucket:
Path structure:
This structure aligns with existing team-based access controls and storage policies.
Faster Whisper Worker
Added a new worker:
Key files:
Worker Flow
meetings.transcription.Generated transcript format:
{ "segments": [ { "speaker": "Speaker", "text": "..." } ] }This format was intentionally chosen to remain compatible with existing database triggers and processing functions.
Validation
The implementation was validated end-to-end using a real uploaded audio file.
Validation Steps
test.m4a) tomeeting-recordings.transcription_jobs.Verified:
meetings.transcriptionpopulated.completed.final_transcription.Verified Pipeline
No modifications were required to existing transcript consumers.
Future Work
This PR establishes the transcription foundation only.
Upcoming work includes:
✅ Checklist