Skip to content

Conversation

@shiragannavar
Copy link

Add character-level alignment metadata support to ElevenLabs TTS plugin for accurate lip sync animation in avatar applications.

Changes:

  • Add CharacterAlignment dataclass to core TTS module
  • Add optional alignment field to SynthesizedAudio
  • Update ElevenLabs plugin to capture and forward alignment data
  • Add enable_alignment_forwarding parameter (default: True)
  • Emit 'alignment_received' events with character timing data
  • Add example application demonstrating lip sync integration
  • Add comprehensive documentation for client-side integration

The feature is fully backward compatible:

  • Existing code without alignment handlers continues to work
  • Can be disabled by setting enable_alignment_forwarding=False
  • No breaking changes to existing TTS functionality

Resolves lip sync timing requirements for avatar applications by providing:

  • Per-character start and end times in seconds
  • LiveKit data channel integration for client delivery
  • Event-based architecture for flexible consumption

Add character-level alignment metadata support to ElevenLabs TTS plugin
for accurate lip sync animation in avatar applications.

Changes:
- Add CharacterAlignment dataclass to core TTS module
- Add optional alignment field to SynthesizedAudio
- Update ElevenLabs plugin to capture and forward alignment data
- Add enable_alignment_forwarding parameter (default: True)
- Emit 'alignment_received' events with character timing data
- Add example application demonstrating lip sync integration
- Add comprehensive documentation for client-side integration

The feature is fully backward compatible:
- Existing code without alignment handlers continues to work
- Can be disabled by setting enable_alignment_forwarding=False
- No breaking changes to existing TTS functionality

Resolves lip sync timing requirements for avatar applications by providing:
- Per-character start and end times in seconds
- LiveKit data channel integration for client delivery
- Event-based architecture for flexible consumption
@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants