Skip to content

feat(pai-voice-system): Add Linux support and CLI configuration #351

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
sti0 wants to merge 1 commit into from
Closed

feat(pai-voice-system): Add Linux support and CLI configuration #351

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
80 changes: 74 additions & 6 deletions Packs/pai-voice-system/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
---
name: PAI Voice System
pack-id: danielmiessler-pai-voice-system-v1.0.1
version: 1.0.1
author: danielmiessler
pack-id: danielmiessler-pai-voice-system-v1.1.0
version: 1.1.0
author: danielmiessler, sti0
description: Voice notification system with ElevenLabs TTS, prosody enhancement for natural speech, and agent personality-driven voice delivery
type: feature
purpose-type: [notifications, accessibility, automation]
platform: macos
platform: macos, linux
dependencies:
- pai-hook-system (required) - Hooks trigger voice notifications
- pai-core-install (required) - Skills, identity, and response format drive voice output
Expand All @@ -30,9 +30,11 @@ keywords: [voice, tts, elevenlabs, notifications, prosody, speech, agents, perso
| Platform | Status | Notes |
|----------|--------|-------|
| **macOS** | ✅ Fully Supported | Uses `afplay` (built-in) for audio playback |
| **Linux** | ⚠️ Experimental | Requires audio player modification |
| **Linux** | ✅ Fully Supported | Auto-detects `mpg123` or `mpv` for audio playback |
| **Windows** | ❌ Not Supported | No current implementation |

**Audio player priority (Linux):** mpg123 → mpv (install one: `sudo apt install mpg123`)

---

## What This Pack Provides
Expand All @@ -44,6 +46,63 @@ keywords: [voice, tts, elevenlabs, notifications, prosody, speech, agents, perso
- **Intelligent Cleaning**: Strips code blocks and artifacts for clean speech
- **Graceful Degradation**: Works silently when voice server is offline

## Audio Player Configuration

Configure extra arguments for audio players (useful for containers or specific audio setups):

**Environment Variable:**
```bash
# In $PAI_DIR/.env
PAI_VOICE_SERVER_EXTRA_ARGS="-o pulse"
```

**CLI Flag (overrides env var):**
```bash
bun run server.ts --extra-args="-o pulse"
```

**Common use cases:**
| Use Case | Configuration |
|----------|---------------|
| Container with PulseAudio | `PAI_VOICE_SERVER_EXTRA_ARGS="-o pulse"` |
| Specific ALSA device | `PAI_VOICE_SERVER_EXTRA_ARGS="-o alsa -a hw:1,0"` |

### Devcontainer Setup

To use voice notifications inside a devcontainer:

1. **Mount PulseAudio socket** from host in `devcontainer.json`
2. **Set `PAI_VOICE_SERVER_EXTRA_ARGS`** in your `.env` file

**Example `devcontainer.json`:**
```json
{
"name": "My Devcontainer",
"image": "mcr.microsoft.com/devcontainers/python:2-3.12-bullseye",
"mounts": [
"source=/run/user/1000/pulse,target=/run/user/1000/pulse,type=bind"
],
"runArgs": [
"--device=/dev/snd:/dev/snd"
],
"containerEnv": {
"PULSE_SERVER": "unix:/run/user/1000/pulse/native"
}
}
```

**In your `$PAI_DIR/.env`:**
```bash
PAI_VOICE_SERVER_EXTRA_ARGS="-o pulse"
```

**Key points:**
- Mount `/run/user/1000/pulse` to access host's PulseAudio
- Set `PULSE_SERVER` in container so audio apps find the socket
- Set `PAI_VOICE_SERVER_EXTRA_ARGS` in `.env` (not devcontainer.json)

---

## Voice Server

The voice server runs locally on port 8888 and:
Expand Down Expand Up @@ -154,7 +213,8 @@ These markers are embedded in the message and the voice server adjusts stability
|----------|----------|---------|-------------|
| `ELEVENLABS_API_KEY` | Yes | - | Your ElevenLabs API key |
| `ELEVENLABS_VOICE_ID` | Yes | - | Default voice ID for TTS |
| `VOICE_SERVER_PORT` | No | 8888 | Voice server port |
| `PAI_VOICE_SERVER_PORT` | No | 8888 | Voice server port |
| `PAI_VOICE_SERVER_EXTRA_ARGS` | No | - | Extra arguments for audio player |
| `VOICE_SERVER_URL` | No | http://localhost:8888 | Voice server URL (for hooks) |
| `PAI_DIR` | No | ~/.config/pai | PAI installation directory |

Expand Down Expand Up @@ -195,6 +255,14 @@ Configure multiple voices in `voice-personalities.json` for multi-agent conversa

## Changelog

### 1.1.0 - 2026-01-09
- Added Linux audio support (mpg123, mpv auto-detection)
- Added Linux notifications via notify-send
- Added CLI arguments: `--port/-p`, `--extra-args`
- Added `PAI_VOICE_SERVER_PORT` and `PAI_VOICE_SERVER_EXTRA_ARGS` env vars
- Enhanced .env loading from `$PAI_DIR/.env` with quote stripping
- Added devcontainer/PulseAudio documentation

### 1.0.1 - 2026-01-09
- **Documentation fixes**: INSTALL.md and VERIFY.md now correctly reference actual files
- Fixed: References to non-existent `start.sh`, `stop.sh`, `restart.sh`, `status.sh` → use `manage.sh`
Expand Down
154 changes: 105 additions & 49 deletions Packs/pai-voice-system/src/voice/server.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,20 @@
* PAI Voice Server - Text-to-Speech notification server using ElevenLabs
*
* Part of the pai-voice-system pack.
* Platforms: macOS, Linux
*
* Usage:
* bun run src/voice/server.ts
* bun run src/voice/server.ts [options]
*
* CLI Options:
* --port, -p <port> Server port (overrides environment variables)
* --extra-args <args> Extra arguments to pass to audio player (space-separated)
*
* Environment Variables:
* ELEVENLABS_API_KEY - Your ElevenLabs API key (required)
* ELEVENLABS_VOICE_ID - Default voice ID (optional)
* VOICE_SERVER_PORT - Server port (default: 8888)
* PAI_VOICE_SERVER_PORT - Server port (default: 8888)
* PAI_VOICE_SERVER_EXTRA_ARGS - Extra arguments for audio player
* PAI_DIR - PAI installation directory (default: ~/.config/pai)
*
* Endpoints:
Expand All @@ -20,32 +26,80 @@
*/

import { serve } from "bun";
import { spawn } from "child_process";
import { spawn, execSync } from "child_process";
import { parseArgs } from "util";
import { homedir } from "os";
import { join } from "path";
import { existsSync, readFileSync } from "fs";

// Load .env from user home directory
const envPath = join(homedir(), '.env');
// Load .env from PAI_DIR (defaults to ~/.config/pai)
const paiDir = process.env.PAI_DIR || join(homedir(), '.config', 'pai');
const envPath = join(paiDir, '.env');
if (existsSync(envPath)) {
const envContent = await Bun.file(envPath).text();
envContent.split('\n').forEach(line => {
const [key, value] = line.split('=');
if (key && value && !key.startsWith('#')) {
process.env[key.trim()] = value.trim();
const eqIndex = line.indexOf('=');
if (eqIndex === -1) return;
const key = line.slice(0, eqIndex).trim();
let value = line.slice(eqIndex + 1).trim();
// Strip surrounding quotes from value
if ((value.startsWith('"') && value.endsWith('"')) ||
(value.startsWith("'") && value.endsWith("'"))) {
value = value.slice(1, -1);
}
if (key && !key.startsWith('#')) {
process.env[key] = value;
}
});
}

const PORT = parseInt(process.env.VOICE_SERVER_PORT || process.env.PORT || "8888");
// CLI argument parsing
const cliArgs = parseArgs({
args: process.argv.slice(2),
options: {
'extra-args': { type: 'string', default: '' },
port: { type: 'string', short: 'p' },
},
allowPositionals: true,
});

const PORT = parseInt(cliArgs.values.port || process.env.PAI_VOICE_SERVER_PORT || process.env.PORT || "8888");
const ELEVENLABS_API_KEY = process.env.ELEVENLABS_API_KEY;
const PAI_DIR = process.env.PAI_DIR || join(homedir(), '.config', 'pai');
const PAI_DIR = paiDir;

if (!ELEVENLABS_API_KEY) {
console.error('⚠️ ELEVENLABS_API_KEY not found in ~/.env');
console.error(`⚠️ ELEVENLABS_API_KEY not found in ${envPath}`);
console.error('Add: ELEVENLABS_API_KEY=your_key_here');
}

// Audio player detection for cross-platform support
function findPlayer(name: string): string | null {
try {
return execSync(`which ${name}`, { encoding: 'utf8' }).trim() || null;
} catch {
return null;
}
}

function detectPlayer(): { path: string | null; type: 'afplay' | 'mpg123' | 'mpv' | null } {
if (process.platform === 'darwin') {
return { path: '/usr/bin/afplay', type: 'afplay' };
}
const mpg123 = findPlayer('mpg123');
if (mpg123) return { path: mpg123, type: 'mpg123' };
const mpv = findPlayer('mpv');
if (mpv) return { path: mpv, type: 'mpv' };
return { path: null, type: null };
}

const DETECTED_PLAYER = detectPlayer();

// Get extra arguments for audio player from CLI or environment
function getExtraArgs(): string[] {
const raw = cliArgs.values['extra-args'] || process.env.PAI_VOICE_SERVER_EXTRA_ARGS || '';
return raw.trim() ? raw.trim().split(/\s+/) : [];
}

// Default voice ID - configure via environment variable
const DEFAULT_VOICE_ID = process.env.ELEVENLABS_VOICE_ID || "";

Expand Down Expand Up @@ -128,10 +182,6 @@ try {
console.warn('⚠️ Failed to load voice personalities, using defaults');
}

// Escape special characters for AppleScript
function escapeForAppleScript(input: string): string {
return input.replace(/\\/g, '\\\\').replace(/"/g, '\\"');
}

// Extract emotional marker from message
// Supports all 13 prosody markers from the expanded system
Expand Down Expand Up @@ -281,57 +331,53 @@ function getVolumeSetting(): number {
return 1.0; // Default to full volume
}

// Play audio using afplay (macOS)
// Play audio using platform-appropriate player (macOS: afplay, Linux: mpg123/mpv)
async function playAudio(audioBuffer: ArrayBuffer): Promise<void> {
const tempFile = `/tmp/voice-${Date.now()}.mp3`;

// Write audio to temp file
await Bun.write(tempFile, audioBuffer);

const volume = getVolumeSetting();

return new Promise((resolve, reject) => {
// afplay -v takes a value from 0.0 to 1.0
const proc = spawn('/usr/bin/afplay', ['-v', volume.toString(), tempFile]);

proc.on('error', (error) => {
console.error('Error playing audio:', error);
reject(error);
});

proc.on('exit', (code) => {
// Clean up temp file
if (!DETECTED_PLAYER.path) {
console.warn('⚠️ No audio player found. Install mpg123 or mpv for audio playback.');
spawn('/bin/rm', [tempFile]);
resolve();
return;
}

if (code === 0) {
resolve();
} else {
reject(new Error(`afplay exited with code ${code}`));
}
});
});
}
const player = DETECTED_PLAYER.path;
let args: string[];

if (DETECTED_PLAYER.type === 'afplay') {
args = ['-v', volume.toString(), ...getExtraArgs(), tempFile];
} else if (DETECTED_PLAYER.type === 'mpg123') {
// mpg123 doesn't support volume control via CLI args; use system mixer instead
args = ['-q', ...getExtraArgs(), tempFile];
} else {
// mpv
args = ['--no-terminal', '--volume=' + (volume * 100), ...getExtraArgs(), tempFile];
}

// Spawn a process safely
function spawnSafe(command: string, args: string[]): Promise<void> {
return new Promise((resolve, reject) => {
const proc = spawn(command, args);
const proc = spawn(player, args);

proc.on('error', (error) => {
console.error(`Error spawning ${command}:`, error);
console.error('Error playing audio:', error);
spawn('/bin/rm', [tempFile]);
reject(error);
});

proc.on('exit', (code) => {
if (code === 0) {
spawn('/bin/rm', [tempFile]);
if (code === 0 || code === null) {
resolve();
} else {
reject(new Error(`${command} exited with code ${code}`));
reject(new Error(`${player} exited with code ${code}`));
}
});
});
}


// Send macOS notification with voice
async function sendNotification(
title: string,
Expand Down Expand Up @@ -392,12 +438,16 @@ async function sendNotification(
}
}

// Display macOS notification - escape for AppleScript
// Display notification - platform-specific
try {
const escapedTitle = escapeForAppleScript(safeTitle);
const escapedMessage = escapeForAppleScript(safeMessage);
const script = `display notification "${escapedMessage}" with title "${escapedTitle}" sound name ""`;
await spawnSafe('/usr/bin/osascript', ['-e', script]);
if (process.platform === 'linux') {
spawn('/usr/bin/notify-send', [safeTitle, safeMessage]);
} else if (process.platform === 'darwin') {
const escapedTitle = safeTitle.replace(/\\/g, '\\\\').replace(/"/g, '\\"');
const escapedMessage = safeMessage.replace(/\\/g, '\\\\').replace(/"/g, '\\"');
const script = `display notification "${escapedMessage}" with title "${escapedTitle}" sound name ""`;
spawn('/usr/bin/osascript', ['-e', script]);
}
} catch (error) {
console.error("Notification display error:", error);
}
Expand Down Expand Up @@ -545,6 +595,12 @@ const server = serve({
});

console.log(`🚀 PAI Voice Server running on port ${PORT}`);
console.log(`🖥️ Platform: ${process.platform}`);
console.log(`🔊 Audio player: ${DETECTED_PLAYER.path || '❌ Not found (install mpg123 or mpv)'}`);
const extraArgs = getExtraArgs();
if (extraArgs.length > 0) {
console.log(`🎛️ Extra player args: ${extraArgs.join(' ')}`);
}
console.log(`🎙️ Using ElevenLabs TTS`);
console.log(`🔊 Default voice: ${DEFAULT_VOICE_ID || '(not configured - set ELEVENLABS_VOICE_ID)'}`);
console.log(`📡 POST to http://localhost:${PORT}/notify`);
Expand Down
Loading