Skip to content

development.md

Latest commit

 

History

History
141 lines (91 loc) · 6.82 KB

development.md

File metadata and controls

141 lines (91 loc) · 6.82 KB
Raw

NiklasBot — Development Notes

Current State

A single-file Discord bot in index.js with ambient voice AI, text chat, and Fortnite integration. Runs on Node.js. Uses Groq (free tier) for chat and transcription, OpenAI for TTS.

Issues — Status

✅ Resolved

#2 Python subprocess for Whisper — Replaced with Groq whisper-large-v3-turbo API. No Python dependency.

#3 Fixed-interval recording / no VAD — Replaced the fixed-timer record/respond loop with continuous per-utterance listening (receiver.speaking start/end drives capture; EndBehaviorType.AfterSilence ends each person's utterance after ~1.3s instead of a flat 5s batch window). Two reply triggers share one output gate:

  • Addressed — fires after every finished utterance; responds immediately if the model judges it was talked to, else returns the literal token SILENT and nothing plays.
  • Lull — polled every 2s; only fires once the whole channel's been quiet for lullThresholdMs (per-guild, /setpatience, default 9s) and a 25s ambient cooldown has passed since the last unprompted comment. Heavily biased toward silence (see prompt.txt).
  • Output gate (speakWhenClear) — holds any reply until nobody's currently speaking (max 8s wait), so Nikbot doesn't talk over people.
  • Barge-in — if anyone starts talking while Nikbot is mid-reply, its AudioPlayer is stopped immediately.

/setrecord and /setrepeat are gone, replaced by the single /setpatience (lull threshold) setting — there's no more fixed recording window to configure.

#4 Global settings — Replaced with per-guild guildSettings.json. /setpatience is server-scoped.

#6 Dual command system — All ! prefix commands removed. Slash commands only.

#7 Outdated AI model — Switched to llama-3.3-70b-versatile via Groq (free tier).

#8 Google TTS quality — Replaced gtts with OpenAI tts-1. Removed gtts from package.json.

#9 No voice reconnection — Added VoiceConnectionStatus.Disconnected handler. Attempts reconnect for 25s, then cleans up and notifies the channel.

#11 Duplicate slash command registration — Extracted SLASH_COMMANDS constant shared by registerSlashCommands() and guildCreate.

#12 No rate limiting — Added 5-second per-user cooldown on /message-nikbot with ephemeral feedback.

#16 README out of sync — Rewritten to match the current command set and ambient-listening behavior.

#5 In-memory text history resets on restart — Replaced the textMemory Map with SQLite persistence (lib/textMemoryStore.js, using the built-in node:sqlite module — see DECISIONS.md for why better-sqlite3 wasn't needed). Same 20-exchange window, survives restarts.

❌ Remaining

#1 Monolithic index.js (critical) Everything still lives in one file. Phase 1 refactor (extract services, commands, events into src/) is deferred until other features stabilize.

#10 Audio filename collision risk (minor) PCM files use millisecond timestamps now — collision risk is negligible but not zero. Can append a random suffix if it ever causes issues.

#13 changingPrompt.txt is shared global state (minor) The fallback prompt for guilds without a custom prompt is loaded from disk. Fine for single-server use; would need to be seeded per-guild for multi-tenant use.

#14 No structured logging (minor) All output is console.log. Add pino for log levels and timestamps when cloud hosting becomes a priority.

#15 Stale voice memory on crash (minor) Voice memory is cleared on clean stop but not on crash. Low priority since voice memory is intentionally session-scoped.

#17 Ambient reply cadence is untuned (needs real testing) lullThresholdMs (9s default), AMBIENT_COOLDOWN_MS (25s), and PER_USER_SILENCE_MS (1.3s) are reasonable starting guesses, not measured. The "should I say something" prompting in prompt.txt also hasn't been tested live — expect it to need iteration once there's an actual multi-person call to test against (too chatty / too quiet / wrong moments).

Overhaul Plan — Phase Status

Phase Description Status
2 Per-guild settings ✅ Done
3 Replace Python transcription ✅ Done
4 Replace gtts with OpenAI TTS ✅ Done
5 Slash commands only ✅ Done
6 Continuous listening + ambient replies ✅ Done (untuned — see #17)
1 Modular src/ structure ❌ Pending

Future Development Plans

Short Term — Next Up

Live-tune ambient cadence (#17) Test in an actual call with multiple people; adjust lullThresholdMs, AMBIENT_COOLDOWN_MS, and the prompt.txt silence bias based on how it actually feels.

Medium Term

Custom voice (cheap + fast) Currently OpenAI tts-1 with a fixed voice. Cartesia Sonic is the cheapest low-latency option with real voice cloning if/when a custom voice becomes a priority — parked for now by choice, not a technical blocker.

Web dashboard Simple Express status page: active guilds, current prompts, memory counts. Add controls later.

League of Legends / Valorant integration

  • /rank <summoner> — LoL rank via Riot API
  • /valorant <username> — Valorant rank/stats

Long Term

Multi-bot personas — Multiple app IDs, same codebase, different prompts/voices.

Image generation — /imagine <prompt> via a free image API (Pollinations.ai or similar).

Scheduled reminders — /remind @user <time> <message> stored in SQLite.

Music playback — /play <YouTube URL> using ytdl-core + existing voice pipeline.

Dependency Status

Package Purpose Status
discord.js Discord framework Active
@discordjs/voice Voice channel support Active
@discordjs/opus Audio encoding Active
openai TTS only (tts-1) Active
fluent-ffmpeg PCM → WAV conversion Active
ffmpeg-static FFmpeg binary Active
prism-media Opus decoding Active
dotenv Env config Active
gtts Google TTS ✅ Removed
node:sqlite (built-in) Persistent text memory Active
pino Structured logging ❌ Not yet added

Environment Variables

DISCORD_TOKEN=          # Discord bot token (discord.com/developers/applications → Bot)
APPLICATION_ID=         # Discord application ID (General Information)
GROQ_API_KEY=           # Groq API key — free tier (console.groq.com)
OPENAI_API_KEY=         # OpenAI key — TTS only (platform.openai.com)
FORTNITE_API_KEY=       # Optional (fortnite-api.com)

Hosting Notes

Oracle VM compatibility is maintained via safeDeleteFile, pre-created audio/ directory, and the 10-minute orphaned file cleanup interval. Keep these regardless of host. Not a priority right now — testing locally against Discord first, hosting decision deferred until the bot's behavior is solid.