jarvis

Clap twice, or say a wake phrase, and your laptop runs a command.

No cloud, no account, no always-on transcription. It is one Python file with no dependencies outside the standard library, listening to a single microphone stream. Linux out of the box, macOS and Windows with two config lines.

Idle cost is one max() per 16 ms of audio — about 0.3% of a core and 16 MB of RAM. Speech recognition only ever runs on a captured segment of speech, never continuously.

                     ┌── loud spike, then another 0.1–0.7 s later ──→ "clap"
mic ─→ 16 ms frames ─┤
                     └── speech segment ─→ transcribe ─→ fuzzy match ─→ "voice"
                                                                          │
                                          play greeting ←─────────────────┤
                                          run your command ←──────────────┘

What it is for

Anything you would otherwise reach for a keyboard shortcut to do, when your hands are busy or you just walked in:

JARVIS_ACTION=hyprctl dispatch exec brave
JARVIS_ACTION=~/bin/start-my-dev-environment
JARVIS_ACTION=playerctl play-pause
JARVIS_ACTION=wtype -M ctrl -M shift -k p        # drive an app that has no CLI

The command runs through sh, so pipes, && and environment variables all work. JARVIS_TRIGGER is set to clap or voice in its environment, so a single script can tell the two apart:

#!/bin/sh
[ "$JARVIS_TRIGGER" = voice ] && notify-send "you rang?"
exec my-real-command

Requirements

• Python 3.8+ — standard library only, no pip install
• a recorder and a player — pw-record and pw-play on Linux by default, already present on most current desktops. Both are swappable, see Other platforms
• systemd user session — for the supplied service file; running jarvis by hand works too
• (optional) a speech-to-text engine, for the wake phrase. Without one, claps still work and the phrase quietly turns itself off.
• (optional) uv for the spoken reply, which is rendered by piper

Install

git clone https://github.com/jamolweb/jarvis-v1
cd jarvis-v1
./install.sh

That copies jarvis to ~/.local/bin, the unit to ~/.config/systemd/user, and an annotated config to ~/.config/jarvis/config.env. Then:

$EDITOR ~/.config/jarvis/config.env   # set JARVIS_ACTION
jarvis --doctor                       # calibrate to your microphone
systemctl --user enable --now jarvis

Uninstall is systemctl --user disable --now jarvis and deleting those three paths.

The wake phrase

Set the engine and the phrase in ~/.config/jarvis/config.env:

JARVIS_PHRASE=wake up daddys home
JARVIS_STT_CMD=whisper-cli --model {model} --file {wav} --no-timestamps --language en
JARVIS_STT_MODEL=~/.local/share/jarvis/ggml-tiny.en.bin

JARVIS_STT_CMD is any command that prints the transcript as the last line of stdout; {wav} and {model} are substituted and shell-quoted. So whisper.cpp is the default, but anything else drops in as a one-line change:

JARVIS_STT_CMD=voxtype -q --model tiny.en --language en transcribe {wav}

Use a small model. tiny.en is the right choice: it runs in a second or two on CPU and you are matching one known phrase, not dictating prose.

Which is also why matching is fuzzy. Small models mishear — tiny.en renders "wake up daddy's home" as "Wake up, that is home." — so an exact string compare would almost never fire. Every window of the transcript is scored against the phrase and anything over JARVIS_MATCH (default 0.65) counts. Check what your own voice scores without triggering anything:

jarvis --check some-recording.wav
heard: 'Wake up, that is home.'  score: 0.77  (need 0.65)

Pick a phrase of three or more syllables that you would not say in normal conversation. Two-word phrases false-trigger.

The spoken reply

jarvis --say "Welcome home sir"

Renders once to a wav with piper, so triggering it later is just playback and costs nothing. Change the voice with JARVIS_TTS_VOICE (any piper voice; the model downloads itself on first use). With no wav rendered, JARVIS_GREETING is shown as a desktop notification instead.

The greeting plays before your command starts, and blocks until it finishes. That is deliberate: an action that spawns a pile of processes will starve the audio client of its startup, and the reply lands half a minute late.

Not triggering twice

JARVIS_GUARD is a command whose exit status 0 means "already running, ignore this". It runs before every trigger.

JARVIS_GUARD=pgrep -x my-app
JARVIS_GUARD=hyprctl clients | grep -q 'initialTitle: dev-'

There is also a 3 second lockout after any trigger, so a burst of applause fires once.

Calibration

Detection thresholds are absolute PCM levels, so they depend on your microphone, its gain, and how noisy the room is. There is no universal number. jarvis --doctor measures yours:

Volume: 0.23
listening for 5 seconds, stay quiet...
  ambient frame peaks: p50 978  p90 1505  p99 2059  max 2371

suggested for this room:
  JARVIS_VOICE_MIN_PEAK=3088
  JARVIS_CLAP_MIN_PEAK=7206

Put those in your config. To watch what it actually hears, stop the service and run it in the foreground — every frame peak, every transcript, every score:

systemctl --user stop jarvis
jarvis --debug

Three knobs matter:

knob	meaning	if it is wrong
JARVIS_VOICE_MIN_PEAK	speech loudness floor	too high: your voice is never captured. too low: constant pointless transcription
JARVIS_CLAP_MIN_PEAK	clap loudness floor	too low: keystrokes and door slams count as claps
JARVIS_CLAP_RATIO	how far a clap must rise above the running background	too low: false claps in a noisy room

When it stops working

Check these in order — the first two account for nearly everything.

1. Is the microphone muted? wpctl get-volume @DEFAULT_AUDIO_SOURCE@. A muted source delivers digital silence, and every symptom looks like a bug in the detector. Many laptops have a dedicated mic-mute key that is easy to hit by accident. jarvis --doctor calls this out, and after a solid minute of perfectly zero samples the running service says so too.
2. Is the input gain saturated? Laptop mics clip badly above roughly 25%. Past that, every frame pegs at full scale, the background estimate climbs to meet it, and nothing is ever loud enough to stand out. Counter-intuitively, lowering the gain fixes a mic that stopped hearing you.
3. Is it listening to the right device? PipeWire's default source can flip to a virtual or loopback input that carries no audio. Pin it: pw-record --list-targets, then JARVIS_MIC=<name>.
4. Read the log. journalctl --user -u jarvis -f on Linux, /tmp/jarvis.log under the launchd agent on macOS.

Steps 1 to 3 are written with Linux tooling; the underlying causes are the same everywhere, checked through the sound settings of your own platform.

Other platforms

Only two pieces of jarvis are platform-specific: what records the microphone and what plays a wav. Both are config values, so the rest — detection, matching, the action, the guard — is the same everywhere.

The recorder must write raw signed 16-bit mono PCM at 16000 Hz to stdout, and the player must play {wav} and exit.

Linux

Works as installed. PipeWire's pw-record and pw-play are the defaults, and install.sh sets up a systemd user service. On a PulseAudio-only system, swap in parecord --rate=16000 --channels=1 --format=s16le and paplay {wav}.

macOS

brew install ffmpeg, then find your microphone's index:

ffmpeg -f avfoundation -list_devices true -i ""

In ~/.config/jarvis/config.env:

JARVIS_REC_CMD=ffmpeg -hide_banner -loglevel error -f avfoundation -i :0 -ac 1 -ar 16000 -f s16le -
JARVIS_PLAY_CMD=afplay {wav}

macOS has a speech synthesiser built in, so you do not need piper for the spoken reply — any audio file works, the extension is irrelevant:

say -o ~/.local/share/jarvis/greeting.aiff "Welcome home sir"

JARVIS_GREET_WAV=~/.local/share/jarvis/greeting.aiff

Speech to text is brew install whisper-cpp, which provides whisper-cli.

Run jarvis once from Terminal by hand before setting up autostart. macOS gates microphone access per application, and that first run is what triggers the permission prompt. A background agent that has never been granted access just receives silence — which looks exactly like a broken detector. Check System Settings → Privacy & Security → Microphone if no prompt appears.

There is no systemd, so autostart is a launchd agent at ~/Library/LaunchAgents/local.jarvis.plist:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>Label</key><string>local.jarvis</string>
  <key>ProgramArguments</key>
  <array>
    <string>/usr/bin/python3</string>
    <string>/Users/YOU/.local/bin/jarvis</string>
  </array>
  <key>RunAtLoad</key><true/>
  <key>KeepAlive</key><true/>
  <key>StandardErrorPath</key><string>/tmp/jarvis.log</string>
</dict>
</plist>

launchctl load ~/Library/LaunchAgents/local.jarvis.plist

Windows

Two routes. WSL2 is the easier one if you already use it — follow the Linux instructions inside the distro — but microphone passthrough depends on your WSLg version and is the first thing to check if nothing triggers.

Running natively works too, with Python from python.org and ffmpeg on PATH. List your inputs:

ffmpeg -list_devices true -f dshow -i dummy

In %USERPROFILE%\.config\jarvis\config.env:

JARVIS_REC_CMD=ffmpeg -hide_banner -loglevel error -f dshow -i audio="Microphone (Realtek(R) Audio)" -ac 1 -ar 16000 -f s16le -
JARVIS_PLAY_CMD=ffplay -autoexit -nodisp -loglevel error {wav}

Commands run through cmd.exe rather than sh, so write JARVIS_ACTION and JARVIS_GUARD in its syntax:

JARVIS_ACTION=start "" "C:\Program Files\Some App\app.exe"
JARVIS_GUARD=tasklist /fi "imagename eq app.exe" | findstr /i app.exe

Autostart is Task Scheduler. pythonw rather than python keeps a console window from appearing:

schtasks /create /tn jarvis /sc onlogon /tr "pythonw %%USERPROFILE%%\.local\bin\jarvis"

Windows has no desktop notification command here, so notifications print to the log instead — render a greeting wav and you get audio feedback either way.

A note on what is tested

Linux with PipeWire is what this is developed and used on daily. The macOS and Windows recipes are each platform's standard tools wired into the same two config values; if one needs adjusting on your machine, open an issue and I will correct the docs.

Development

./jarvis --selftest

Twelve assertions covering the detector against synthetic frames — single claps must not fire, slow claps must not fire, triple claps must fire once, segments too short to be the phrase must be discarded — plus the fuzzy matcher against real mishearings from tiny.en, including the check that the spoken greeting cannot score high enough to retrigger the thing that just said it.

License

MIT