Stream Deck Integration for a Command Center

[egns-ai]

Hi everyone!

First, I want to say thanks to @chadbyte for making and maintaining this repo. I bought a mini PC solely to run Clay and connect from anywhere using Tailscale with MagicDNS. It's very convenient. I can work and debug from wherever I am, even when I'm away from my main computer. I do miss the Claude CLI, but I don't miss the overhead of initializing everything every morning and restarting my laptop just to clear old sessions before gaming.
Since everything is managed by WebSockets, I had an idea to use external buttons to manage sessions and surface notifications. I was originally going to connect an ESP32 with an external LCD display, but then I remembered I had an old Stream Deck lying around.
I went down the rabbit hole and built this out, though it could easily be adapted to macro keys if you prefer.
This works with a phone, iPad, or even a virtual device on your screen through the Stream Deck app. The free trial will eventually run out and cost $25/year, but for now I'm using it on both my phone and iPad simultaneously.

If there's interest, I'm happy to share the code, though right now it's tailored to my own workflow. In a perfect world, you just select a project/session and choose to voice type. Getting pretty close to Jarvis

---

What it does:

The Stream Deck plugin connects to Clay's WebSocket API and turns physical buttons into a live dashboard for managing all your projects, worktrees, and sessions at a glance. It supports multiple Stream Deck devices simultaneously — a small one as a "hub" showing an overview, and a larger one for detailed session management.

Hub device (small Stream Deck):

• Shows all your worktrees sorted by recent activity, each with a unique accent color per project
• Buttons pulse orange when a session is processing, flash red when permissions are pending
• A dedicated usage widget always visible in the corner showing your 5-hour cost, weekly cost, and total spend — tap it to cycle through cost, rate limit gauge, and token breakdown views (still needs more work)
• When a session finishes, a green "Done" notification button appears — press it to jump straight to that session in the browser
• Approve/deny permission requests right from the buttons without touching the browser

Detail device (large Stream Deck):

• Row-based layout — each row is a worktree with its sessions laid out horizontally
• Press a session button to switch to it, long-press a worktree header to focus its browser window
• Row order stays stable while you're working (no jumping around on activity updates), but pressing a worktree on the hub promotes it to the top

How it works:

It's a standard Elgato Stream Deck plugin (SDK v3) with a single action UUID placed on every key. The plugin renders dynamic SVG images (144x144px) for each button and handles all layout logic internally — no static icons.

The architecture is layered:

• Connection layer — Authenticates with Clay via PIN, discovers projects from /info, then opens a WebSocket per project slug for real-time events (projects_updated, session_list, permission_request_pending, result for usage tracking, rate_limit)
• DataStore — Pure shared state that accumulates data from all WebSocket connections and emits change events. Tracks projects, sessions, permissions, rolling usage costs/tokens, and detects session completions
• Layout engines — Each device gets a layout based on its role. The hub gets WorktreeOverviewLayout (flat grid with widget slots), the detail device gets SessionGridLayout (row-per-worktree). Layouts compute a ButtonDef for each grid position
• Render pipeline — DeviceRenderer per device with independent caching and pulse animation timers. Only re-renders buttons whose ButtonDef actually changed. Pulsing buttons (processing, permissions) animate on a 500ms interval

Cross-device communication is handled in the action's button press handler — pressing a project on the hub navigates the detail device to that worktree's row and focuses the browser window.

The whole thing bundles to a single ESM file via Rollup, lives inside the com.clay.streamdeck.sdPlugin/ package, and deploys by tarring it up on Linux (on my clay server) and pulling it down on the Windows machine (my main machine) running Stream Deck.

[chadbyte]

This is impressive, especially since Clay doesn't actually expose a public WebSocket API yet. The fact that you built this on top of the internal communication layer shows real dedication.

If you're open to sharing the code, I'd love to take a look. Seeing how you approached this could help us think about what a platform API could look like.

I'm also planning to add a community projects section to the README. Would love to feature this there.

Also, buying a mini PC just to run Clay made my day.

[chadbyte]

@egns-ai By the way, I'm curious, how are you using Clay day to day? What's your typical workflow like, and what was the moment that made you think "I need a Stream Deck for this"?

[egns-ai]

Let me start with Claude Code. I use it pretty much vanilla other than superpowers, specifically for brainstorming. I only brainstorm when I am making a new feature, and I do it in an isolated git worktree (other than some uses cases I mention below) so I can easily test but also dismantle.
Dismantling and starting over is very important because Claude Code will assume lower level logic and never look at it again unless you tell it to, and even then it will sometimes skim over it. I would rather go for a better initial one shot than spend time debugging and nitpicking after the fact. I nitpick at the plan, not the code.
For almost everything else I have a conversation with the agent and go through the details about a bug or a fix. I also really like to feed plans back into claude.ai to get a third perspective on what I am making, without code or any other assumptions attached. Claude really likes to assume what you are making and jumps the gun on a lot of things. Removing and finding assumptions is very important in what I do because it will assume one way to calculate or build a metric when there is a proper way that is not present in training data and cannot just be looked up. I build dashboards for people with big data systems who do not really know how to exploit their data, and I build agentic underwriting for non-QM lending. There is not much real data or real use cases available publicly. You cannot just assume how to calculate something based on what someone else did, you need to talk to the people, ask how they do it, and look at the manual reports they make by hand. Those reports are the ground truth.

Workflow:
Before Clay, I used Zellij, a tmux-adjacent CLI that managed windows and moved them around. It was okay, but it lacked file browsing and git worktree management at a glance. Chats would not persist, so if it crashed I would have to initialize 5 different CLIs and hunt for my old session to pick back up.
Depending on what I am doing, I could have 1–5 agents running at a given moment across multiple projects. Clay was a step up because I can see my worktrees and previous agent sessions at a glance, and I can easily pull in old context from a previous chat into a new one. Sometimes I also run multiple sessions inside a single worktree. They are totally independent threads. I do not want to manage a git merge or push everything to yet another branch just to test it. A lot of what I do is sequential: test fixes, run a dashboard for 10 minutes, check again. It is easier to do 10 batch fixes with individual context and generate once than to spin up 10 worktrees and reconcile them later.

Why the Stream Deck?
When you have 4 sessions open you get lost, either in one chat or on your phone, so having an overview of what is going on and what is done becomes important. I also wanted a physical way of seeing when a chat is done. I had an old Stream Deck lying around, and one Google search showed there is a whole marketplace of apps with their own CLI.
I mainly wanted it as a focus window: an agent is done, click the button, see the results, respond, keep going. Doing this across 4+ sessions spread left to right over 3 monitors got confusing. I did not want to memorize macros either — I don't want to remember that Ctrl+J goes to my dashboard agent, only to have it spawn a fresh instance of the project instead. Too many scenarios. I just want to click a done agent.

I do not look at notifications. I have them turned off on almost everything, and one more would just annoy me. A status bar is not enough for what I wanted either. Physical buttons make it real. You do not scroll Instagram or watch reels on a Stream Deck, you just manage your agents, with custom buttons for each.

The Stream Deck solved it. Each button is a session, not a shortcut to memorize. When an agent is done, the button tells me. I press it, I am there, I read the result, I respond, I move on. No hunting across monitors, no mental overhead of keyboard macros, no accidentally spawning a fresh instance when I just wanted to check in.
Right now it works as a control center — quick project switching, quick worktree focus, a high level view of what is running and what needs my attention. It makes context management easier and keeps me at the steering wheel instead of getting pulled into the weeds of managing windows.

It started as a proof of concept just to see if it would even work. I will evolve it as I need to. One thing I want to add is a voice input button , capture a quick thought and send it to a focused window without breaking what I am doing. When you are managing multiple agents, the friction between having an idea and getting it to the right place adds up. A physical button that just listens and drops the thought where it needs to go fits the whole workflow.

The through line in all of this is staying high level. The more I can review instead of manage, the better the output. I have gotten closer to just explaining my idea and letting the agent roam, but I still watch most steps: what commands it calls, why it does something a specific way. Your CLAUDE.md can only get you so far. You can bake preferences in, and I do, but as you build out there is always more. Planning is important because Claude will assume you want something a certain way and just implement it past you. Once it is implemented it is harder to question. Catch it early. Code has gravity, a plan is just words, and you can push back freely on words.
Claude can do a lot, but it still needs someone watching. Not out of mistrust, but because judgment is the one thing you cannot put in a CLAUDE.md file.

Anyways, here is the repo for the plugin: https://github.com/egns-ai/clay-streamdeck-plugin

Another thing is that I was debating on getting a mac mini just for work, isolate everything on one machine to not cross contaminate. It was also hurting my fps in CS2 (before this, I would develop on a encrypted SSD through WSL on windows). Once I got the machine, I was free to wipe my laptop and spec it to gaming. The day after I got Clay up and could connect through into my phone, I went all in and got the mini pc (and the fact the mac mini was back in stock at microcenter, but when I got there, I found better options with better specs, so I should never need a new development machine). Also I needed it to be quiet and efficient because I planned to have (ideally) agents running over night without me interrupting to play games, school work, or anything else I wanted on my computer. I did look into remote tunnels (teleport), but I did not like the fact Anthropic help manage connections, and terminus was still just a remote ssh window. Not the best for developing on phone.