A desk robot that keeps me company while I work. It looks at me every few minutes, remembers the day, and occasionally says something. Usually dry. Sometimes kind. Mostly silent.
"It's 11. Breakfast is a morning thing. Just saying."
Tap its head and it becomes a voice interface to my Claude Code sessions and calendar.
Built on the M5Stack Stackchan (K151 kit, ESP32-S3) and Claude.
- Notices things. Every few minutes it grabs a low-res camera frame. Claude Haiku decides, with the day's log, whether anything is worth saying. Most looks end in silence on purpose.
- Remembers the day. A small local log lets it say "you haven't eaten yet" instead of judging one frame.
- Talks back. Tap the head, or answer within a few seconds of a remark. Speech is transcribed locally, Claude Sonnet answers, and the robot speaks with a local neural voice. The face and head move while it talks.
- Runs my sessions by voice. "What's running?", "does anything need me?", "start a session on Acme to check the deal ranking." Anything that acts is read back first and only runs on a clear "yes".
- Knows when to shut up. Silent during calendar meetings, during calls (any app using the mic), outside working hours, and when I'm away from the desk.
- The only cloud is the Anthropic API on my own key. No M5Stack or third-party robot cloud.
- Speech-to-text (whisper.cpp), voice activity detection (Silero) and text-to-speech (Kokoro) all run on the Mac.
- Camera frames and audio are never written to disk.
- It never comments on anyone else in frame, and never sends messages to other people.
Stackchan (ESP32-S3) Mac
┌─────────────────────┐ WebSocket ┌──────────────────────────────────────┐
│ camera, mic, touch │──────────────▶│ companion (Python) │
│ face, servos, speaker│◀──────────────│ observe → Claude Haiku → remark │
│ Moddable JS mod │ speech WAV │ listen → Silero VAD → whisper.cpp │
└─────────────────────┘ │ answer → Claude Sonnet + tools │
│ speak → Kokoro TTS │
│ gates → calendar (EventKit), mic, │
│ idle time, work hours │
└──────────────────────────────────────┘
| Path | What lives there |
|---|---|
companion/ |
Mac-side service: observation loop, voice turns, gates, tools, socket server |
firmware/mods/companion/ |
Moddable mod that runs on the robot: camera, mic streaming, face, head moves, offline screen |
tools/calnow/ |
Tiny Swift CLI that reads calendar events through EventKit (recurring events included) |
launchd/, scripts/ |
Run as a LaunchAgent; build and flash helpers |
vendor/stack-chan |
Upstream stack-chan firmware, pinned to v1.1.0 |
You need the K151 kit, a Mac, an Anthropic API key, and the local models (whisper.cpp small.en, Silero VAD v5, Kokoro).
git clone --recurse-submodules https://github.com/jiggsfoo/stackchan.git
cd stackchan
python3 -m venv .venv && .venv/bin/pip install -r requirements.txt
cp config.example.toml config.toml # calendars, hours, voice, model paths
cp .env.example .env # ANTHROPIC_API_KEY, COMPANION_TOKEN
cp firmware/mods/companion/secrets.example.js firmware/mods/companion/secrets.js
scripts/fw.sh deploy # host firmware (Moddable SDK 9.5 via xs-dev)
scripts/fw.sh mod -- $PWD/firmware/mods/companion/manifest.json # the companion mod
scripts/install-launchd.sh # start the Mac serviceSet the Mac's mDNS name in firmware/mods/companion/manifest.json before flashing. docs/spike-notes.md has the hardware gotchas.
The session tools expect my personal Claude Code session manager. Without it, voice still works for chat and calendar questions.
.venv/bin/python -m pytest # Mac side
node firmware/mods/companion/commands.test.mjs # firmware command logicA personal project that runs on my desk every day. Not a product. Issues and ideas are welcome, but I'm not promising support.