Independent · LLM × robotics · 2026
Moose.
The agents I design at work live behind glass. Moose is the same problem with legs: a small robot that knows who it's looking at, does what it's told, works out when to keep quiet, and gets itself back on the charger. It walks and it sees. The rest is still ahead of it. This is the build log, written as it goes.
The short version
- Everything I know about agent autonomy came from arguing about dialog boxes. I wanted the version where being wrong makes a noise.
- Four features. Know who you are. Take orders. Speak up only when it's welcome. Charge itself.
- Two phases in: Linux drives the servo bus, the dog carries its own compute, and it recognizes my face on-device.
- The model is Llama, running on the robot. Nothing leaves the house.
- Posted while it's half-built, because the half-built part is where the decisions are.
01 · Why
The same argument, with legs
My job is deciding what an agent gets to do without asking first. In enterprise software that debate ends in a modal dialog. When it goes wrong someone clicks a button they didn't mean to click, and we fix it next sprint.
Put the thing on four legs and the stakes change. Interrupting becomes a noise in a room. Getting my name wrong becomes a small insult. Running the battery flat becomes a robot asleep on the floor at 4pm. The same design question, with louder answers.
So: four features, cheap parts, no pretense that it's a product. I've wanted to build this for about two years.
02 · The parts
What is on the bench
Two components so far. One of them decides most of what follows.
Arduino UNO Q, 4GB. A Qualcomm QRB2210 running Debian sits next to an STM32 microcontroller on the same board, so the thinking and the servo timing don't have to talk over a cable. Beside it the XGO quadruped: fifteen servos, a 2500mAh pack good for about two hours, 500g before I add anything to it.
The board was picked for one reason: Linux and a real-time chip, no wire between them. And 500g with small servos means every gram of camera, speaker and battery is an argument.
03 · The brief
Four features, and what they cost
My own difficulty rating, written before I started so I can't fudge it later. Two of these are much harder than they sound, for completely different reasons.
| Feature | What it takes | Difficulty |
|---|---|---|
| Know who it's talking to | Camera, face detection, a handful of enrolled faces matched on the robot | B |
| Take voice commands | Wake word, speech to text, Llama with a fixed set of tools, speech back out | B |
| Speak up at the right time | Calendar, seated-presence detection, and knowing whether I'm already talking to someone | D |
| Charge itself | Battery telemetry, find the dock, walk to it, and line up well enough to couple | D |
04 · The model
Llama, running in the room
Moose runs Llama rather than calling out to something hosted. Partly because a robot that stops working when the internet does is a bad robot. Mostly because the whole point of this is watching what an agent does with a body, and I didn't want a monthly bill sitting between me and that.
The open question is where it runs. A small quantised Llama on the QRB2210 is plausible and will be slow, and slow matters when someone has just asked you a question out loud. The alternative is a box in the closet doing inference over Wi-Fi, which is faster and less interesting. I'll measure the on-board version first and see how bad a two-second pause feels.
05 · The plan
What has to happen, in order
Sequenced so each phase leaves something that works on its own. If Moose stalls after phase two it's still a robot that says my name when I sit down.
Phase 0 · BenchComplete · the two halves talk
- Acquire the UNO Q 4GB and the XGO quadruped done
- Boot Debian on the UNO Q, get it on Wi-Fi, set up remote access done
- Confirm the MPU–MCU bridge with a trivial round tripLinux sends a command, the STM32 acts, state comes back done
- Find the XGO's serial protocol and drive one servo from LinuxThe kit ships as a micro:bit robot, so the question was whether the UNO Q could talk to the servo bus at all done
- Read battery voltage over the same linkPhase four is worthless if this number lies done
- Power budget: run the UNO Q off the dog's 8.4V pack and measure the cost in runtime done
- Weigh the whole stack against the servo torqueThe one that could have killed it done
Phase 1 · EyesFeature 1 · working, minus one decision
- Mount a camera and hold a steady frame rate on the QRB221010fps is plenty for faces done
- Run face detection on the robot, quantised, and measure real latency done
- Build enrolment as an explicit, consenting moment done
- Store the embeddings locally and match against themNo face data leaves the robot done
- Design what it does when it isn't sureA robot that gets your name wrong with confidence is worse than one that asks in progress
Phase 2 · Ears and voiceFeature 2 · spoken commands through Llama
- USB microphone and speaker working on the UNO Q queued
- Local wake word, so nothing is recorded until it's spoken to queued
- Speech to text, benchmarked on-device queued
- Benchmark Llama on the QRB2210 and decide: on the robot, or a box in the closetLatency decides this, not principle queued
- Write the tool schemaThe list of things Moose may do is the autonomy policy. This is the actual design work queued
- Pass the recognized person in as contextWho it's looking at should change what it says and what it refuses queued
- Speech back out, with a hard budget from wake word to first syllable queued
- Decide what happens when it mishears queued
Phase 3 · JudgementFeature 3 · interrupt only when welcome
- Read-only calendar access queued
- Detect that someone is present, seated, and has been for a while queued
- Work out whether I'm already in a conversationThe hard one. Probably a small agent on my laptop reporting mic and camera state, rather than the robot guessing from room audio queued
- Write the interruption rules down: quiet hours, one reminder per event, never twice queued
- Give it a way to look like it has something to say, so it can wait to be asked queued
Phase 4 · Self-preservationFeature 4 · get back on the charger
- Pick the coupling: inductive is tidy and fussy, contacts are ugly and forgivingLeaning toward a funnel that fixes a sloppy approach mechanically queued
- Build the dock and put a marker on it the camera can find queued
- See the dock, walk to it, square up queued
- Line up, then check current is flowing before calling it doneA robot that thinks it's charging and isn't will be dead by morning queued
- Battery state machine: threshold, stop what it's doing, say so, dock, sleep queued
- Decide what it does when it can't find the dock queued
The rest of the list (following me room to room, picking things up with the arm, learning where things live, telling the dog apart from the cat) is real and isn't on this plan. These four make it worth keeping on the desk.
06 · Design position
Three rules I'm building against
-
01
It asks before it does anything physical
Same line I hold at work, easier to defend here. Looking, listening and talking are fine on its own. Moving with intent gets confirmed, and so does anything that touches my calendar. A wrong answer on a screen is a wrong answer. A wrong movement is a knocked-over coffee.
-
02
Faces stay on the robot
Recognition runs locally, against people who agreed to be in it. No images to a server, no cloud index, and a real answer for anyone who wants out. It makes the vision work harder. I'd rather design inside that than write an apology later.
-
03
Quiet unless there's a reason
Every ambient gadget I've owned lost me the first time it talked when I didn't want it to. Moose starts silent. Each interruption has to get past a rule I wrote down before it happened.
07 · Risks
What could still sink it
Payload, which is now settled. A 500g dog with 2.3kg servos wasn't obviously able to carry a compute board, a camera, a microphone and a speaker. The backup plan was off-board compute over Wi-Fi, which is a duller project. It carries its own head. That was the go/no-go and it passed, which is why the rest of this is worth writing down.
Knowing whether I'm on a call. Reading that off room audio is a research problem, not a Saturday. I'm going to cheat and have my laptop tell the robot.
The dock. Inductive charging wants millimeter alignment. A walking quadruped gives you centimeters. Either the dock corrects for that physically or the coupling changes, and either way it's a hardware problem wearing a software problem's clothes. It's the part I'm worst equipped for.
08 · Where it stands
Where it stands
- Phases zero and one done. The UNO Q drives the servo bus, the payload maths worked out, and it recognizes enrolled faces on the robot.
- Next up: what it says when it half-recognizes you. Then wake word, speech, and the Llama tool schema.
- Published half-finished on purpose. Portfolios usually delete this part.