I Gave an AI Hands on My Mac

There's a moment I keep coming back to. I asked Arthur, the AI assistant I built for my Mac, to tidy a folder. Then I watched the cursor move without me. Files renamed themselves. A directory appeared. The whole thing took maybe eight seconds, and for all eight of them I sat with my hand hovering over the trackpad like a driving instructor riding the brake.

That hover is the real story of desktop AI right now. Not the capability. The hover.

What Arthur actually is

Arthur is a personal AI chief of staff that lives on one machine. Mine. It has a voice, so I can talk to it from across the room. It has sight, so it knows what's on my screen when I ask about it. And it has hands, in the sense that it can actually operate the computer. Claude runs underneath, wrapped in a native macOS body I built around it.

It isn't a product. Nobody else can log in. That was deliberate: building for exactly one user means every design decision gets to be honest, because there's no imagined customer to flatter. If a feature turns out to be theatre, the only person I'm fooling is me.

Hands are a permissions problem, not an AI problem

The model was the easy half. Modern models handle "here's a goal, work the machine until it's done" better than most people expect. The hard half is everything wrapped around the hands. When is the system allowed to act on its own? What must it ask about first? What should it refuse even when I'm the one asking carelessly?

I landed on an approvals system. Small reversible things flow freely: reading, summarising, drafting. Anything that changes state on my machine gets surfaced before it happens, with a plain sentence about what it's about to do and why. On paper that sounds bureaucratic. In practice it costs half a second, and it's the only reason I can walk out of the room while it works.

An agent you have to supervise isn't an assistant yet. Autonomy gets earned in increments.

The pattern I've settled into is that trust gets granted in tiers, and tiers get earned by track record. Arthur holds permissions today it didn't hold at the start, because the approval log kept being right, over and over, across a long stretch of ordinary use. That's not affection for my own project. That's just what the log says.

The vault, and keeping secrets from a housemate

Arthur has a vault for things I genuinely don't want sitting loose on the machine. The key is spoken out loud, in my voice, and it's never stored anywhere. Not in a config file, not in an environment variable, not "encrypted somewhere". If I don't say it, the vault stays shut, including to Arthur.

Designing secrets management for an AI that shares your computer turns out to be a strange new domestic problem. It's less like configuring software and more like working out what you'd tell a housemate. A very capable housemate who never sleeps. The answer, in my case, was less than you'd think, behind a door that only opens to a voice.

I built security theatre on purpose, and it turned into policy

Early on I gave Arthur dramatic modes. Lockdown, overwatch, go dark, all of it in a commanding red interface that was frankly over the top. I knew some of it was theatre while I was building it. But theatre makes a decent prototype for policy. A staged "go dark" switch forces genuinely useful questions. What should that actually cut off? Network access? The watchers? Voice? Answer those a few times and the staged switch quietly becomes a real one.

If you're building agents, I'd recommend the exercise. Building the dramatic version first makes the boundaries visible enough to argue with.

What it feels like once the novelty wears off

Day to day, Arthur is boring in the best possible way. It's ambient. I talk to it mid-task the way you'd talk to someone at the next desk. The interactions that stuck around aren't the impressive ones. They're the two-second ones. What's this error. Rename these to match. Where did I put that spec.

It still gets things wrong. It misreads a screen now and then, and occasionally it solves the wrong problem with total confidence. The approval layer catches anything that matters. The rest is the same friction you'd get from any capable new colleague finding their feet, except this colleague improves every time the underlying model does.

The hover over the trackpad is mostly gone now. Not because I sat down one day and decided to trust it, but because it kept being right in small ways, thousands of times, until the ledger tipped. That's how trust works with machines. Roughly how it works with people too, only with better record keeping.