Claude Computer Use

Letting an agent drive a real desktop is useful and uncomfortable at the same time. It needs a mouse, a keyboard and a browser to be of any use, and the moment it has those on my machine, a wrong move is my problem, not a sandbox’s.

The second problem is throughput. One agent on one desktop is a queue. Anything worth automating needs several running at once, and they cannot share a screen.

And an agent working invisibly is not something I will trust. I need to see what it is doing while it does it, and be able to take the mouse back.

Every session is its own Docker container running a full XFCE desktop, with an action server inside it that moves the real mouse and keyboard. The host machine is never in the loop.

A daemon pools those containers and hands them out, so several agents run in parallel without knowing about each other.

The pool is exposed three ways, because the same infrastructure gets used differently depending on the job: an MCP server for an agent in a conversation, a FastAPI dashboard with live noVNC views and take-over for watching, and a CLI for launching headless runs.

Several agents working at once, each isolated, each watchable, each interruptible from one page.

No agent has ever had access to the host machine, which is the only property in this project I actually care about.