Why the version of my agent system that looked like a step backward is the one I can actually improve.
Part 1 of a series on building agent systems with a human in the loop.
After I retired OpenClaw, I rebuilt my agent system twice.
The first rebuild was more automated. The second put me back in the middle. The second one is better—not because it does more on day one, but because it's the one I can see into, and a system you can't see into is a system you can't refine.
That's the whole argument. The rest of this is how I got there.
Where this started
OpenClaw was a multi-agent ops system I ran at Tabbris on two Mac Minis. I retired it in early August 2026. The reason was simple: maintenance outweighed the value. Keeping it running took more of my time than it gave back.
Retiring it didn't mean giving up on the idea. It meant rebuilding with a different shape.
Version 1: one agent in the middle
The first rebuild routed everything through a single Hermes agent.
The bet was that if I designed enough up front—routing, responsibilities, handoffs—the system would run with minimal input from me. One orchestrator. Everything flows through it.
In some ways, that bet paid off early. Version 1 was more functional sooner. A lot of the work happened without me touching it.
Why version 1 felt better early
Up-front design front-loads progress. You make a lot of decisions at once, encode them, and the system starts executing against all of them immediately.
That feels like momentum, and some of it is. But the decisions you encode up front are the ones you can see before anything runs. The ones that matter most usually show up later, once real work hits the system.
Version 1 had answers to the questions I knew to ask. It didn't have a good way to surface the ones I didn't.
Where it broke down: the far end of the cycle
The problem with version 1 wasn't that it failed. It was that I couldn't tell where it needed to change.
When everything routes through one agent, the judgment calls get buried inside it. Something comes out wrong at the end, and the cause sits somewhere upstream: a routing choice, a handoff, an assumption baked in at design time.
The further out in the cycle we went, the harder it got to discern where the adjustment needed to be made. Tight loops near the start were easy to reason about. Loops that ran several steps deep were not. By the time an output looked off, there were too many places it could have gone off.
So I could see that the system needed refinement. I couldn't see where. That's the worst kind of problem to have with an automated system, because the only fixes available are broad ones.
Version 2: me in the middle
The second rebuild puts me at the center. I route through ChatGPT Astra to agents on Claude, Grok Bot, and Muse.
On paper, this looks like a step backward. More of the work passes through a person. Less of it runs on its own.
It isn't a step backward. It's a different bet: that being able to see the system matters more early than how much it does without me.
Why the middle makes refinement easier
Being in the middle means every routing decision is visible to me when it happens, not reconstructed after something goes wrong.
That changes what I can learn. I see which handoffs are routine. I see which ones need a real decision. I see where the agents consistently get it right and where they consistently need correction.
With Hermes, those patterns were inside the system. With me in the middle, they're in front of me.
That's what makes the loops easier to refine. I'm not debugging a black box from the far end of the cycle. I'm watching the cycle as it runs, and adjusting the specific loop that needs it.
How judgment moves out of the middle
This is the part I didn't expect.
Being in the middle makes it easier to get out of the middle—selectively. Once a loop has shown that it doesn't need my judgment, it's easy to move it out. I've already watched it work. I know what the decision looks like and what good output looks like. Handing it off is a small, specific change, not a redesign.
The loops that do need judgment stay with me. And I know exactly which ones those are, because I've been making those calls.
Version 1 tried to decide all of that up front. Version 2 decides it one loop at a time, based on what actually happened.
I don't need to be the primary keyboard. I need to be the primary judge. In version 2, being the judge is how the system learns what can run without one.
The tradeoffs
This isn't free.
- It's slower at the start. More passes through me means less runs unattended. Version 1 was more functional early, and that was real.
- I'm a bottleneck by design. That's the point, but it's still a bottleneck. It only works if loops actually move out over time.
- It takes discipline to let go. The value of being in the middle depends on moving routine loops out. If I don't, I've just rebuilt manual work with extra steps.
The tradeoff I'd make again: slower early, in exchange for a system I can actually see into and improve.
Not "move fast and break things"
I don't buy "move fast and break things" as a way to build systems people depend on. Breaking things isn't the goal. Learning is.
Growth comes from taking action, making mistakes, learning from them, and doing that alongside the people around you. Version 1 wasn't a mistake to hide. It's how I learned where judgment actually lives in this system. That learning didn't happen alone, either. It takes a village.
What this means for client builds
This changed how I scope work.
The temptation with any agent build is to design the whole automated system up front and hand it over. That's version 1. It looks great early, and it gets harder to tune the longer it runs.
The approach I trust now:
- Start with a person in the middle of the workflow—usually someone on the client's team who already makes those calls.
- Make every decision visible while it happens.
- Move loops out of the middle once they've shown they don't need judgment.
- Keep the ones that do, and be explicit about why.
The goal isn't maximum automation on day one. It's a system that gets more automated for the right reasons, and that the people running it can still see into.
Next: the open agent system
Version 2 settled where judgment belongs. It didn't settle how the rest of the system scales around it.
Part 2 takes the human-in-the-middle loop a step further. Today I'm setting up Ando, an open agent system. The point is to build more of this ourselves and keep refining the cycle: judgment integrated where it actually belongs, inside an agent system smooth enough to help everything scale.
The experiment is testing how much of this holds up, iterating and refining along the way.
Part 2: Building the Open Agent System.
If any of this lines up with a build you're thinking about, here's how I work with teams: friendtechventures.com/services