A loop for things you can’t measure
I built an agent to inspect a product and propose improvements. The difficult part wasn’t generating suggestions; it was deciding what evidence could show that the experience actually got better.
The basic idea behind a self-improving loop is appealing: let a model try something, check the result, feed that result back, and try again. Each pass has evidence from the previous one.
The convincing examples usually have a cheap, objective check. Code compiles or it doesn’t. A test passes. A maths answer checks out. A benchmark moves. The model can keep trying because the system can score each attempt without asking a person.
The interesting question is everything else
Much of product quality doesn’t come with that kind of check. Is the flow smooth? Is the screen confusing? Is the product pleasant to use? There’s no test that returns true when an experience feels right, even though this is exactly where I’d like continuous improvement.
I wanted to know whether a loop could improve an app’s experience as well as its correctness. After building one, I think the answer is a qualified yes. But it’s a different problem from the verifiable loops we usually point to.
The model is not the bottleneck
Models can already give useful design criticism. Show one a screen or describe a flow and it will often spot confusing hierarchy, missing feedback, or an awkward step. Producing an opinion wasn’t the main obstacle I encountered.
The harder questions were what evidence to give the model, and how the loop could tell whether its suggestion helped after it shipped.
What I built, and what it taught me
At Akarii, I built an agent that drove a real browser through the running app, walked the important flows, evaluated what it found, and filed proposals. A person reviewed the suggestions. When the agent found a clear, verifiable bug, it could go further and open the pull request. We shipped it, and it was useful.
It also made the limits of the idea much easier to see.
The inputs are harder than they look
At first I thought the browser would give the agent eyes. In practice, the model relied heavily on the page’s structure and behaviour: what loaded, what errored, how long it took, and which words appeared. The screenshot was often more useful to the human reviewer than to the model. The agent was still judging a description of the page as much as the rendered experience.
The DOM can’t capture the rhythm of a screen or the small hesitation before someone understands what to do. Browser access helps, but it doesn’t automatically give the loop a user’s experience of the product.
The signal is the harder half
Suppose the agent proposes a change and we ship it. How does the loop know whether it worked? A coding loop reruns the test. An experience loop has no equivalent assertion for “better.”
That’s why the Akarii loop stops at a proposal for experience issues. A person makes the call. For a reproducible bug, the agent can continue to a pull request because there is something concrete to verify. For a judgement about how the product feels, asking a person isn’t merely a safety fallback; it’s the only honest source of the decision.
Closing it with softer signals
The useful version behaves more like a habit than a proof. Run the agent every day and it surfaces a few small issues in the background. Over several weeks, that steady attention catches things a person is unlikely to inspect on the same cadence.
To learn whether the changes helped, you still need people: what users complain about, where they leave, and what they return to. Those signals are noisy and slower than a test suite, but they describe the experience we’re actually trying to improve.
Experience is defined by the people having it, so the signal that closes the loop has to come from them.
This might be the real shape of it
For a while, this made the loop feel unfinished to me. It couldn’t close itself like a coding loop. I’ve since come to see the human judgement as part of the system rather than an embarrassing gap in it. The people using the product define whether the experience improved.
The engineering goal becomes leverage: make observation and proposal cheap enough that a small amount of human judgement can steer a large amount of routine inspection.
The loop I want is quiet and persistent. It inspects the product every day, files useful observations, and leaves the final call to the people who use and design it. I’m still interested in how much of that judgement can be turned into a signal without flattening the experience into one convenient metric. I don’t have a complete answer yet, but building the first version convinced me that pretending “better” is a single number won’t get us there.