Your Agents Are Employees With No Org Chart

I build agents for real work. The hard part is rarely the model. It’s what happens after the first win, when the system starts rewriting its own memory.

I was talking with Lance Bryant-Grigg about LLMs and agentic coding. He had a line that stuck: you can go fast with AI, California fast, and still get what Lance affectionately called “speeding tickets”: technical debt. Weird patterns. Future maintenance you just bought with today’s velocity. That shows up hard when you’re coding with AI. The accelerator is real. So is the ticket.

The same thing happens with recursive self-improvement. The agent does useful work, then summarizes what mattered, then writes that summary back into itself. A few iterations later, sometimes just a couple of distill passes, two failure modes show up.

Drift is when the meaning moves. The memory still looks like instructions, but the job quietly changed. Constraints soften. Priorities shift. The agent starts optimizing for a slightly different task than the one you defined.

Dilution is when the meaning thins out. Summarize enough times and sharp rules become vague vibes. The agent still “remembers something.” It just doesn’t remember enough of what made it good.

Either way, you get capability degradation. The agent fails the intended task, not because the model got dumber overnight, but because the memory that steered it got damaged.

That’s not a reason to avoid agents. It’s the difference between shipping a demo and standing up a practice that lasts.

Here’s how you do it correctly.

Don’t tell an agent to “learn” and walk away. Treat it like an employee. Human in the loop is the management layer. You coach. You correct. You decide what gets promoted into the permanent playbook, and you keep working notes out of the handbook.

People get knowledge drift too. An employee who never gets coached invents their own process and forgets why the original one existed. Agents do the same thing, just faster and with more confidence.

Speed is fine. Unsupervised self-improvement is the speeding ticket. Build the agent. Then manage it.