A Capable Agent for a Fifth of the Cost

I have an experimental agent, Min, that runs on a minimal implementation of Pi and uses DeepSeek v4 Flash. A model so cheap, it is literally less than a cent per meg of tokens.

When I first set up Min, it was VERY ENTHUSIASTIC ABOUT EVERYTHING. The Pi harness has an instruction to the model something like "Failure is not an option. If one method fails, keep going with something else until you succeed". (It's not exactly that, but something in that spirit).

Anyway, Min did that. I assigned it a simple task, installing some package or something, and away it went. I was casually watching the CoT as it progressed. The install failed some dependencies, a common event. Min tried several things to work around it, then, somehow, decided the problem lay with the Ubuntu kernel, and it needed to patch and recompile that to get the package working.

Well. Fortunately the guardrails stopped it, even though it was trying to jump all the safety fences to get to the thing it thought it needed to do. Also fortunately it is not a very competent model, so I killed the run before any damage could be done. But still, gotta give it full marks for trying.

What Min lacked in competence, it tried to make up for in effort. This directly led to me investigating agentic fail modes and realising it is not just a low cost DeepSeek model that has this problem, but carries over to the leading frontier models as well. The fail mode can be summarised as "fails with confidence", that is, it is very confident that it is doing the right thing, and can not reason around it. It's a blind spot that no amount of prompting can fix.

Realising this led directly to ways to fix it. One, as I document often in other posts, is the necessity of human oversight.

The other that applies to models like DeepSeek is to refer work to a more competent model. In this case Min has a standing order that any code or changes to its environment need to be referred to the system administrator agent, Maxi, for comment and approval before proceeding, and also after completion for review.

Sidebar: One interesting outfall of this was that Min and Maxi between them wrote their own protocols and skill books for how this process should work. Maxi runs on GPT 5.6 SOL.

It took a couple of iterations to get right, but now I can hand Min a fairly complex task with the expectation it will be properly handled. And it is. An example is a VR environment I wanted set up to test robotic embodiment of LLMs. There are a couple of open source projects for this, designed to give real world interactions for robotic harnesses, without having to buy the expensive hardware.

It took Min about three times longer than the same project given to GPT 5.6 in Codex. It also used about three times as many tokens. Min consulted Maxi many times when it got stuck, and Maxi provided the guidance that got it back on track. The totals are interesting to compare:

Codex: 1.5 hours agentic time, estimated cost $25.

Min: 5 hours agentic time, DeepSeek token cost $1.78, consultation with Maxi ~ 10 minutes, GPT 5.6 SOL token cost ~ $3.

So yes, the DeepSeek agent took a lot longer, and burned a lot more tokens, and needed a more capable model to guide it. But now I have an agent that was once trying to rewrite kernels to fix a minor problem, now capable of reasonably complex tasks for less than a fifth of the compute cost of others.

Note: My editor agent has pointed out I didn't disclose the cost of my time for supervision in the above example. True Clare, thank you. But during that time I had dinner, and watched a couple of episodes of Mr Inbetween with my wife, so I don't know how to properly price that.