What I mean by harness engineering
A language model is one API call. Everything that makes it useful is the harness: the loop around it, the tools it can reach, the gates it has to pass, and the record of what it did. That is the part I write myself.
The model is the easy part
The model call is a few lines of code. You send some text, you get some text back. The hard part is everything around it: deciding when to call, with what context, what to do with the answer, when to stop, and how to prove afterwards what happened. That surrounding code is the harness, and it is where the engineering actually is.
Why I write my own, with no dependencies
War Room and JujHub, 2 systems I build on my own, run on a harness with no third-party packages. No NuGet in the C#, nothing pulled in for the agent loop. That is a deliberate choice, and it costs me time. I write plumbing that a framework would otherwise hand me.
I take that cost for 3 reasons. I can read every line, so there is no behaviour I cannot account for. Nothing changes under me on someone else's release schedule. And when something breaks, I am debugging my own code, not guessing at a library's internals. For work I want to trust, that trade is worth it.
The gates matter more than the loop
JujHub is an autonomous studio: agents take a goal and produce a change, and they can edit their own code. But nothing they write reaches the main branch on its own. Promotion is always the operator's call, and the self-editing part runs behind gates it cannot route around. One gate keys off a test run's exit code. If the tests fail, the change is rejected.
In one recorded run, the self-editing loop fired several times. Every attempt was logged as either no change or tests failed. Zero landed. That is not the harness failing. That is the harness working: the gate held, and nothing unproven got in.
If it is not logged, you cannot trust it
Both systems are event-sourced. Every session writes an append-only log, and War Room keeps an append-only record of every model call and what it cost. I do not keep those logs to be tidy. I keep them because a model's output is a claim, and a claim you cannot inspect is a guess. The log is the evidence.
A harness you can read is a harness you can fix.
It is the same job as the banking work
This is the habit I brought from 8 years on banking systems: make the machine's behaviour observable, and make every failure recoverable. The model gets the attention. The harness is the part that makes it safe to run.
Both harnesses are personal projects, built dependency-free in .NET. If you work on agent systems, evaluation, or the reliability side of running models in production, I would enjoy the conversation.