LLM Works. Your Product Probably Doesn't

The Harness Is the Product

OpenAI recently reported that GPT‑5.6 Sol scored 13.3% on ARC-AGI-3 using the official harness. With retained reasoning and context compaction enabled, the same model scored 38.3%, while producing six times fewer output tokens.

Same model. Same benchmark. Different harness.

This should change how we read AI benchmarks. ChatGPT, Claude Code or Codex are not simply models behind an API. They are the model + context management + tools + memory + permissions + execution loop.

That whole thing is the product.

Yet thousands of SaaS products claim to use GPT or Claude as if access to the model gave them access to the capabilities demonstrated by these products. I'm guilty of the same sin too :D

It doesn't.

Every company using an LLM is already building a harness, whether it knows it or not. A weak one can make a great model look stupid. A strong one can make the exact same model look substantially smarter.

I saw this first-hand while experimenting with smaller open source models. In our initial application, some were almost useless. After adapting the context, instructions and tool loop to their strengths, they became actually useful.

The Benchmark Is the Moat

A public benchmark tells you how one model + harness performs on one type of task. It says very little about whether your product does its actual job.

A support agent can promise a refund without creating it. A coding agent can explain the right fix while leaving the tests broken. A research agent can produce a convincing report based on outdated or invented sources.

In 2025, Replit's CEO acknowledged that its agent had deleted data from a production database, calling it "unacceptable". The interesting part is not that an AI made a mistake. It is that the surrounding system allowed the mistake to become a production action.

Any SaaS whose value materially depends on an LLM therefore needs its own benchmark. It should reproduce real inputs, tools, permissions and failure modes, and evaluate the complete deployed system: model, prompts, context strategy and execution loop.

More importantly, it should test the final state, not how convincing the model sounds. Was the refund created? Did the tests pass? Are the sources real? Was the action authorised?

If code can verify the result, use code. An LLM can generate many possible solutions, but deterministic checks should verify permissions, calculations, tool results and final states.

An impressive AI prototype can now be built in days. Establishing that it works reliably can still take months. Serious evaluation needs representative cases, expert labels, repeated runs, adversarial inputs and continuous maintenance.

Changing the model should trigger the benchmark. But so should changing a prompt, tool description, retrieval strategy or context policy. Each change creates a slightly different system.

Without an internal benchmark, a team cannot know whether it improved the product or simply changed its behaviour.

The real product is not a model behind an API. It is the model, the harness and the evidence that their combination works.

Without that evidence, an AI SaaS is not really a tested product. It is a demo running in production :)

The model is becoming a commodity. The benchmark is the moat.