The model gets the headline. The harness sets the bill.
That line kept showing up in my client work this spring, and a mid-June digest pointed at the same tension: a developer ran Claude Code head-to-head against two open-source coding tools and published numbers. The write-up that stuck with me was Tensorlake's 14-day harness comparison. Same repos, same hosted tool router, same model on every run. Only the wrapper changed.
If you are routing agent spend in 2026, that is the question worth answering before you buy another frontier subscription.
What a harness actually controls
Strip the branding and every terminal agent is the same loop: read task, call tool, observe result, repeat. The differences sit outside the model weights:
| Layer | What varies between harnesses |
|---|---|
| Tool surface | Pi ships 4 tools. Claude Code ships 10+. |
| System prompt | Community tests put Claude Code around 33k tokens of prompt and schemas before your first message. OpenCode around 6.9k. |
| Context policy | Silent compaction versus manual pruning. |
| Permissions | OS kernel sandbox (Codex) versus approval prompts (Claude Code) versus config rules (OpenCode) versus none (Pi). |
| Model routing | Claude-only versus 75+ providers versus BYOK everywhere. |
Cursor and Claude Code trained model and harness together for years. OpenCode and Pi treat the model as a swappable dependency. That split is why benchmark posts that only compare GPT versus Claude miss half the cost story.
The controlled benchmark setup
Tensorlake ran two tests:
- Daily driving for 14 days across a messy Next.js app, a Go API, and smaller TypeScript repos. All harnesses pointed at OpenRouter for fairness.
- Controlled eval: 30 hard agentic tool-use tasks on real apps. Every harness used DeepSeek V4 Flash through the same hosted MCP tool router. 900 second cap per task.
Same model. Same tools. Same tasks. Any gap is harness architecture, not weights.
| Harness | Passed | Median time | Avg tokens / task | Total cost | Cost / success |
|---|---|---|---|---|---|
| Pi | 20/30 (66.7%) | 132.2s | 558,885 | $0.56 | $0.028 |
| Claude Code | 16/30 (53.3%) | 122.7s | 741,659 | $3.12 | $0.195 |
| Codex | 16/30 (53.3%) | 245.0s | 664,772 | $1.29 | $0.081 |
| OpenCode | 14/30 (46.7%) | 129.7s | 692,195 | $1.03 | $0.073 |
Pi winning on pass rate with four tools is the headline. Claude Code winning on median time while burning the most tokens is the finance headline.
How I read the results
Pi (minimal harness). Four tools, tiny default prompt, extensions instead of MCP schema dumps. Tensorlake's cost-per-success gap ($0.028 versus $0.195 for Claude Code) is what I would show a CFO before arguing for another Max seat. Tradeoff: almost no guardrails out of the box. You are expected to run inside Docker or a VM if the repo is untrusted.
Claude Code (batteries included). Fastest median completion, richest ecosystem (skills, sub-agents, Agent Teams, hooks, MCP client and server). Best fit when the team already lives in Anthropic's stack and subscription economics hide token volume. On API keys, 741k tokens per task adds up fast.
Codex (safety first). Tied Claude Code on pass rate at under half the dollar cost, but 245s median (roughly 2x the others). macOS Seatbelt and Linux bubblewrap plus seccomp enforce sandboxing at the kernel, not through dialogs. Worth it for unfamiliar repos even when it feels slow.
OpenCode (platform play). Weakest pass rate in this eval, strongest provider flexibility (75+ routes, local Ollama, MIT license). Community framing matches OpenCode versus Claude Code: you are choosing whether to own the wrapper. Terminal-Bench scores on OpenCode are mostly the model you plug in, not a fixed harness score.
The Databricks datapoint that should worry you
Tensorlake cites a Databricks internal run with the same Opus 4.8 model across harnesses on their codebase:
| Harness | Tokens per task | Success rate |
|---|---|---|
| Claude Code | 742,000 | Same |
| Pi | 236,999 | Same |
Same model. Same outcome. Three times the tokens in one wrapper. That is not a rounding error. It is context policy, tool verbosity, and what stays in the rolling window after each grep.
I have seen similar swings shipping agent search for clients. Teams buy bigger context windows while the harness keeps re-injecting entire file reads. Fix the librarian before you rent a bigger desk.
Practical routing for teams I advise
| If your priority is... | Start here |
|---|---|
| Daily velocity on trusted repos | Claude Code or Cursor with your best frontier model |
| Untrusted or forked open source | Codex for kernel sandboxing |
| Cost per successful task on API keys | Pi with a strong but cheap model |
| Provider freedom and self-hosting | OpenCode plus local GLM or Qwen routes |
| Compliance and auditability | Open harness you can fork (OpenCode, Pi) plus your VPC |
None of these is a purity test. Tensorlake's author still reaches for Claude Code for day-to-day work while using Pi when token economics dominate. I do the same with clients: harness per workload, not one religion.
What this does not prove
- One benchmark on DeepSeek V4 Flash is not your monorepo. Re-run on your stack before switching contracts.
- Pass rate ignores code review quality. A fast diff that fails lint is still a fail in production.
- Subscription plans hide token math. Claude Code "feels cheap" on Max until you compare against BYOK Pi on the same task list.
- OpenCode's GitHub star count (~170k to 180k in mid-2026) measures interest in open wrappers, not daily active quality.
Bottom line
Pick the harness before you pick the model. The June digest story was not "Claude Code is bad." It was "Claude Code is optimized for a product experience that trades tokens for polish." Open-source harnesses can win on cost and pass rate when you control the sandbox and the model route.
Before your next agent rollout, run 20 real tickets through two wrappers on the same model. Log tokens, wall time, and human fix-up hours. That table beats any vendor benchmark slide.
Shipping agent workflows with measurable token and success metrics? Book a free discovery call.