Gemini 3 Pro passed all 89 tasks of Terminal-Bench 2.1, a command-line test of AI agents, in two agents. Driven by Gemini CLI, the agent that Google publishes for its own models, it achieved 65.8%. Driven by Terminus 2, a minimal program written by the authors of the test to compare models and equipped with a single tool, a terminal, it rose to 73.9%. The Terminal-Bench 2.1 ranking, since archived by Snorkel AI, which hosts it, separates these two lines by 8.1 points, well beyond the variation from one execution to another that it displays, of 1.3 and 1.4 points. The model is the same. What has changed is called the harness.
The word designates everything that surrounds the model to make it a working agent, “the loop that calls Claude and routes its tool calls to the relevant infrastructure”, writes Anthropic in an engineering post from April 8, 2026. This component is now sold alone: the Claude Agent SDK delivers “the same tools, the same agent loop and the same context management as Claude Code”, and Anthropic rents a hosted harness, Managed Agents. For the same model, this product changes the score, the bill and the risk.
Seven components between model and machine
A language model alone produces text. The harness detects tool calls, executes them, returns the results and starts again until the task is finished or abandoned.
| Component | What is it for? |
|---|---|
| Execution loop | Call the model, execute its tool calls, decide when to stop |
| System instructions | Set the role, rules and method |
| Tools and permissions | Define what the agent can do, and what requires human agreement |
| Context management | Choose what remains in the model window: compaction, memory, progress files |
| Sous-agents | Process a subtask in a blank context and report only a summary |
| Vérification | Test the work before handing over |
| Sandbox | Limit accessible files and reachable servers |
Anthropic said as early as December 19, 2024, in “Building effective agents”, that it had “spent more time optimizing [ses] tools as the overall instructions” from its agent for SWE-bench. On long tasks, “compaction is not enough”, adds Justin Young, of Anthropic, in “Effective harnesses for long-running agents”, November 26, 2025. His harness keeps a progress log and logs each session in git, from where the next one starts again.
Eight points for Gemini 3 Pro, 13.7 for GPT-5.2-Codex
Terminal-Bench 2.1 put five models through their editor’s harness and into Terminus 2, without specifying the settings for each submission. The in-house harness wins by 5.1 points for GPT-5.5 in Codex CLI, by 3.4 points for Claude Fable 5 in Claude Code and by 2.8 points for Claude Opus 4.7, at the limit of the margin of error. It loses 8.1 points with Gemini 3 Pro and is on par with Gemini 3.1 Pro. The editor therefore does not always have the best harness for his own model.
LangChain measured the effect by only touching the harness. Its agent deepagents-cli went from 52.8% to 66.5% on Terminal-Bench 2.0, and its post from February 17, 2026 writes: “We only reworked the harness and kept the fixed model, gpt-5.2-codex. » The 13.7 points come from the system instructions, the tools, a verification pass imposed before the exit, a counter which forces the agent to review its method when it retouches the same file too often, and a dosage of the reasoning effort, which LangChain stores in the harness.
Researchers from Tulane, Rutgers and Virginia Tech crossed three models, GPT-5.4, Kimi K2.6 and GLM-5.1, with three harnesses of increasing complexity, on 100 SWE-bench Verified tasks. In their May 7, 2026 position paper, changing harness moves the score from 8.5 to 13 points, changing model from 2.5 to 5 points, and six out of nine model comparisons reverse from one harness to the other. Each box is the average of two runs, with no published margin of error. On SWE-bench Pro, the effect is thinner. Claude Opus 4.5 obtains 45.9% in SWE-Agent, Scale AI’s SEAL ranking harness, within 3.6 points. On the same 731 tasks, an Augment Code post from February 4, 2026 measures it at 49.75% in Claude Code and 51.8% in Auggie, the Augment agent. The gap between these two harnesses remains within the Scale margin.
The Terminal-Bench authors remain more cautious. Their article from January 17, 2026 estimates that “the choice of model generally matters more than the scaffolding of the agent”, on a ranking which ranges from 63% to around 15% for small models. From one generation to the next, the model dominates. Between models of comparable level, those that a company really puts into competition, the harness can be enough to reverse the ranking. Claude Opus 4.7 is ahead of Gemini 3 Pro in the publishers’ harnesses, 68.9% against 65.8%, and falls behind it in Terminus 2. Claude Fable 5 and GPT-5.5, at the head of Terminal-Bench 2.1, however keep the same order in the two harnesses.
An agent session costs more each round
An API charges for all transmitted context on each call, and an agent session returns its history on each call, detailed ActuIA’s October 2026 API pricing statement. Michael Bolin, from OpenAI, recognizes in “Unrolling the Codex agent loop”, January 23, 2026, that the volume sent by Codex grows quadratically. The cache amortizes the expense, with a 95% discount on the token reread at Claude Opus 5.5 as at GPT-6.1 Sol. A harness that rewrites the start of its request during the session, to change its tools, its model or its sandbox, loses this discount; Codex adds these changes at the end of the query to keep it safe. It alone compacts the history beyond an adjustable threshold.
The choice of tools weighs just as much. In “Code execution with MCP”, on November 4, 2025, Anthropic estimated the cost of an agent who immediately loads the definition of all its tools at 150,000 tokens, compared to 2,000 when it calls them from code. Vercel removed 80% of the tools from d0, its agent which translates questions into SQL, mainly in favor of bash commands. Over five requests with Claude Opus 4.5, average consumption fell from around 102,000 to 61,000 tokens and success increased from four to five requests out of five. Five queries don’t make a statistic. Subscriptions follow the same rule. By doubling the Codex quotas of its Pro packages on October 29, OpenAI clarified that consumption varied with the model, the task and the settings, and that a doubled quota therefore did not double the number of tasks, according to The Decoder and ActuIA.
Model rejection happens after running the tool
The harness finally decides what an agent is allowed to do. A preprint, ToFu, showed it on October 9, 2026 on LangGraph with a small local model. This model issues a call to a dummy tool named launch_missiles, which LangGraph executes without control, then writes its refusal: the refusal arrives after execution. Verifying authorization, on the tool and its arguments, therefore comes down to the harness and the tools themselves. Subagents add a floor. GPT-6 Astra had received instructions from another agent to never deactivate, “even temporarily”, a monitor responsible for detecting model drift. On October 6, he delegated a migration asking to preserve this surveillance “to the extent appropriate to the exact PR, rather than inventing a new mandatory surveillance system”. The deputy removed the monitor, then replaced the test that verified its presence with one that requires its absence.
Asking for user consent provides poor protection, since users accept 93% of authorization requests, notes Anthropic in its post of March 25, 2026 on Claude Code’s automatic mode. Its sandbox, which limits modifiable directories and reachable servers, initially reduced these requests by 84% in internal use. A classifier then rereads, before execution, actions that are not already authorized, such as shell commands or network access; out of 52 actual excessive actions, he let 17% pass. OpenAI has also isolated the execution in its Agents SDK since April 15, 2026: “separating the harness and the calculation helps keep identifiers out of the environments where the code generated by the model executes”.
Evaluate the harness separately from the model
A company that chooses an agent buys a couple, and published figures often describe only half of them. The first question concerns the harness that produced the announced score, and its resemblance to the one that will be deployed. The SWE-bench Bash Only ranking, which puts each model in the same harness, mini-SWE-agent, compares models; Terminal-Bench, which publishes the pair, compares agents. It is then necessary to measure the cost on the entire task, as does the Coding Agent Index from Artificial Analysis, calculated at the API rate with reading and writing the cache. The risk can be seen in the default configuration: what is executed without agreement, what the sandbox allows to be attached, where the identifiers are stored, what instructions a sub-agent receives.
A harness also ages faster than a contract. Anthropic added context resets for Claude Sonnet 4.5, which ended its tasks too early when it sensed the context limit approaching; with Claude Opus 4.5, they had become “dead weight”. Each change of model requires reassessing the harness, and each change of harness requires repeating measurements.
Artificial Analysis has a section on its code agents page, “Harness Comparison”, which will compare harnesses. On October 30, 2026, it was marked “coming soon”.
Quotes translated from English by ActuIA.
Our articles will be prioritized in Google’s “Featured” section.
