AI Agent Harnesses in 2026: Gemini 3.6, GPT-5.6 and Production Controls

Google’s July 2026 Gemini releases are not simply another model-number update. Gemini 3.6 Flash lowers the cost of capable agent work, while Gemini API Managed Agents now provide the execution layer that turns a model response into a multi-step job: reasoning, web retrieval, code execution, file handling and tool use inside a hosted sandbox. Google has also added environment hooks, token budgets, scheduled triggers and a free tier. For marketers, SEO teams and agencies, the important shift is from “ask Gemini for an answer” to “give a controlled agent a job, a budget and an approval boundary.”

This guide separates the model launch from the managed-agent platform, checks the headline claims against Google’s official documentation, and translates the changes into practical marketing workflows. It also explains the limits: Managed Agents are still in public preview, one interaction may consume hundreds of thousands or even millions of tokens, and a sandbox does not make an autonomous action safe by itself.

Gemini 3.6 Flash and Managed Agents: the quick answer

Release	What changed	Why marketers should care
Gemini 3.6 Flash	New Flash workhorse with stronger coding, knowledge work, multimodal and computer-use results; $1.50 per million input tokens and $7.50 per million output tokens	Potentially better cost per completed research, analysis or production task than using a premium frontier model for everything
Gemini 3.5 Flash-Lite	High-throughput model priced at $0.30 input and $2.50 output per million tokens	Useful for classification, extraction, document processing and other high-volume steps that can be checked mechanically
Managed Agents	A hosted agent harness that can reason, browse, run code, install packages and manage files in an isolated Linux environment	Teams can automate multi-step jobs without building every orchestration component themselves
Environment hooks	Custom checks can run before or after agent tool calls	Teams can block risky writes, validate outputs, enforce naming rules or create audit records
Budgets and triggers	max_total_tokens can stop runaway work; cron-style triggers can run jobs on a schedule	Recurring monitoring becomes possible, but it needs explicit review and spend controls
Free tier	Managed Agents can be tested on projects without active billing, within free quotas	Small teams can evaluate the workflow before committing production budget

My verdict: Gemini 3.6 Flash is the model update, but hooks and budgets are the more consequential operational news. They address two reasons marketing agents fail in production: they act without deterministic checks, and they keep working after the marginal value has disappeared.

What exactly is Gemini 3.6 Flash?

Google released Gemini 3.6 Flash on 21 July 2026 as the main workhorse in its Flash family. The official positioning is not “maximum intelligence at any price.” It is a balance of reasoning, coding, multimodal work, latency and token efficiency for production agents. Google lists it at $1.50 per million input tokens and $7.50 per million output tokens.

Google says an Artificial Analysis run used 17% fewer output tokens than Gemini 3.5 Flash. It also reports model improvements on selected evaluations: DeepSWE rose from 37% to 49%, MLE-Bench from 49.7% to 63.9%, OSWorld-Verified from 78.4% to 83.0%, and GDPval-AA v2 from 1349 to 1421. These are useful indicators of direction, not a universal productivity guarantee. The outcome of an agent task also depends on the harness, tool descriptions, environment, retries, context, grader and permission design.

The cost implication is more subtle than “17% cheaper.” A model that uses fewer output tokens, takes fewer tool loops and needs fewer retries may lower the cost of an accepted result. But an agent that receives an open-ended brief can still spend far more than a short chat response. DMT’s analysis of cost per accepted AI marketing result is the right framework: include model tokens, tool calls, failed attempts, human review and correction—not only the list price.

Gemini 3.6 Flash versus Gemini 3.5 Flash-Lite

Use 3.6 Flash for work where judgment and tool coordination matter: synthesising mixed evidence, interpreting a spreadsheet, planning an experiment, coding an analysis or navigating a multi-step workflow. Use 3.5 Flash-Lite when the task is high-volume, narrower and easier to validate: extracting fields, classifying queries, cleaning product attributes, normalising campaign names or summarising many similar documents.

Do not route solely by price. If Flash-Lite’s output needs frequent repair, the lower token rate can become the more expensive workflow. Start with a representative evaluation set, measure acceptance rate and review time, then route each task class to the least expensive model that consistently passes.

What are Gemini API Managed Agents?

Managed Agents are a Google-hosted agent harness exposed through the Gemini API. A single interaction can provision a Linux sandbox in which the agent reasons, executes code, manages files and retrieves information from the web. The general-purpose Antigravity agent now uses Gemini 3.6 Flash by default, although developers can pin another supported model.

This is different from a normal model API call. A standard call usually receives input and returns output. A managed agent may plan, use several tools, inspect the result, revise its approach and continue across multiple loops before returning. That makes it more capable for outcome-based work—and harder to predict on cost, time and side effects.

Google’s documentation says a single managed-agent interaction typically consumes 100,000 to 3 million tokens. Environment compute is not billed during the public preview, but model tokens and tool usage still matter. That range is why budgets and evaluation are essential rather than optional.

The sandbox is useful, but it is not the safety boundary

Each agent runs in an operating-system-isolated environment. However, Google says outbound network access is unrestricted by default. Teams can restrict it with an allowlist, and they should. If an agent only needs Google Search Console exports, a specific analytics endpoint and a controlled storage location, it should not be able to browse or call every domain on the internet.

Credentials also need least privilege. Google warns that an agent may use any credential it can access. Use short-lived, narrowly scoped service identities and assume the full scope could be exercised. For a broader architecture discussion, see DMT’s guide to AI agent harnesses, context and control.

Why environment hooks are the most important update

Environment hooks let a team run its own script before or after an agent tool call. A pre-tool hook can inspect an attempted action and deny it. A post-tool hook can validate or transform what was produced. Matchers can target a particular tool or group of tools.

That turns a natural-language preference into an enforceable rule. “Please do not publish without approval” is only an instruction. A hook that blocks every publish or write call unless an approved record exists is a control.

Useful pre-tool hooks for marketing teams

• Publishing guard: block a CMS publish call unless status and approval fields are present and current.
• Budget guard: deny ad-platform changes above an authorised daily or campaign-level threshold.
• Domain allowlist: stop research from sending data to unapproved endpoints.
• PII check: prevent customer-level exports from being copied into a prompt or external tool.
• SEO mutation guard: block canonical, robots, redirect or structured-data changes unless the exact target was approved.

Useful post-tool hooks

• Run a link checker and source-count audit after an article draft is created.
• Validate campaign naming, UTM syntax and required columns after a media plan is generated.
• Check image dimensions, aspect ratio and file size after creative assets are produced.
• Compare a generated report against its source totals and fail on reconciliation differences.
• Create an immutable manifest of files, sources, model version and actions for audit.

Hooks do not make model judgment deterministic. They are best for rules that can be expressed as data checks, permissions, allowed values or executable tests.

Budget controls: how to stop an agent before it becomes wasteful

Managed Agents now accept a max_total_tokens value covering input, output and thinking. When the limit is reached, the interaction pauses with an incomplete status and preserves the environment. A team can inspect progress and continue later with a new budget.

This is a sound design for long jobs because “incomplete but resumable” is safer than either unlimited execution or losing all progress. The initial budget should be based on a tested task class, not an arbitrary large number.

1. Build a 20–50 task evaluation set from real marketing work.
2. Measure the token distribution, wall-clock time and accepted-output rate.
3. Set the normal budget near the upper end of successful routine tasks.
4. Pause exceptions for review rather than automatically doubling the limit.
5. Track cost per accepted output and the reason each job needed continuation.

This approach is compatible with a broader 2026 digital marketing strategy: use automation to compress repeatable work while preserving human judgment where a decision changes spend, messaging, customer data or public content.

Scheduled triggers: useful automation, not permission to run unattended

Scheduled triggers bind an agent, its environment, a prompt and a cron schedule. The sandbox persists across runs, which is useful for comparison files, prior exports and checkpoints. It also creates state-management risk: a stale file, outdated assumption or compromised dependency can affect later runs.

Good scheduled jobs are read-mostly, bounded and easy to verify. Examples include:

• Weekly extraction of Search Console query changes into a review report.
• Daily monitoring for broken campaign URLs or missing tracking parameters.
• Monthly content-inventory classification into keep, refresh, merge and remove candidates.
• Competitor and platform announcement monitoring that produces a source ledger, not an auto-published story.
• Creative-asset checks for dimensions, naming and expired campaign dates.

Risky scheduled actions include live budget changes, publishing, customer messaging, deleting files, changing redirects and altering access controls. Those need narrow authority and an approval checkpoint.

Five practical Gemini agent workflows for marketers

1. Source-backed AI and SEO news monitor

Give the agent an allowlist of official sources, a date window and a structured output: announcement, date, primary URL, facts, uncertainty, affected audience and potential DMT overlap. The agent can collect and rank candidates; an editor should decide the angle and publication. This avoids turning social-media velocity into unverified news.

2. Search Console opportunity review

Load a controlled export, have the agent calculate query and page changes, then group opportunities by intent. A post-tool hook can reconcile click and impression totals before the report is accepted. The output should recommend actions, not modify pages. DMT’s Search Console platform-properties guide shows how this model also applies to Instagram, TikTok, X and YouTube discovery data.

3. Content refresh package

The agent can compare a live article against current official sources, identify recency and information debt, draft replacement sections and create a change log. A publishing hook should block WordPress mutation until the target URL, diff and approval are present. Use a dedicated SEO content strategy and duplicate-intent check so the agent refreshes the canonical owner rather than spawning a competing page.

4. Campaign QA

Provide a campaign export and a deterministic checklist: final URLs, UTM fields, geographic targeting, conversion action, budget, spelling and policy-sensitive claims. Let Gemini analyse and explain anomalies. Keep the actual ad-platform mutation separate unless every allowed change is narrowly defined.

5. Multimodal document and creative audit

Gemini 3.6 Flash’s document, chart and image capabilities make it suitable for checking reports, screenshots and creative packages together. Use it to compare a media-plan spreadsheet, landing-page screenshot and messaging brief, then surface inconsistencies. Human review remains mandatory where visual judgment, legal claims or brand risk are involved.

A production-ready task contract

Goal: Produce a weekly AI and SEO opportunity report for DMT.
Sources: Only official vendor newsrooms, official documentation and named first-party posts.
Window: Announcements published in the last 10 days.
Output: A table with source URL, publication date, confirmed facts, audience impact,
search intent, existing DMT owner, refresh-vs-new decision and confidence.
Boundaries: Do not publish, message, change a CMS record or use unverified social claims as facts.
Budget: Stop at the configured token limit and return incomplete rather than expanding scope.
Checks: Reconcile source count; verify every date; flag conflicting vendor claims;
fail if a proposed new URL overlaps an existing canonical owner.

The value is not the wording of the prompt. It is the combination of clear acceptance criteria, restricted sources, least privilege, a hard budget and executable checks.

Limitations and unanswered questions

• Public preview: interfaces, limits, pricing and behaviour can change.
• Vendor benchmarks: Google’s results are useful launch evidence, but teams should run their own tasks and graders.
• Token range: 100,000 to 3 million tokens per interaction is broad; list price alone cannot predict job cost.
• Default network access: the sandbox allows unrestricted outbound access unless the developer limits it.
• Persistent state: scheduled runs can benefit from saved files, but stale state can contaminate later work.
• Human oversight: Google’s own documentation instructs users to verify generated code, transformations and configuration changes before deployment.
• Model versus system: a strong benchmark does not prove the surrounding agent, tools and permissions are reliable.

Frequently asked questions

Is Gemini 3.6 Flash available now?

Yes. Google says it is available through the Gemini API in Google AI Studio, Android Studio, Google Antigravity, Gemini Enterprise and the Gemini app. Product and regional availability can differ, so check the current official page before deployment.

How much does Gemini 3.6 Flash cost?

Google’s launch pricing is $1.50 per million input tokens and $7.50 per million output tokens. Managed-agent jobs can use multiple reasoning and tool loops, so the job cost may be much higher than one short completion.

Are Gemini Managed Agents free?

Google offers a free-tier quota for experimentation. Production use follows pay-as-you-go model and tool pricing. Environment compute is not billed during the preview, according to current documentation.

Can Managed Agents publish marketing content automatically?

They can technically call external tools if you provide access, but that does not make automatic publishing a good default. Require an approved record, validate sources and metadata, and use a pre-tool hook to block publication when the approval contract is not satisfied.

Does a sandbox make the agent safe?

No. Isolation reduces some risks, but the environment has unrestricted outbound network access by default and can exercise the credentials you provide. Use domain allowlists, least-privilege credentials, hooks, budgets and human review.

Should marketers use Gemini 3.6 Flash instead of Claude Opus 5 or GPT-5.6 Sol?

Not universally. Gemini 3.6 Flash is compelling for cost-sensitive multimodal and managed-agent workflows. Claude Opus 5 and GPT-5.6 Sol may be better for other task classes. Compare accepted-result rate, review burden, latency and total cost on your own evaluations. DMT’s Claude Opus 5 frontier comparison provides a source-audited starting point.

Bottom line

Gemini 3.6 Flash makes Google’s agent stack cheaper and more efficient, but the bigger advance is operational: Managed Agents now expose stronger ways to constrain, inspect and stop work. Marketers should not begin with “What can the agent automate?” Begin with “What outcome can we test, what authority does it need, what can a hook verify, and where must a person approve?”

The winning system is unlikely to be the model with the loudest launch benchmark. It will be the workflow that produces the highest proportion of accepted, source-backed results without losing control of spend, data or public actions.

Official sources and methodology

• Google: Introducing Gemini 3.6 Flash, 3.5 Flash-Lite and 3.5 Flash Cyber, published 21 July 2026.
• Google: Gemini API Managed Agents—3.6 Flash, hooks and more, published 28 July 2026.
• Google AI for Developers: Managed Agents overview, checked 9 August 2026.
• Google AI for Developers: Environment hooks documentation, checked 9 August 2026.
• Google: July 2026 Gemini Drop, published 31 July 2026.

Editorial method: Product facts, prices, availability and safeguards were taken from official Google launch posts and documentation. Vendor benchmark claims are labelled as such and interpreted with their system-level limitations. X discussions were reviewed for emerging questions, but no unverified tweet or trend summary was used as factual evidence. Featured image: original DMT editorial illustration generated for this article; it does not depict product UI or benchmark data.

---

GPT-5.6 case study: why the agent harness changes model performance

The Gemini Managed Agents release makes the harness a product. The following source-audited GPT-5.6 case study explains why the same model can perform very differently when reasoning continuity, compaction, tools and validation change.

Fact-checked July 30, 2026: An AI agent harness is the software layer around a model that manages instructions, tools, memory, permissions, execution loops, context and validation. OpenAI has now published an unusually clear demonstration of why that layer matters. In its ARC-AGI-3 public-set experiment, GPT-5.6 Sol scored 13.3% with the benchmark’s generic harness and 38.3% after OpenAI enabled retained reasoning and context compaction. The modified setup also used roughly six times fewer output tokens.

The model did not change. The task did not change. The surrounding system changed.

That is the useful lesson in Tibo’s July 29 post. The result is not evidence that every weak AI output can be fixed with two settings, and it does not make one public benchmark a final measure of intelligence. It does show that evaluating a model without evaluating its harness can produce a badly incomplete picture.

The GPT-5.6 Sol harness result in one table

What is an AI agent harness?

A language model predicts and generates the next useful output from the context it receives. An agent harness turns that model into a system that can work through a task over time. Depending on the product, the harness can include:

• Instructions: the objective, operating rules, scope and completion criteria.
• Tools: browsers, code execution, files, databases, search, APIs and connected applications.
• An execution loop: observe, reason, act, inspect the result and decide what to do next.
• Context management: what the model can see now, what is summarized, what is retrieved and what is discarded.
• Memory and state: durable facts, working notes, decisions, artifacts and task progress.
• Permissions: which reads and writes are allowed, which actions require approval and which destinations are out of scope.
• Validation: tests, graders, schema checks, source checks and human review.
• Recovery: retries, checkpoints, rollback paths, timeouts and escalation rules.
• Observability: logs, costs, latency, tool failures, state transitions and evidence of completion.

The simple mental model is agent = model + harness. That equation is useful because it stops teams from attributing every success or failure to the model alone.

This distinction is especially important when comparing products. Two agents can use the same model but perform differently because one has better tools, clearer permissions, more relevant context, stronger validation or a more reliable memory strategy. Conversely, an expensive model can look weak when the harness repeatedly removes the information it needs.

What Tibo’s post surfaced

Tibo, who works on Codex and ChatGPT at OpenAI, highlighted the company’s ARC-AGI-3 investigation on July 29. His summary focused on two settings: allow the model to carry its reasoning forward, and let it continue across context windows with compaction.

The post spread quickly because it offered a clean explanation for a common practitioner experience: an agent may look capable at the beginning of a task, then become repetitive, forget an earlier discovery or spend tokens rebuilding a plan it had already formed.

The tempting headline is that a benchmark score tripled. The more durable insight is that continuity is a capability. A model can only build on earlier work when the surrounding system preserves the right state and returns it in a usable form.

Why the original ARC-AGI-3 setup held the agent back

ARC-AGI-3 presents unfamiliar interactive environments without explicit instructions. An agent must explore, form hypotheses, test them, recover from mistakes and transfer what it learns from one level to the next. The official score, Relative Human Action Efficiency or RHAE, considers both completion and how efficiently the system acts relative to a human baseline.

OpenAI identified two context failures in the generic harness.

1. Reasoning was discarded after every action

The agent could see a record of earlier moves and short notes, but the deeper reasoning that produced those moves was not retained. After each interaction, it had to spend effort reconstructing the situation: What changed? Which hypothesis was being tested? Which rule had already been rejected? What should happen next?

That is similar to asking an analyst to investigate a campaign, deleting their working notes after every spreadsheet edit, and leaving only a terse audit log. The analyst still knows which cell changed, but not why the change mattered or which theory it was meant to test.

2. Rolling truncation removed older actions

The harness also handled a growing history by discarding the oldest messages after it crossed 175,000 characters. This kept the input bounded, but it created a moving blind spot. Earlier observations and actions disappeared precisely when a long task needed them for cumulative learning.

Rolling truncation is simple, but simplicity is not the same as neutrality. It decides that the oldest information is the least useful. In long-horizon work, the oldest information may contain the original objective, a critical constraint, an early discovery or the reason a tempting approach was abandoned.

Together, discarded reasoning and rolling truncation made the system repeatedly relearn the task. The model looked less capable because its harness kept weakening its continuity.

Retained reasoning: preserving the thread between actions

OpenAI rebuilt the experiment using the Responses API. For GPT-5.6, the company says that chaining with a previous response ID can preserve reasoning across tool calls and turns. The model is not shown a public, human-readable chain of thought; the API carries encrypted reasoning items and relevant state forward within the supported conversation flow.

That distinction matters. Retained reasoning does not mean publishing hidden reasoning to the user. It means the system does not force the model to begin every action as if the earlier thinking never happened.

OpenAI observed two changes after retaining reasoning:

• The model spent less time reinterpreting the environment before each action.
• It maintained more coherent strategies and learned more effectively across a longer sequence.

For developers, the current conversation-state documentation describes two broad patterns. A stateless implementation can replay the full response output, including encrypted reasoning items. A stateful implementation can chain turns using previous_response_id or a durable Conversation object. The important operational point is not the exact API style; it is that the next turn receives the state required to continue the same task.

Context compaction: compressing history without resetting the job

Every long-running agent eventually meets a context limit. Keeping the entire transcript forever increases input size, latency and cost, and it can leave the model working inside a crowded context. Deleting the oldest entries is cheap, but it can erase crucial state. Compaction is a third option: reduce the context while preserving the information needed for the next phase.

OpenAI replaced the benchmark harness’s rolling truncation with Responses API compaction. The returned compaction item is encrypted and opaque. It carries forward key state and reasoning in fewer tokens; it is not intended to be edited as a human-readable summary.

The current API documentation offers two implementation styles:

• Server-side compaction: configure a token threshold in a Responses request. When the rendered context crosses that threshold, the server compacts it within the same flow.
• Standalone compaction: send the current window to the stateless /responses/compact endpoint, then use the returned window as the canonical input for the next request.

Compaction is not free memory and it is not perfectly lossless. It is a controlled reduction. Good systems still keep important constraints, source artifacts and task state in durable forms outside a transient model context.

A minimal implementation pattern

The following example is deliberately small. It shows the shape of server-side compaction, not a complete production agent:

from openai import OpenAI

client = OpenAI()
response = client.responses.create(
    model="gpt-5.6",
    input="Audit the attached workflow and preserve the evidence trail.",
    store=False,
    context_management=[
        {"type": "compaction", "compact_threshold": 200000}
    ],
)

# On the next turn, either append the returned output items
# or continue with previous_response_id. Do not mix the two patterns.

The threshold should be tested against the selected model, task shape, tool-output volume, latency target and cost target. Do not copy one number into every workflow. A research agent reading long documents, a browser agent working through forms and a coding agent producing tool logs accumulate different kinds of context.

What changed in the reported score

OpenAI reports several numbers that can look contradictory unless their scopes are separated:

• GPT-5.6 Sol’s earlier reported ARC-AGI-3 score was 7.8% in the broader evaluation context discussed by OpenAI.
• On the public task set used for the controlled harness comparison, the official harness scored 13.3%.
• On that same public set, the Responses API harness with retained reasoning and compaction scored 38.3%.
• OpenAI estimates the average human tester at about 48% on the comparison shown in its publication.

The valid comparison is 13.3% versus 38.3% on the same public task set. Comparing 7.8% directly with 38.3% would mix different scopes. This is why benchmark coverage needs more than a screenshot of the highest number.

The token result is equally interesting: OpenAI says the improved setup achieved roughly three times the score with six times fewer output tokens. Retained reasoning reduced the need to reconstruct the task repeatedly, while compaction preserved useful history without letting the active window grow indefinitely.

What the experiment proves—and what it does not

There is a real evaluation tradeoff here. A generic harness helps standardize comparisons and reveal model behavior under equal constraints. A model-aware harness better reflects how a vendor deploys the model in a product. Neither answers every question. Responsible reporting should name which question the evaluation is trying to answer.

This is also why I would not put “state of the art” in the article title. Tibo used that framing in the fast-moving X discussion, but benchmark leaderboards and harness implementations can change quickly. The stable story is the controlled configuration result and the engineering lesson behind it.

Why capable agents can look inconsistent in real work

When an agent performs well in one session and poorly in another, the explanation may be a model change—but it may also be any of the following:

• The second run started without the original files, decisions or constraints.
• A tool returned more noisy output and crowded the context.
• The execution loop did not feed tool errors back to the model clearly.
• A summary omitted a rule that had previously prevented a bad action.
• The agent lost the connection between a plan, an action and the evidence used to judge it.
• A retry created a new conversation instead of continuing the same state.
• The product used a different model, reasoning level, permission set or context policy.
• The evaluator scored only the final answer and ignored how the system reached it.

This is the missing dimension in many model comparisons. My detailed Claude Opus 5 versus GPT-5.6 benchmark comparison separates model capability, pricing and evidence. The ARC-AGI-3 case adds another rule: the harness configuration belongs in the evidence table too.

What this means for marketers and business teams

Most marketers will not build an ARC benchmark runner, but they increasingly use agents for research, SEO, reporting, creative production and website work. Those tasks have the same long-horizon risks.

SEO research and content operations

An SEO agent may collect Search Console queries, inspect existing URLs, research sources, build an outline, draft, check duplicates, add internal links and prepare publishing metadata. If the system forgets the existing URL inventory after drafting begins, it can create competing pages for the same search intent. If it loses the source ledger during compaction, it can turn a verified fact into an unsupported claim.

A better harness stores the URL inventory, approval state, source ledger and publishing gates as durable artifacts. The model context can be compacted; the evidence should not disappear. This is the same reason my GPT-5.6 prompting and workflow guide treats the prompt as only one part of a repeatable process.

Paid-media analysis

A paid-media agent can spend hours moving from account identity to campaign data, search terms, landing pages, conversion tracking and CRM outcomes. A weak harness may remember the latest chart but forget the approved date range, excluded campaigns or instruction not to change budgets.

The solution is not simply a bigger context window. Put non-negotiable scope and permissions in pinned system controls, maintain a structured audit state and require the final recommendation to cite the reviewed evidence. Compaction should preserve the analysis; it should not rewrite the authorization boundary.

Website migrations and technical audits

Long website tasks accumulate routes, redirects, templates, test results, screenshots and deployment constraints. If earlier failures fall out of context, the agent can repeat a broken fix. If a successful verification is not recorded outside the conversation, the final report can overstate what was actually checked.

Use checkpoints: current state, changed files, test receipts, unresolved failures and next safe action. My ChatGPT Work and Codex guide for marketers covers how tools and workflows fit together; the harness lesson explains why durable state between those tools is essential.

Multi-agent content and research pipelines

Multiple agents increase parallelism, but they also increase coordination overhead. A research agent, writer and reviewer need a shared specification, source policy and definition of done. Passing a long chat transcript between them is not enough. The harness should route structured artifacts and preserve which claims are confirmed, uncertain or rejected.

This is where context engineering becomes operational rather than rhetorical. The Claude 5 context-engineering guide explores files, memory and tools from another ecosystem; the same principle applies here: the useful unit is not the clever prompt but the maintained working state.

A practical AI agent harness audit

Before blaming or upgrading the model, review the system around it.

1. Define the outcome. Write a concrete completion condition that can be checked without trusting the agent’s own confidence.
2. Inventory the tools. Record which tools are available, which destinations they can reach and which actions are read-only or mutating.
3. Separate durable state from chat history. Store scope, decisions, approvals, evidence and progress in structured artifacts.
4. Trace state across turns. Confirm that plans, tool results and reasoning continuity survive the next action.
5. Inspect truncation behavior. Find out what disappears first when the context grows.
6. Test compaction. Compare the pre- and post-compaction task state. Check whether constraints, open questions and evidence links remain available.
7. Pin safety-critical rules. Do not rely on a lossy summary to preserve authorization boundaries, legal constraints or irreversible-action rules.
8. Control tool noise. Summarize repetitive logs and keep raw evidence in files that can be retrieved when needed.
9. Add checkpoints. After each major phase, write what changed, what passed, what failed and what remains.
10. Validate externally. Run tests, compare totals, inspect rendered pages or ask a human reviewer to check high-impact decisions.
11. Measure cost and latency. Track input tokens, output tokens, tool time, retries and compaction events—not only the final model bill.
12. Run the same task twice. Reliability includes variance. One impressive demonstration is not a production guarantee.

How to compare two agent systems fairly

A fair agent comparison needs more than the same prompt. Use the same task, source material, success criteria and maximum permissions, then disclose the variables that remain different.

For long Codex sessions, context size can also affect usage economics. The evidence-based GPT-5.6 Sol usage guide explains the separate question of context, pricing and plan limits. Do not treat the ARC experiment’s sixfold output-token reduction as a promise about ChatGPT subscription allowances or every API workload.

Should you build your own agent harness?

Usually, not at first.

If a managed product already provides the needed tools, permissions, state management and audit trail, use it and evaluate the complete workflow. Build custom orchestration when the business requirement cannot be met safely through configuration: specialized tools, regulated approval steps, proprietary state, unusual latency needs or a repeatable high-volume workflow with measurable economics.

A custom harness creates its own obligations:

• Secure credential and permission handling.
• State migration when models or APIs change.
• Observability for tool calls, failures, retries and costs.
• Evaluation sets that represent real work rather than demos.
• Human escalation and rollback paths.
• Protection against prompt injection and malicious tool output.
• Compaction and retrieval tests that confirm important constraints survive.

The goal is not to own more infrastructure. The goal is to make the agent more reliable, inspectable and useful.

My take: harness engineering is becoming part of model selection

Model selection used to look like a leaderboard exercise: choose the model with the best benchmark score within the financial limit. Agentic work makes that incomplete. Teams now need to choose a model-harness combination.

The best model for a task may be the one that works predictably with the available tools, remembers the right state, compacts without losing constraints and produces evidence a reviewer can inspect. A theoretically stronger model inside a brittle loop can be the worse business system.

OpenAI’s ARC-AGI-3 result is valuable because it makes this visible with a controlled example. It should encourage better experiments, not end the benchmark debate. Vendors should disclose harness settings. Benchmark publishers should distinguish standardized and production-style runs. Practitioners should test their own workflows instead of importing a score as a universal verdict.

Frequently asked questions

What is the difference between an AI model and an agent harness?

The model generates outputs from context. The harness manages how the model receives instructions and state, which tools it can use, how actions are executed, what is remembered, what requires approval and how results are checked.

What is context compaction?

Context compaction reduces the size of a long interaction while preserving the state needed to continue. It is different from deleting the oldest messages blindly. OpenAI’s implementation returns an encrypted, opaque compaction item that carries key state into the next window.

Is context compaction the same as a larger context window?

No. A larger window lets a request contain more tokens. Compaction reduces accumulated history so a long-running task can continue within a bounded window. Systems may use both.

Did GPT-5.6 Sol become three times smarter?

No. OpenAI reported roughly three times the RHAE score on the ARC-AGI-3 public-set comparison after changing retained-reasoning and compaction settings. That result is specific to the tested configuration and tasks.

Why were output tokens lower in the better run?

With useful reasoning and state preserved, the model spent less effort reconstructing the task after each action. Compaction also prevented the active context from growing through simple accumulation. The combination produced better task progress with less repeated output in this experiment.

Can compaction lose important information?

Yes. Any compressed representation can omit details. Keep safety rules, approvals, source evidence and critical task state in durable structures, then test what survives compaction before relying on it in high-impact workflows.

Does a generic benchmark harness unfairly penalize a model?

It depends on the evaluation question. A generic harness helps standardize conditions across models. A model-aware harness can better represent production deployment. Good reporting should label the setup and avoid presenting either result as the only valid view.

What should a marketer check first in an AI agent workflow?

Check whether the agent preserves scope, source evidence, approvals and progress across turns. Then verify tool permissions, compaction behavior, final-output validation, cost and variance across repeated runs.

Sources and editorial method

This article began with a review of Tibo’s recent posts on X, then checked the linked OpenAI publication and current OpenAI developer documentation. Benchmark definitions and scoring were checked against ARC Prize’s official methodology. Keyword demand was reviewed through Google Keyword Planner for India; “agent harness” showed approximately 880 monthly searches, while the more specific terms were emerging and lower volume.

The article deliberately separates confirmed experiment results from interpretation. It does not treat engagement on X as evidence of technical truth, does not convert a public-set result into a hidden-set claim and does not extrapolate the reported token reduction to every product or pricing plan.

Author note: I write DMT’s AI coverage from a digital-marketing practitioner’s perspective: what changed, what the evidence supports and how a team should alter its workflow. Product names and benchmark results in this article are current as of July 30, 2026 and should be rechecked before procurement or production decisions.