{"id":2718,"date":"2026-07-30T06:14:36","date_gmt":"2026-07-30T06:14:36","guid":{"rendered":"https:\/\/dmarketertayeeb.com\/blog\/ai-agent-harness-context-compaction\/"},"modified":"2026-07-30T06:15:23","modified_gmt":"2026-07-30T06:15:23","slug":"ai-agent-harness-context-compaction","status":"publish","type":"post","link":"https:\/\/dmarketertayeeb.com\/blog\/ai-agent-harness-context-compaction\/","title":{"rendered":"AI Agent Harnesses: GPT-5.6&#8217;s ARC-AGI-3 Lesson"},"content":{"rendered":"\n<p><strong>Fact-checked July 30, 2026:<\/strong> An AI agent harness is the software layer around a model that manages instructions, tools, memory, permissions, execution loops, context and validation. OpenAI has now published an unusually clear demonstration of why that layer matters. In its ARC-AGI-3 public-set experiment, GPT-5.6 Sol scored 13.3% with the benchmark&#8217;s generic harness and 38.3% after OpenAI enabled retained reasoning and context compaction. The modified setup also used roughly six times fewer output tokens.<\/p>\n\n\n\n<p>The model did not change. The task did not change. The surrounding system changed.<\/p>\n\n\n\n<p>That is the useful lesson in <a href=\"https:\/\/x.com\/thsottiaux\/status\/2082609662231502932\" rel=\"nofollow noopener\" target=\"_blank\">Tibo&#8217;s July 29 post<\/a>. The result is not evidence that every weak AI output can be fixed with two settings, and it does not make one public benchmark a final measure of intelligence. It does show that evaluating a model without evaluating its harness can produce a badly incomplete picture.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">The GPT-5.6 Sol harness result in one table<\/h2>\n\n\n<div class=\"wp-block-table\">\n\n<figure class=\"wp-block-table\"><table><thead><tr><th>Question<\/th><th>Evidence-backed answer<\/th><\/tr><\/thead><tbody>\n<tr><td>What is an AI agent harness?<\/td><td>The orchestration layer that supplies context, exposes tools, runs the action loop, preserves state, applies permissions and checks results around a foundation model.<\/td><\/tr>\n<tr><td>What benchmark did OpenAI examine?<\/td><td>ARC-AGI-3, an interactive benchmark in which agents explore unfamiliar 2D environments and infer their rules.<\/td><\/tr>\n<tr><td>What was wrong with the generic setup?<\/td><td>It discarded private reasoning after each action and removed older history with a rolling truncation window.<\/td><\/tr>\n<tr><td>What did OpenAI change?<\/td><td>It retained reasoning across turns and replaced rolling truncation with compaction through the Responses API.<\/td><\/tr>\n<tr><td>What happened on the public task set?<\/td><td>The reported RHAE score rose from 13.3% to 38.3%, while output-token use fell by about six times.<\/td><\/tr>\n<tr><td>Did the model weights change?<\/td><td>No. This was a harness and API-settings change, not a new model checkpoint.<\/td><\/tr>\n<tr><td>Does this prove the model is three times smarter?<\/td><td>No. It proves this configuration performed roughly three times better on this public-set evaluation.<\/td><\/tr>\n<tr><td>What should practitioners do?<\/td><td>Measure model, prompt and harness as separate variables; preserve useful state; compact deliberately; and validate the complete workflow on representative tasks.<\/td><\/tr>\n<\/tbody><\/table><\/figure>\n\n<\/div>\n\n\n<h2 class=\"wp-block-heading\">What is an AI agent harness?<\/h2>\n\n\n\n<p>A language model predicts and generates the next useful output from the context it receives. An agent harness turns that model into a system that can work through a task over time. Depending on the product, the harness can include:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n  <li><strong>Instructions:<\/strong> the objective, operating rules, scope and completion criteria.<\/li>\n  <li><strong>Tools:<\/strong> browsers, code execution, files, databases, search, APIs and connected applications.<\/li>\n  <li><strong>An execution loop:<\/strong> observe, reason, act, inspect the result and decide what to do next.<\/li>\n  <li><strong>Context management:<\/strong> what the model can see now, what is summarized, what is retrieved and what is discarded.<\/li>\n  <li><strong>Memory and state:<\/strong> durable facts, working notes, decisions, artifacts and task progress.<\/li>\n  <li><strong>Permissions:<\/strong> which reads and writes are allowed, which actions require approval and which destinations are out of scope.<\/li>\n  <li><strong>Validation:<\/strong> tests, graders, schema checks, source checks and human review.<\/li>\n  <li><strong>Recovery:<\/strong> retries, checkpoints, rollback paths, timeouts and escalation rules.<\/li>\n  <li><strong>Observability:<\/strong> logs, costs, latency, tool failures, state transitions and evidence of completion.<\/li>\n<\/ul>\n\n\n\n<p>The simple mental model is <em>agent = model + harness<\/em>. That equation is useful because it stops teams from attributing every success or failure to the model alone.<\/p>\n\n\n\n<p>This distinction is especially important when comparing products. Two agents can use the same model but perform differently because one has better tools, clearer permissions, more relevant context, stronger validation or a more reliable memory strategy. Conversely, an expensive model can look weak when the harness repeatedly removes the information it needs.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">What Tibo&#8217;s post surfaced<\/h2>\n\n\n\n<p>Tibo, who works on Codex and ChatGPT at OpenAI, highlighted the company&#8217;s <a href=\"https:\/\/openai.com\/index\/how-two-settings-tripled-our-arc-agi-3-scores\/\" rel=\"nofollow noopener\" target=\"_blank\">ARC-AGI-3 investigation<\/a> on July 29. His summary focused on two settings: allow the model to carry its reasoning forward, and let it continue across context windows with compaction.<\/p>\n\n\n\n<p>The post spread quickly because it offered a clean explanation for a common practitioner experience: an agent may look capable at the beginning of a task, then become repetitive, forget an earlier discovery or spend tokens rebuilding a plan it had already formed.<\/p>\n\n\n\n<p>The tempting headline is that a benchmark score tripled. The more durable insight is that <strong>continuity is a capability<\/strong>. A model can only build on earlier work when the surrounding system preserves the right state and returns it in a usable form.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Why the original ARC-AGI-3 setup held the agent back<\/h2>\n\n\n\n<p><a href=\"https:\/\/arcprize.org\/arc-agi\/3\" rel=\"nofollow noopener\" target=\"_blank\">ARC-AGI-3<\/a> presents unfamiliar interactive environments without explicit instructions. An agent must explore, form hypotheses, test them, recover from mistakes and transfer what it learns from one level to the next. The official score, Relative Human Action Efficiency or RHAE, considers both completion and how efficiently the system acts relative to a human baseline.<\/p>\n\n\n\n<p>OpenAI identified two context failures in the generic harness.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">1. Reasoning was discarded after every action<\/h3>\n\n\n\n<p>The agent could see a record of earlier moves and short notes, but the deeper reasoning that produced those moves was not retained. After each interaction, it had to spend effort reconstructing the situation: What changed? Which hypothesis was being tested? Which rule had already been rejected? What should happen next?<\/p>\n\n\n\n<p>That is similar to asking an analyst to investigate a campaign, deleting their working notes after every spreadsheet edit, and leaving only a terse audit log. The analyst still knows which cell changed, but not why the change mattered or which theory it was meant to test.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">2. Rolling truncation removed older actions<\/h3>\n\n\n\n<p>The harness also handled a growing history by discarding the oldest messages after it crossed 175,000 characters. This kept the input bounded, but it created a moving blind spot. Earlier observations and actions disappeared precisely when a long task needed them for cumulative learning.<\/p>\n\n\n\n<p>Rolling truncation is simple, but simplicity is not the same as neutrality. It decides that the oldest information is the least useful. In long-horizon work, the oldest information may contain the original objective, a critical constraint, an early discovery or the reason a tempting approach was abandoned.<\/p>\n\n\n\n<p>Together, discarded reasoning and rolling truncation made the system repeatedly relearn the task. The model looked less capable because its harness kept weakening its continuity.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Retained reasoning: preserving the thread between actions<\/h2>\n\n\n\n<p>OpenAI rebuilt the experiment using the Responses API. For GPT-5.6, the company says that chaining with a previous response ID can preserve reasoning across tool calls and turns. The model is not shown a public, human-readable chain of thought; the API carries encrypted reasoning items and relevant state forward within the supported conversation flow.<\/p>\n\n\n\n<p>That distinction matters. Retained reasoning does not mean publishing hidden reasoning to the user. It means the system does not force the model to begin every action as if the earlier thinking never happened.<\/p>\n\n\n\n<p>OpenAI observed two changes after retaining reasoning:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n  <li>The model spent less time reinterpreting the environment before each action.<\/li>\n  <li>It maintained more coherent strategies and learned more effectively across a longer sequence.<\/li>\n<\/ul>\n\n\n\n<p>For developers, the current <a href=\"https:\/\/developers.openai.com\/api\/docs\/guides\/conversation-state\" rel=\"nofollow noopener\" target=\"_blank\">conversation-state documentation<\/a> describes two broad patterns. A stateless implementation can replay the full response output, including encrypted reasoning items. A stateful implementation can chain turns using <code>previous_response_id<\/code> or a durable Conversation object. The important operational point is not the exact API style; it is that the next turn receives the state required to continue the same task.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Context compaction: compressing history without resetting the job<\/h2>\n\n\n\n<p>Every long-running agent eventually meets a context limit. Keeping the entire transcript forever increases input size, latency and cost, and it can leave the model working inside a crowded context. Deleting the oldest entries is cheap, but it can erase crucial state. Compaction is a third option: reduce the context while preserving the information needed for the next phase.<\/p>\n\n\n\n<p>OpenAI replaced the benchmark harness&#8217;s rolling truncation with <a href=\"https:\/\/developers.openai.com\/api\/docs\/guides\/compaction\" rel=\"nofollow noopener\" target=\"_blank\">Responses API compaction<\/a>. The returned compaction item is encrypted and opaque. It carries forward key state and reasoning in fewer tokens; it is not intended to be edited as a human-readable summary.<\/p>\n\n\n\n<p>The current API documentation offers two implementation styles:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n  <li><strong>Server-side compaction:<\/strong> configure a token threshold in a Responses request. When the rendered context crosses that threshold, the server compacts it within the same flow.<\/li>\n  <li><strong>Standalone compaction:<\/strong> send the current window to the stateless <code>\/responses\/compact<\/code> endpoint, then use the returned window as the canonical input for the next request.<\/li>\n<\/ul>\n\n\n\n<p>Compaction is not free memory and it is not perfectly lossless. It is a controlled reduction. Good systems still keep important constraints, source artifacts and task state in durable forms outside a transient model context.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">A minimal implementation pattern<\/h3>\n\n\n\n<p>The following example is deliberately small. It shows the shape of server-side compaction, not a complete production agent:<\/p>\n\n\n\n<pre><code>from openai import OpenAI\n\nclient = OpenAI()\nresponse = client.responses.create(\n    model=\"gpt-5.6\",\n    input=\"Audit the attached workflow and preserve the evidence trail.\",\n    store=False,\n    context_management=[\n        {\"type\": \"compaction\", \"compact_threshold\": 200000}\n    ],\n)\n\n# On the next turn, either append the returned output items\n# or continue with previous_response_id. Do not mix the two patterns.<\/code><\/pre>\n\n<p>The threshold should be tested against the selected model, task shape, tool-output volume, latency target and cost target. Do not copy one number into every workflow. A research agent reading long documents, a browser agent working through forms and a coding agent producing tool logs accumulate different kinds of context.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">What changed in the reported score<\/h2>\n\n\n\n<p>OpenAI reports several numbers that can look contradictory unless their scopes are separated:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n  <li>GPT-5.6 Sol&#8217;s earlier reported ARC-AGI-3 score was 7.8% in the broader evaluation context discussed by OpenAI.<\/li>\n  <li>On the public task set used for the controlled harness comparison, the official harness scored 13.3%.<\/li>\n  <li>On that same public set, the Responses API harness with retained reasoning and compaction scored 38.3%.<\/li>\n  <li>OpenAI estimates the average human tester at about 48% on the comparison shown in its publication.<\/li>\n<\/ul>\n\n\n\n<p>The valid comparison is 13.3% versus 38.3% on the same public task set. Comparing 7.8% directly with 38.3% would mix different scopes. This is why benchmark coverage needs more than a screenshot of the highest number.<\/p>\n\n\n\n<p>The token result is equally interesting: OpenAI says the improved setup achieved roughly three times the score with six times fewer output tokens. Retained reasoning reduced the need to reconstruct the task repeatedly, while compaction preserved useful history without letting the active window grow indefinitely.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">What the experiment proves\u2014and what it does not<\/h2>\n\n\n<div class=\"wp-block-table\">\n\n<figure class=\"wp-block-table\"><table><thead><tr><th>Supported conclusion<\/th><th>Unsupported leap<\/th><\/tr><\/thead><tbody>\n<tr><td>Harness configuration materially affected GPT-5.6 Sol&#8217;s result on this evaluation.<\/td><td>The model became three times more intelligent in every domain.<\/td><\/tr>\n<tr><td>Retaining reasoning and using compaction improved continuity in a long interactive task.<\/td><td>Compaction always improves accuracy or can never lose important information.<\/td><\/tr>\n<tr><td>Generic harnesses can understate a model&#8217;s production-style performance.<\/td><td>Every model should be evaluated only with a vendor-optimized harness.<\/td><\/tr>\n<tr><td>Model comparisons should document context and orchestration settings.<\/td><td>Benchmark results are useless.<\/td><\/tr>\n<tr><td>Lower output-token use can coexist with better task performance when repeated reconstruction is removed.<\/td><td>A sixfold token reduction will apply to every workload or pricing plan.<\/td><\/tr>\n<tr><td>Public-set experiments are valuable for diagnosis.<\/td><td>A public-set result is equivalent to a verified hidden-set score.<\/td><\/tr>\n<\/tbody><\/table><\/figure>\n\n<\/div>\n\n\n<p>There is a real evaluation tradeoff here. A generic harness helps standardize comparisons and reveal model behavior under equal constraints. A model-aware harness better reflects how a vendor deploys the model in a product. Neither answers every question. Responsible reporting should name which question the evaluation is trying to answer.<\/p>\n\n\n\n<p>This is also why I would not put &#8220;state of the art&#8221; in the article title. Tibo used that framing in the fast-moving X discussion, but benchmark leaderboards and harness implementations can change quickly. The stable story is the controlled configuration result and the engineering lesson behind it.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Why capable agents can look inconsistent in real work<\/h2>\n\n\n\n<p>When an agent performs well in one session and poorly in another, the explanation may be a model change\u2014but it may also be any of the following:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n  <li>The second run started without the original files, decisions or constraints.<\/li>\n  <li>A tool returned more noisy output and crowded the context.<\/li>\n  <li>The execution loop did not feed tool errors back to the model clearly.<\/li>\n  <li>A summary omitted a rule that had previously prevented a bad action.<\/li>\n  <li>The agent lost the connection between a plan, an action and the evidence used to judge it.<\/li>\n  <li>A retry created a new conversation instead of continuing the same state.<\/li>\n  <li>The product used a different model, reasoning level, permission set or context policy.<\/li>\n  <li>The evaluator scored only the final answer and ignored how the system reached it.<\/li>\n<\/ul>\n\n\n\n<p>This is the missing dimension in many model comparisons. My detailed <a href=\"https:\/\/dmarketertayeeb.com\/blog\/claude-opus-5-benchmarks-pricing-comparison\">Claude Opus 5 versus GPT-5.6 benchmark comparison<\/a> separates model capability, pricing and evidence. The ARC-AGI-3 case adds another rule: the harness configuration belongs in the evidence table too.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">What this means for marketers and business teams<\/h2>\n\n\n\n<p>Most marketers will not build an ARC benchmark runner, but they increasingly use agents for research, SEO, reporting, creative production and website work. Those tasks have the same long-horizon risks.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">SEO research and content operations<\/h3>\n\n\n\n<p>An SEO agent may collect Search Console queries, inspect existing URLs, research sources, build an outline, draft, check duplicates, add internal links and prepare publishing metadata. If the system forgets the existing URL inventory after drafting begins, it can create competing pages for the same search intent. If it loses the source ledger during compaction, it can turn a verified fact into an unsupported claim.<\/p>\n\n\n\n<p>A better harness stores the URL inventory, approval state, source ledger and publishing gates as durable artifacts. The model context can be compacted; the evidence should not disappear. This is the same reason my <a href=\"https:\/\/dmarketertayeeb.com\/blog\/gpt-5-6-sol-terra-luna-marketers-guide\">GPT-5.6 prompting and workflow guide<\/a> treats the prompt as only one part of a repeatable process.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Paid-media analysis<\/h3>\n\n\n\n<p>A paid-media agent can spend hours moving from account identity to campaign data, search terms, landing pages, conversion tracking and CRM outcomes. A weak harness may remember the latest chart but forget the approved date range, excluded campaigns or instruction not to change budgets.<\/p>\n\n\n\n<p>The solution is not simply a bigger context window. Put non-negotiable scope and permissions in pinned system controls, maintain a structured audit state and require the final recommendation to cite the reviewed evidence. Compaction should preserve the analysis; it should not rewrite the authorization boundary.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Website migrations and technical audits<\/h3>\n\n\n\n<p>Long website tasks accumulate routes, redirects, templates, test results, screenshots and deployment constraints. If earlier failures fall out of context, the agent can repeat a broken fix. If a successful verification is not recorded outside the conversation, the final report can overstate what was actually checked.<\/p>\n\n\n\n<p>Use checkpoints: current state, changed files, test receipts, unresolved failures and next safe action. My <a href=\"https:\/\/dmarketertayeeb.com\/blog\/openai-codex-marketers-plugins-sites-workflows\">ChatGPT Work and Codex guide for marketers<\/a> covers how tools and workflows fit together; the harness lesson explains why durable state between those tools is essential.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Multi-agent content and research pipelines<\/h3>\n\n\n\n<p>Multiple agents increase parallelism, but they also increase coordination overhead. A research agent, writer and reviewer need a shared specification, source policy and definition of done. Passing a long chat transcript between them is not enough. The harness should route structured artifacts and preserve which claims are confirmed, uncertain or rejected.<\/p>\n\n\n\n<p>This is where context engineering becomes operational rather than rhetorical. The <a href=\"https:\/\/dmarketertayeeb.com\/blog\/claude-opus-5-prompting-workflow-guide\">Claude 5 context-engineering guide<\/a> explores files, memory and tools from another ecosystem; the same principle applies here: the useful unit is not the clever prompt but the maintained working state.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">A practical AI agent harness audit<\/h2>\n\n\n\n<p>Before blaming or upgrading the model, review the system around it.<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n  <li><strong>Define the outcome.<\/strong> Write a concrete completion condition that can be checked without trusting the agent&#8217;s own confidence.<\/li>\n  <li><strong>Inventory the tools.<\/strong> Record which tools are available, which destinations they can reach and which actions are read-only or mutating.<\/li>\n  <li><strong>Separate durable state from chat history.<\/strong> Store scope, decisions, approvals, evidence and progress in structured artifacts.<\/li>\n  <li><strong>Trace state across turns.<\/strong> Confirm that plans, tool results and reasoning continuity survive the next action.<\/li>\n  <li><strong>Inspect truncation behavior.<\/strong> Find out what disappears first when the context grows.<\/li>\n  <li><strong>Test compaction.<\/strong> Compare the pre- and post-compaction task state. Check whether constraints, open questions and evidence links remain available.<\/li>\n  <li><strong>Pin safety-critical rules.<\/strong> Do not rely on a lossy summary to preserve authorization boundaries, legal constraints or irreversible-action rules.<\/li>\n  <li><strong>Control tool noise.<\/strong> Summarize repetitive logs and keep raw evidence in files that can be retrieved when needed.<\/li>\n  <li><strong>Add checkpoints.<\/strong> After each major phase, write what changed, what passed, what failed and what remains.<\/li>\n  <li><strong>Validate externally.<\/strong> Run tests, compare totals, inspect rendered pages or ask a human reviewer to check high-impact decisions.<\/li>\n  <li><strong>Measure cost and latency.<\/strong> Track input tokens, output tokens, tool time, retries and compaction events\u2014not only the final model bill.<\/li>\n  <li><strong>Run the same task twice.<\/strong> Reliability includes variance. One impressive demonstration is not a production guarantee.<\/li>\n<\/ol>\n\n\n\n<h2 class=\"wp-block-heading\">How to compare two agent systems fairly<\/h2>\n\n\n\n<p>A fair agent comparison needs more than the same prompt. Use the same task, source material, success criteria and maximum permissions, then disclose the variables that remain different.<\/p>\n\n\n<div class=\"wp-block-table\">\n\n<figure class=\"wp-block-table\"><table><thead><tr><th>Layer<\/th><th>What to record<\/th><th>Why it matters<\/th><\/tr><\/thead><tbody>\n<tr><td>Model<\/td><td>Exact model, version and reasoning setting<\/td><td>Capability and cost can change across versions and effort levels.<\/td><\/tr>\n<tr><td>Prompt<\/td><td>System instructions, task requirements and examples<\/td><td>Different definitions of done can produce different behavior.<\/td><\/tr>\n<tr><td>Harness<\/td><td>Execution loop, retries, planning method and tool routing<\/td><td>The same model can take a different path through the task.<\/td><\/tr>\n<tr><td>Context<\/td><td>Input artifacts, history, retrieval, truncation and compaction<\/td><td>Continuity determines whether the agent builds on earlier work.<\/td><\/tr>\n<tr><td>Tools<\/td><td>Available tools, versions, permissions and failure handling<\/td><td>A stronger tool surface can look like stronger reasoning.<\/td><\/tr>\n<tr><td>Validation<\/td><td>Tests, graders, human checks and hidden holdouts<\/td><td>Self-reported completion is not independent evidence.<\/td><\/tr>\n<tr><td>Economics<\/td><td>Input, output, reasoning, tool, retry and latency costs<\/td><td>A lower model price can be offset by repeated or inefficient work.<\/td><\/tr>\n<\/tbody><\/table><\/figure>\n\n<\/div>\n\n\n<p>For long Codex sessions, context size can also affect usage economics. The <a href=\"https:\/\/dmarketertayeeb.com\/blog\/gpt-5-6-sol-codex-usage-272k-context-pricing\">evidence-based GPT-5.6 Sol usage guide<\/a> explains the separate question of context, pricing and plan limits. Do not treat the ARC experiment&#8217;s sixfold output-token reduction as a promise about ChatGPT subscription allowances or every API workload.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Should you build your own agent harness?<\/h2>\n\n\n\n<p>Usually, not at first.<\/p>\n\n\n\n<p>If a managed product already provides the needed tools, permissions, state management and audit trail, use it and evaluate the complete workflow. Build custom orchestration when the business requirement cannot be met safely through configuration: specialized tools, regulated approval steps, proprietary state, unusual latency needs or a repeatable high-volume workflow with measurable economics.<\/p>\n\n\n\n<p>A custom harness creates its own obligations:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n  <li>Secure credential and permission handling.<\/li>\n  <li>State migration when models or APIs change.<\/li>\n  <li>Observability for tool calls, failures, retries and costs.<\/li>\n  <li>Evaluation sets that represent real work rather than demos.<\/li>\n  <li>Human escalation and rollback paths.<\/li>\n  <li>Protection against prompt injection and malicious tool output.<\/li>\n  <li>Compaction and retrieval tests that confirm important constraints survive.<\/li>\n<\/ul>\n\n\n\n<p>The goal is not to own more infrastructure. The goal is to make the agent more reliable, inspectable and useful.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">My take: harness engineering is becoming part of model selection<\/h2>\n\n\n\n<p>Model selection used to look like a leaderboard exercise: choose the model with the best benchmark score within the financial limit. Agentic work makes that incomplete. Teams now need to choose a model-harness combination.<\/p>\n\n\n\n<p>The best model for a task may be the one that works predictably with the available tools, remembers the right state, compacts without losing constraints and produces evidence a reviewer can inspect. A theoretically stronger model inside a brittle loop can be the worse business system.<\/p>\n\n\n\n<p>OpenAI&#8217;s ARC-AGI-3 result is valuable because it makes this visible with a controlled example. It should encourage better experiments, not end the benchmark debate. Vendors should disclose harness settings. Benchmark publishers should distinguish standardized and production-style runs. Practitioners should test their own workflows instead of importing a score as a universal verdict.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Frequently asked questions<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">What is the difference between an AI model and an agent harness?<\/h3>\n\n\n\n<p>The model generates outputs from context. The harness manages how the model receives instructions and state, which tools it can use, how actions are executed, what is remembered, what requires approval and how results are checked.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">What is context compaction?<\/h3>\n\n\n\n<p>Context compaction reduces the size of a long interaction while preserving the state needed to continue. It is different from deleting the oldest messages blindly. OpenAI&#8217;s implementation returns an encrypted, opaque compaction item that carries key state into the next window.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Is context compaction the same as a larger context window?<\/h3>\n\n\n\n<p>No. A larger window lets a request contain more tokens. Compaction reduces accumulated history so a long-running task can continue within a bounded window. Systems may use both.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Did GPT-5.6 Sol become three times smarter?<\/h3>\n\n\n\n<p>No. OpenAI reported roughly three times the RHAE score on the ARC-AGI-3 public-set comparison after changing retained-reasoning and compaction settings. That result is specific to the tested configuration and tasks.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Why were output tokens lower in the better run?<\/h3>\n\n\n\n<p>With useful reasoning and state preserved, the model spent less effort reconstructing the task after each action. Compaction also prevented the active context from growing through simple accumulation. The combination produced better task progress with less repeated output in this experiment.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Can compaction lose important information?<\/h3>\n\n\n\n<p>Yes. Any compressed representation can omit details. Keep safety rules, approvals, source evidence and critical task state in durable structures, then test what survives compaction before relying on it in high-impact workflows.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Does a generic benchmark harness unfairly penalize a model?<\/h3>\n\n\n\n<p>It depends on the evaluation question. A generic harness helps standardize conditions across models. A model-aware harness can better represent production deployment. Good reporting should label the setup and avoid presenting either result as the only valid view.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">What should a marketer check first in an AI agent workflow?<\/h3>\n\n\n\n<p>Check whether the agent preserves scope, source evidence, approvals and progress across turns. Then verify tool permissions, compaction behavior, final-output validation, cost and variance across repeated runs.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Sources and editorial method<\/h2>\n\n\n\n<p>This article began with a review of Tibo&#8217;s recent posts on X, then checked the linked OpenAI publication and current OpenAI developer documentation. Benchmark definitions and scoring were checked against <a href=\"https:\/\/docs.arcprize.org\/methodology\" rel=\"nofollow noopener\" target=\"_blank\">ARC Prize&#8217;s official methodology<\/a>. Keyword demand was reviewed through Google Keyword Planner for India; \u201cagent harness\u201d showed approximately 880 monthly searches, while the more specific terms were emerging and lower volume.<\/p>\n\n\n\n<p>The article deliberately separates confirmed experiment results from interpretation. It does not treat engagement on X as evidence of technical truth, does not convert a public-set result into a hidden-set claim and does not extrapolate the reported token reduction to every product or pricing plan.<\/p>\n\n\n\n<p><em>Author note:<\/em> I write DMT&#8217;s AI coverage from a digital-marketing practitioner&#8217;s perspective: what changed, what the evidence supports and how a team should alter its workflow. Product names and benchmark results in this article are current as of July 30, 2026 and should be rechecked before procurement or production decisions.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>What OpenAI&#8217;s GPT-5.6 Sol ARC-AGI-3 experiment teaches about AI agent harnesses, retained reasoning, context compaction, cost and reliable workflows.<\/p>\n","protected":false},"author":1,"featured_media":2719,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[183,180,178],"tags":[369,319,370,299,296],"class_list":["post-2718","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-in-marketing","category-ai-news","category-artificial-intelligence","tag-ai-agent-harness","tag-ai-workflows","tag-context-compaction","tag-gpt-5-6","tag-openai","has-featured-image"],"_links":{"self":[{"href":"https:\/\/dmarketertayeeb.com\/blog\/wp-json\/wp\/v2\/posts\/2718","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/dmarketertayeeb.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dmarketertayeeb.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dmarketertayeeb.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/dmarketertayeeb.com\/blog\/wp-json\/wp\/v2\/comments?post=2718"}],"version-history":[{"count":1,"href":"https:\/\/dmarketertayeeb.com\/blog\/wp-json\/wp\/v2\/posts\/2718\/revisions"}],"predecessor-version":[{"id":2720,"href":"https:\/\/dmarketertayeeb.com\/blog\/wp-json\/wp\/v2\/posts\/2718\/revisions\/2720"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/dmarketertayeeb.com\/blog\/wp-json\/wp\/v2\/media\/2719"}],"wp:attachment":[{"href":"https:\/\/dmarketertayeeb.com\/blog\/wp-json\/wp\/v2\/media?parent=2718"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dmarketertayeeb.com\/blog\/wp-json\/wp\/v2\/categories?post=2718"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dmarketertayeeb.com\/blog\/wp-json\/wp\/v2\/tags?post=2718"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}