Back to your brief

OpenAI “deep research” vs Real Deep Research

The same four questions, asked of both. What came back is below, side by side — open either one and read it.

AI ENGINEERING

How to build an AI agent that actually works

OpenAI “deep research” gives you a long, wordy, superficial map. Real Deep Research challenges your hypotheses and tells you the two things to build, the four to skip, and why.

OpenAI “deep research”· 1 pageOpen

State of the Art in Agent Harness Design: What Actually Moves Task Success

Executive summary

The strongest evidence from 2023–2026 is that the harness is not merely packaging around the model. Several non-model choices produce double-digit changes in task success with the model held fixed: tool-interface design, recovery semantics, context preservation, tool routing, and the match between orchestration architecture and task structure. Conversely, some choices that receive substantial engineering attention—framework DSL, protocol envelope, dashboard UI, or whether a logically sequential loop is implemented with synchronous versus asynchronous application code—have little or no controlled evidence of improving task success by themselves.

The most important finding is that architecture should follow the dependency structure of the task. In a 2026 controlled study of 260 configurations across six benchmarks, centralized multi-agent orchestration improved Finance-Agent from 0.349 to 0.631, a +80.8% relative gain, yet reduced PlanCraft from 0.568 to 0.282, a −50.3% relative loss; an independent-agent architecture was even worse on PlanCraft at −70.0%. Agent count itself had no statistically significant direct effect after controlling for other variables, while tool diversity and coordination-task interactions did. [1][2] Anthropic reports a compatible but less controlled production result: its breadth-first research system with an orchestrator and parallel subagents beat a single-agent configuration by 90.2% on an internal research evaluation, at the cost of roughly 15× chat token consumption. [3]

Tool-interface design has some of the cleanest causal evidence. With GPT-4 Turbo fixed on SWE-bench Lite, SWE-agent's purpose-built agent-computer interface resolved 18.0% of tasks versus 11.0% for the shell-only agent. Within the interface, summarized search scored 18.0% versus 12.0% for an inefficient iterative search; a linting edit guardrail scored 18.0% versus 15.0% without linting; and a 100-line file window scored 18.0% versus 14.3% for 30 lines and 12.7% for whole-file viewing. 4(https://github.com/SWE-agent/SWE-agent)5(https://github.com/SWE-agent/SWE-agent) CodeAct independently found that changing the action representation from JSON/text to executable code raised GPT-4-1106-preview success on a multi-tool benchmark from 53.7%/52.4% to 74.4%, while reducing average interaction turns from about 7.6–7.7 to 5.5. [6]7(https://github.com/xingyaoww/code-act) These results say that **affordances, feedback, o

Real Deep Research· 26 pagesOpen

Agent Harness Effectiveness

What we found

Scope: this holds for software-engineering and stateful agent tasks under a fixed model. On frontier models, competent harnesses plateau on correctness and differ only on cost; on weak models the cross-harness spread opens up.

The five ideas most worth stealing:

  1. The interface is the lever, the loop is not. SWE-agent's central ablation shows the agent-computer interface (ACI) moves SWE-bench Lite from 2.67% (RAG) to 18.00% under the same GPT-4 Turbo, while removing the autonomous loop entirely (Agentless) gains — 32.00% vs the best open-source agent's 30.67%, at $0.70/issue. The harness's value is "actions the agent can execute and feedback it can read," not "the agent gets to act autonomously." Measured, not asserted.papers.nips.ccarxiv.org

  2. Watch the run and intervene — that's a measured ~10-point lever. Online process-monitoring (trajectory graph + automatic intervention) raises issue-resolution 6.9–23.5% across models at <10ms overhead, and a cleaner rule-based follow-up (LivePlan) independently reproduces +9.9pp avg / +15.2pp max. Telling the agent what to do (plan content) is noise; watching what it does and stepping in is measured-positive on the pathological tail. But note the denominator is ~10% "lucky passes," so part of the gain is converting luck into verified success, not new skill.arxiv.orgarxiv.orgmicrosoft.com

  3. Components don't stack — the "all-in" harness loses. A full 2⁵ factorial (32,000+ evals) finds the agent bundling every scaffold component underperforms the best subset in every setting: Tool Use alone beats All-In by 32% on HotpotQA, and 56.3% of triples violate submodularity with 90% of violations being sign-flips (adding a component makes it worse). Tool Use carries ~70% of the value; Planning is significantly negative. Do not "add until marginal turns negative" — that greedy rule is empiri

AI ENGINEERING

How to run a whole team of AI agents at once

OpenAI “deep research” hands you a tidy plan and never asks whether it works. Real Deep Research measures every gate, tells you which one fails, and why no gate can catch prose that passes every check and is still wrong.

OpenAI “deep research”· 1 pageOpen

GitHub as an Operating System for Non-Code Agent Teams

Executive summary

The GitHub model transfers surprisingly well to non-code agent work, but only after one important correction: “merge” and “execution” must be treated as separate operations. In software, merging source code usually changes a canonical repository but does not necessarily deploy it. The same separation is essential for writing, design, labeling, decisions, business operations, and especially robotics. A merged change should mean “this is now the approved canonical intent”; applying that intent to customers, databases, money, machinery, or the physical world should be a second, independently authorized transition. Git itself already embodies the useful foundations: a staging area prepares the contents of a future commit, commits create identifiable snapshots, pull requests support discussion and approval, protected branches impose governance, and merge queues re-test changes in the context in which they will actually land. 1(https://git-scm.com/docs/git-add)2(https://docs.github.com/en/pull-requests/reference/pull-request-reviews)3(https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/configuring-pull-request-merges/managing-a-merge-queue)

The most useful generalized pipeline is therefore:

private workspace → staged change set → immutable hand-in → review proposal → automated checks → approval → integration queue → canonical state → execution gate → observed outcome → compensating change if necessary.

This model is strongest when work is decomposable into small, versionable artifacts and correctness can be partially expressed as executable invariants. It becomes weaker as work becomes subjective, globally coupled, or irreversible. Writing is highly versionable but semantically subjective; design has mergeable structure but visual/system-level interactions; labeling benefits from redundancy and statistical QA; organizational decisions are versionable but cannot literally be “unmade”; business transactions need compensating actions; robotics adds resource conflicts, real-time state drift, and irreversible physical consequences. Manubot demonstrates that ordinary GitHub review can govern collaborative manuscripts; Figma implements a native branch/review/merge model for designs; Labelbox separates labeling, review, approval, and rework; lakeFS provides Git-like branches and commits for data; Temporal implements durable human approval workflows; and Open-RMF maintains a changing traffic schedule and negotiates conflicts amo

Real Deep Research· 26 pagesOpen

Software Work Beyond Code

What we found

Scope: the mechanism is documented and real; the outcomes (latency, escape rate, merge frequency) for non-code artifacts are almost entirely unmeasured, and the one thing that is measured — post-approval error — shows the human gate fails at a material rate precisely because its target (intent) has no oracle.

The five ideas most worth stealing:

  1. A non-code PR runs a real state machine, not a metaphor. ASAM's automotive-standards workflow defines a 9-state machine (New → Classified → Accepted → … → Ready for Merge → Closed) with a named responsible role per state, and a Ready for Merge state that only exists if the artifact is genuinely managed as code. MCP SEP-1850 runs Draft → In-Review → Accepted → Final with an 8-Yes/0-No maintainer vote. The PR is the "safety boundary" past which an unapproved change cannot reach the live artifact. This is documented process, not measured adoption — no source reports transition timing or rejection rates.asam-ev.github.ioraw.githubusercontent.com

  2. Non-code CI is real, named, and sometimes blocking — but "implemented" ≠ "measured to block." Four production repos run documented non-code checks (OpenAPI Spec's markdownlint+linkspector with fail_level: any; microsoft/edge-ai's frontmatter check that exit 1s while its link check is continue-on-error: true; rungalileo's six-job docs CI; codeforamerica's four-stage contract pipeline). The minimal/starter set converges on build + link check, not the hypothesized link+spell+lint. But no source has ever counted how often a non-code check actually blocks a merge — the code-CI side is measured (258,300 GHA runs), the non-code side is not.deepwiki.comgithub.comdocslikecode.com

  3. **The human gate is domain clarification,

INVESTING

Whether humanoid robots are actually shipping

OpenAI “deep research” tells you the boom is smaller than it looks. Real Deep Research traces the numbers everyone repeats, finds no source, and shows the whole boom is one vendor renting robots by the hour.

OpenAI “deep research”· 1 pageOpen

Commercial Humanoid Robots: Field Deployment, Unit Economics, and Technical Reality, 2022–2026

Executive summary

As of August 31, 2026, the public evidence does not support the often-implied narrative that thousands or tens of thousands of humanoid robots are already doing economically productive work in factories and warehouses. Roughly 20,000 humanoids were shipped globally in 2025, according to BofA data reported by Reuters, and individual Chinese manufacturers report thousands of sales, but shipments overwhelmingly include research systems, government procurement, training/data-collection fleets, demonstrations, entertainment, and industrial pilots. Reuters' examination of hundreds of filings and procurement records concluded that widespread commercial demand remains limited and that many supposed industrial deployments are still validation or training programs. [1][2]

My strict deployment ledger counts a robot only when there is evidence that it is physically operating in a normal commercial workflow and generating economic output, rather than proving a concept, collecting training data, or demonstrating technology.

Under that definition:

For bipedal humanoids, the public record establishes only a single OEM—Agility Robotics—with clearly documented, ongoing commercial customer operations at meaningful scale. Agility reports eight commercial deployments, three additional booked deployments, and current commercial deployments involving GXO, Schaeffler, Toyota Motor Manufacturing Canada, Amazon, and Mercado Libre. The company does not disclose how many Digit robots those deployments contain. A conservative inferential floor is therefore at least five simultaneously deployed Digit units, one per named commercial customer organization, but the real number is almost certainly higher. The SEC materials should not be read as saying “eight robots”: they say eight deployments, not eight units. [3][4]

If wheeled humanoids/mobile manipulators are included, Galbot materially changes the answer. A Chinese government source reported in May 2026 that Galbot humanoids were operating inside more than 120 retail stores and in smart pharmacies across dozens of cities, performing product retrieval, medicine picking, order processing, packaging, and customer hand-off around the clock. By August 31, Galbot said its mainland network had expanded to about 200 outlets in 50 cities. Because the official May wording establishes robots inside more than 120 stores, **>120 physical Galbot units is a defensible lower b

Real Deep Research· 26 pagesOpen

Humanoid Robot Paid Work

What we found

The scope that makes this true: "paid work" means ongoing paid operation beyond trials/pilots, counted at the research cutoff, with a named employer and a disclosed payment relationship. Under that bar, the field collapses to one vendor.

The ideas worth stealing

  1. Paid humanoid labor is a monoculture — every documented paying employer is an Agility RaaS customer. Agility's own SEC-filed merger press release names the commercially-deployed set: "Schaeffler, GXO, Toyota Motor Manufacturing Canada, and Mercado Libre," with Amazon added by the investor deck's CEO letter. Everyone else — Figure, UBTECH, AgiBot, Tesla — sells or ships robots but discloses no paying customer with verifiable terms. The mechanism in every case is RaaS (rental by the hour), not purchase, because no operations director will own the reliability risk of a six-figure robot. If you are modeling the market, model a rental market, not a hardware market.sec.gov

  2. The largest named currently-working unit count in the entire field is "7+" — Toyota's Woodstock plant. Toyota Motor Manufacturing Canada runs "seven or more Digit units" under RaaS for RAV4 logistics. That single number is the ceiling of what anyone has disclosed as a currently-working fleet; every other named deployment is either "undisclosed" or a journalistic singular. If a vendor or analyst tells you "N robots doing paid work now," the honest ceiling is single digits.registry.deploy.report

  3. The "~1,200 worldwide doing productive paid work" headline is an un-sourced analyst estimate that is internally inconsistent by ~2x. Robonaissance's "Generously counted, perhaps 1,200" carries no citation, and its own scorecard tags only two vendors "Productive" — Agility (~100 sold) and UBTECH (500+ delivered) — which sums to ~600, not 1,200. The article's own subtitle says "fewer than 200 outside China," a third number it never reconciles. The only anchorable figure is "~100 Western," and that is itself a cumulative-sold claim, not a curr

LAW AND COMPLIANCE

What the EU AI Act actually makes you do

OpenAI “deep research” gives you a long reference of what the law says. Real Deep Research tests what you already believe against it, finds six of your assumptions wrong, and names the one question that decides everything.

OpenAI “deep research”· 1 pageOpen

EU AI Act: Real Obligations for a Commercial Software Company Selling into the EU

Research date: 31 August 2026. The operative statute is Regulation (EU) 2024/1689 (Artificial Intelligence Act), not “Regulation (EU) 2021/XXXX.” The “2021” reference comes from the Commission’s original legislative proposal. The AI Act was adopted on 13 June 2024, published on 12 July 2024, and entered into force on 1 August 2024. It has since been materially amended by Regulation (EU) 2026/1744, the Digital Omnibus on AI, adopted on 8 July 2026, published on 24 July 2026, and in force from 27 July 2026. This report therefore uses the EUR-Lex consolidated text dated 27 July 2026, read together with the authentic amending act. [1][2][3]

Executive summary

For a normal commercial software company, the AI Act is not a single certification requirement applying to every AI feature. The legal outcome depends first on what the company is doing in relation to each feature: provider, deployer, importer, distributor, product manufacturer, downstream integrator, or provider of a general-purpose AI model. It then depends on the system's risk classification and functionality. A SaaS company that builds an AI feature and sells it under its own name will ordinarily be the provider of that AI system; the same company is separately a deployer when it uses AI internally, for example for hiring, employee management, customer support, fraud decisions, or sales. [4][5]

The most important practical conclusions as of 31 August 2026 are:

Takeaway What it means for a software vendor now
Some obligations are already live for essentially all relevant providers/deployers. AI-literacy measures under Article 4 have applied since 2 February 2025; most Article 5 prohibited practices have applied since the same date; and Article 50 transparency rules became applicable on 2 August 2026. [6][7][8]
The major high-risk compliance package is not yet generally due. Following the July 2026 Omnibus, Chapter III Sections 1–3 apply to Annex III high-risk systems from 2 December 2027, and to Article 6(1)/Annex I product-related systems from 2 August 2028. The Commission expressly attributed the postponement in part to delayed standards. [7][9][10]
Generative-AI transparency is a live obligation now. Providers of systems generating synthetic audio, images, video or text generally need machine-readable/detectable marking under Article 50(2); providers of conversational systems have a disclosure obligation under Article 50(1); depl
Real Deep Research· 26 pagesOpen

EU AI Act Duties

What we found

Scope: this is the answer for any organisation, because the one fact that would let us close it to a verdict — what the commissioning organisation actually does, where it is established, and whether its use is professional — is not recorded anywhere in this pack. The decision tools are complete; the verdict is blocked on those three missing facts.

The ideas most worth stealing

  1. Classify before you comply — and read Article 5 first, then Article 6/Annex III, then Article 50. The Act is a four-level risk ladder (unacceptable / high / transparency / minimal), and the category decides which duties exist at all. The correct order is: is it banned (Art 5)? is it high-risk (Art 6 + Annexes I/III)? is it a transparency case (Art 50)? Everything else is unregulated. The Commission's own summary states the four levels.digital-strategy.ec.europa.eu

  2. There is no single "2026" date — the expensive duties run to 2 Dec 2027 (Annex III) and 2 Aug 2028 (Annex I). A duty map keyed to one date is wrong by up to two years for the highest-cost obligations. The Digital Omnibus (Reg 2026/1744) deferred the high-risk tranches (Annex III 2 Aug 2026 → 2 Dec 2027; Annex I 2 Aug 2027 → 2 Aug 2028) while leaving the cheap-and-immediate duties (Article 5 prohibitions, Article 50 transparency from 2 Aug 2026) on the near-term schedule — and adding two new prohibitions from 2 Dec 2026. Public-authority high-risk systems are deferred further, to 2 Aug 2030.ai-act-service-desk.ec.europa.euartificialintelligenceact.eu

  3. A deployer's Article 26 duties are non-transferable by contract — buying a high-risk system does not discharge them. Monitoring, human oversight, six-month log retention, worker notice, incident reporting, registration (for public authorities) sit with the user, independent of the vendor's Article 16 conformity load. A deployer flips into a provider (taking the full Article 16 burden) under Article 25 if it re-brands, substantially modifies, or changes the inte