Topics›Autonomous Agents with Reliable Outcomes›How ChatGPT Optimizes Its Agent Loop: Persistent WebSockets, Deferred Tool Discovery, Code Mode
★Very HighAct: MediumFull content
How ChatGPT Optimizes Its Agent Loop: Persistent WebSockets, Deferred Tool Discovery, Code Mode
ByteByteGo·July 29, 2026
New
Summary
Deep technical walkthrough of OpenAI's harness/API/inference optimization stack for Codex and ChatGPT Work at ~10M users: persistent WebSockets replacing per-call HTTPS, stable prompt-prefix caching, deferred tool discovery via BM25 search, and 'Code Mode' where the model writes a program instead of one tool call at a time.
Why this signal
Technical depth on agent infrastructure reliability and cost at real production scale — directly useful if Thomas Engineering builds or evaluates agent harness infrastructure; the 'agent optimizes its own stack' pattern (Codex wrote its own serving-API migration) is a notable end-to-end dev data point too.
Original email
ByteByteGo · 7/29/2026
How ChatGPT Optimizes its Agent Loop: Harness, API, and Inference