Carl Ross, Member of Technical Staff at OpenAI, joins us in the studio at Agent Conf 2026 in Warsaw to discuss what happens when teams experiment with AI coding agents without a token budget.
We talk about why capable models have not yet delivered the engineering acceleration Carl expects, what large agent fleets could work on, and how groups of agents can continuously improve code quality. Carl also explains the difference between briefly launching a million agents and getting useful work out of them.
What removing the token budget lets teams explore
Carl describes an experiment that used OpenAI's access to models and inference capacity to explore software engineering workflows ahead of broader availability. He explains why the team wanted to test what might become practical as models improve and token costs fall.
That access let the team set aside cost-effectiveness while investigating scale. Carl acknowledges that other teams still face those constraints, and that continuous execution at very high agent counts remains difficult to make economical.
A million agents and the question of useful work
Carl says the team briefly spun up around a million agents, but that those agents were not doing useful work at that scale. The operating scale for productive work was substantially lower.
The experiment grew out of a question about engineering productivity: if models are so capable, why are teams not seeing faster delivery of high-quality features? Running more agents was one hypothesis the team explored, alongside the frameworks and development practices needed to coordinate them.
Large agent fleets for technical debt
We ask what kind of task could justify thousands of agents. Carl points to technical debt across OpenAI's product suite and gives rewriting the stack and removing production errors as examples of broad objectives.
To make that work manageable, the team divided the problem into specific areas. Groups could focus on code quality, optimization, or packaging and bundling, much as engineering teams are organized around particular responsibilities. Carl says alignment at scale remains an unsolved problem.
Competing candidates and continuous code cleanup
Why assign many agents to code quality, you may ask? Carl explains that agents can propose competing changes, with a grader selecting the best candidate. Much of the proposed work is deliberately discarded. Agents can then inspect the selected result and propose further improvements.
He also describes keeping a subgroup focused on code quality over time. As feature development adds complexity, agents act as a continuous cleanup crew. Carl compares this with the familiar process of shipping features, identifying common patterns later, and consolidating them into shared frameworks and infrastructure.
How long will agent harnesses remain useful?
We ask whether increasingly capable models could eventually handle their own infrastructure and coordination. Carl sees today's tooling and agent harnesses as support for capabilities models do not yet have.
He also explains why the team asks whether an approach will be useful six months from now, while predictions further into the future become much less certain.
About the guest
Carl Ross is a Member of Technical Staff at OpenAI. At Agent Conf 2026, he presented "No Token Budget: What Breaks When Agentic Software Factories Run on Kubernetes." This studio interview follows up on agent scale, engineering productivity, and continuous code quality work.
Resources
