The short version
Warp cut its cost per PR from about $80 to about $20 or $15, Zach Lloyd said on Latent Space.
Warp credits one change of default model with about $50 of that, and outside research suggests where else to look.
Work out your own cost per merged PR, then try the six fixes below.
A few times below open the interview at that second.
Warp went from about $80 per pull request to about $20 or $15, Zach Lloyd said on Latent Space 23:57. He said the revert rate, the share of changes later undone, “hasn’t gone up”. Warp sells the software it used, so these are a vendor’s numbers.
Warp used Opus “where we shouldn’t have” and ran a high-powered model on triage. It also started too many sub agents, helpers that a main agent spawns.
I think you can spot that waste without a platform, though Lloyd credits Warp’s own tools. Below is how to find your number, then six fixes in the order I’d try them.
Ramp’s $93 request shows one session can hide most of the cost
Ramp Labs traced one bug request through Inspect, its coding agent. It became three objectives, two merged PRs, eight linked sessions, roughly 5.6 million tokens and $93. The first session held only about one-fifth of that spend.
So count the whole run. Anthropic says Claude Code averages around $13 per developer per active day across enterprise deployments.
With no exact number to hand, Tenex put token spend at one to two times employee spend. So nobody else’s number will tell you yours.
This week. Divide last month’s agent spend by last month’s merged PRs, counting every linked session.
Put lines per PR beside it, since Lindy’s tripled in three months.
1. Warp credits one default model change with its biggest drop
Warp’s Sept. 2 page reports “a drop from about $80 to $30 per completed PR” after one change. That was about a week in, after a test on simple UI and backend tasks. Its old default was its own router, which mostly picked Opus 5 for complex tasks.
Warp switched to Grok 4.6 (high), and merge rate “stayed constant”. That’s $50 of a total fall of $60 to $65.
Anthropic sells Opus and says much the same. It traces surprise bills mostly to long sessions never cleared or Opus left as the default.
Epoch AI says the cost of a given level of performance has fallen about 47% per quarter since 2023. That assumes a user always hunting for the cheapest model, and Epoch says real users switch less often.
Its main benchmarks cover math, science and games. Its coding benchmark fell about 27.5% per quarter.
I read that as a reason to move your default on purpose, since users who switch less save less.
“recently our costs have been driven just way down from Opus 5.5”
Alex Lieberman of Tenex, which helps large companies adopt AI, so it’s a seller too 15:40
So Warp saved by moving off Opus 5, and Tenex by moving onto Opus 5.5 from pricier models.
This week. Replay five to 10 past tasks on the next cheaper model and read each diff. If they hold up, switch, then watch merges and reverts.
Keep the test small. A hand test costs far less than Warp’s $2,130.57 benchmark, though it proves less.
2. Triage and small jobs can run on far cheaper models
Triage is the step that sorts incoming work, and Warp had a high-powered model on it. Anthropic lists Opus 5.5 at $4 per million input tokens and Haiku 5.5 at $0.10 on shorter prompts. That’s a gap of 40 times ($4 ÷ $0.10).
“I don’t need to hire Albert Einstein to solve every problem”
Guillermo Rauch, CEO of Vercel, which sells access to many models
OutSystems built a router that sends some jobs to lower-cost models. CEO Woodson Martin said its spending now runs below its Q3 forecast. He gave no figure and credited better software around the models too.
Cheap has a cost when it’s wrong. Lloyd’s chart had GPT 5.6 Sol at “like 95% correct” for “six bucks on average” 17:34. GLM 5.3 Flash was 93% for “two bucks”.
He called those models slightly older and didn’t say these were small jobs. If correct means pass or fail, two fewer wrong runs per 100 cost $400, or about $200 each. Compare that with what a wrong run costs you to catch and redo.
This week. List your agent steps. Move any step where a wrong answer is cheap to catch to the cheapest model that passes your test.
3. Lloyd says Warp ran too many agents, and Ramp points at goals
Lloyd gave no split of what fewer sub agents saved 24:17. Anthropic’s Claude Code docs say agent teams in plan mode use about seven times the tokens of a standard session.
Those docs also recommend sub agents for verbose work.
Ramp’s data points at goals as well. Across roughly 200,000 runs, the 17% with more than one objective took 58% of model spend. Most of those runs, 61%, stayed in one session.
Ramp doesn’t call that waste. I’d still read those runs first, since that’s where most of the money sits.
Fewer isn’t always better. Warp’s agents flagged 54 failure cases where triage could have used a sub agent. Lloyd said it can’t yet test such changes by replay.
This week. Pull your 20 costliest runs. Note the agents used and the goals pursued.
Try splitting multi-goal runs and compare spend. Clear sessions between unrelated tasks.
4. In one study, caching cut costs 41% to 80%, so check yours
Caching stores the repeating part of a prompt so you pay less to resend it. PwC researchers tested it over 500 agent sessions on four models, and put savings at 41% to 80%.
The catch is that these were research tasks rather than coding, with 10,000-token system prompts. They also warn that caching everything can slow responses.
Claude Code caches automatically, Anthropic says, so this matters most if you run your own agents.
This week. Divide cached tokens by input tokens for your main tool.
If the share is low, keep changing text out of the cached part, or ask your vendor why.
5. Agents ignore their own budgets, so set limits outside them
Ramp Labs, whose company sells controls on agent spend, showed coding agents a live token count on SWE-bench tasks. Across over 14,000 agent messages, none referenced the budget.
Forced to choose at the limit, an agent with approval as the default approved 97% of the time. Most separate AI approvers did no better than chance from the work alone. Ramp reports no dollar saving.
I think a limit per run catches runaways without rationing people.
Runaways do happen. One Anthropic PR went through more than 50 rounds of review and fixes overnight before its author stopped it.
This week. Set a hard token limit per run.
Have a person or a fixed rule approve any more.
6. Review may cost more than the tokens, so measure it
Warp’s $20 may leave people out, since Lloyd didn’t say what it counts. Suppose an engineer costs $100 an hour, a round number of mine. Then 15 minutes of review is $25 ($100 ÷ 4), more than the whole $20.
Faros AI, which sells engineering analytics, compared AI adoption across 1,255 teams in 2025. High adoption came with 98% more merged PRs and 91% longer review.
Its 2026 report, on 22,000 developers, compares each firm’s lowest and highest adoption. Median time in review rose 441.5%. That shows a link and can’t prove a cause.
Anthropic’s Claude Code team describes one way through, on its own product. From June to September, merged PRs more than doubled while the share reverted fell by more than 40%.
After changes including a cap of three review rounds, mean reviews to pass fell from 2.08 to 1.32. Low-risk changes from trusted authors need no person there. Uber keeps an engineer on every PR, but moved simple checks to before it opens 56:25.
This week. Measure human minutes per merged PR.
Run lint and tests before a PR opens, and sort changes by risk.
After three AI review rounds, have the author rethink the approach.
Warp’s numbers lack a fixed base, so test your own
The start. The $80 dates to “when we first started using the factory approach”. Nothing here says what a PR cost before that.
The window. Warp gave its 40% PR throughput rise as one month on Oct. 15, then three months on X. In the episode posted Nov. 3, Lloyd said “over the last month”.
The cut. That X post says 67% lower cost per PR, but $80 to $20 is 75% ($60 ÷ $80).
For a few people, Lloyd called a factory “overkill” and pointed to a Claude Code or Codex subscription 27:12. His floor is maybe five to ten engineers, a seller’s read.
Write your answer as one line for your team.
We spent $X on agents last month and merged Y PRs at $Z each. That came with L lines per PR, R reverts and H human minutes per PR.
Send this to whoever signs off your agent bill.
Play the 3 minutes where Lloyd names the waste.
Where it comes from
The moments cut from this episode
Agents can improve the factory that runs them, from 12:59
Factory tuning cut Warp’s cost per PR by about 75%, from 23:51
Factories start to make sense around five to ten engineers, from 26:54
Other people’s moments
Most enterprise workloads do not need frontier models, Woodson Martin on The Cognitive Revolution, Oct. 7, 2026
Open models now handle most token traffic on Vercel, Guillermo Rauch on Sourcery, Oct. 16, 2026
Tenex’s token spend is roughly one to two times employee spend, Alex Lieberman and CJ Hess on Latent Space, Oct. 21, 2026
Lindy uses its AI teammate to optimize its own CI, Flo Crivello on The Cognitive Revolution, Aug. 10, 2026
Uber still requires human review on every PR, Uday Kiran Medisetty on The Pragmatic Engineer, Oct. 14, 2026
The pages outside the episode
WarpBench, Tactics for rolling out software factories and Warp’s post on X, Warp, Sept. to Oct. 2026
Building the missing layer between AI spend and ROI, René Sultan, Ramp Labs, Sept. 1, 2026
Coding agents ignore their own budgets, Shaiyon Hariri, Ramp Labs, April 21, 2026
The plunging price of thought, Epoch AI, Sept. 22, 2026
Don’t Break the Cache, Lumer and others, PwC, revised Jan. 31, 2026
The AI Productivity Paradox, Faros AI, July 23, 2025, and The AI Engineering Report 2026, Faros AI, April 12, 2026
How the Claude Code team does code review, Anthropic, Oct. 15, 2026, with its cost docs and price list, read Nov. 4, 2026
Real Alpha sends the best minutes from the week’s long AI interviews, twice a week, free. Get it here.