Warp got its cost per PR from about $80 to about $20 or $15, Zach Lloyd said on Latent Space.

Warp’s own page ties the first $50 to one model switch, from a base Lloyd dates to the factory’s start.

Before weighing Lloyd’s floor of five to ten engineers, work out your own cost per merged PR.

A few times below open the interview at that second.

Warp went from about $80 per pull request to about $20 or $15, Zach Lloyd said on Latent Space 24:29. He said throughput rose 40% over the last month. The revert rate, the share of changes later undone, “hasn’t gone up”.

If you run engineering, that reads as a case for a software factory. That’s one shared agent setup carrying work from request to merged code. Warp sells one, so these are a vendor’s numbers.

I think they make a strong case for measuring your agents now. They don’t yet show that you need a factory to do it.

Real Alpha cut three moments, about 12 minutes of a 34:48 episode. Anything marked as left out below applies to those minutes only.

Lloyd dates the $80 to Warp’s first factory setup, not before it

Lloyd was specific about the base. It cost about $80 per PR “when we first started using the factory approach,” he said 23:57.

He named the waste himself. Warp used Opus “where we shouldn’t have” and ran a high-powered model on triage. It also started too many sub agents, helpers that a main agent spawns.

So this is a fall from a costly start. Going from $80 to $20 is 75% ($60 ÷ $80), and $80 to $15 is about 81% ($65 ÷ $80). Neither sum says what a PR cost before the factory.

That earlier number is the one a founder needs. “Once work starts flowing through the factory infra, you’ll have a baseline,” Warp’s rollout post says. So the factory’s own early days set the start of its chart.

The dates clash too. Warp’s Oct. 15 post puts the fall from $80 to $20 in the last six weeks. That window starts about Sept. 3, yet its Sept. 2 page already gave $30.

I read the fall as waste removed, not as a saving against engineers running agents on their own machines. A before figure from those months would change my reading. Lloyd gave none in these minutes.

Warp’s own page ties most of the drop to one model switch

Warp gave a narrower account on Sept. 2. Its WarpBench page reports “a drop from about $80 to $30 per completed PR.” That followed a change to the default model that writes code.

That old default was Warp’s own product. It had used its “auto (genius)” router, which “for complex tasks routes most frequently to Opus 5.” The new default was Grok 4.6 (high).

The share is easy to work out. That’s $50 of a total fall of $60 to $65. It comes to 77% to 83% ($50 ÷ $65, $50 ÷ $60).

The page then expected “about another 25%” improvement from moving to GPT 5.6 Sol. If that means cost, it would be $22.50 ($30 × 0.75). On that forecast, two model changes nearly reach October’s $20.

Lloyd’s wider list of fixes came with no split. A split crediting sub agents over models would change my reading.

The test behind the first switch, on simple UI and backend tasks, showed “the same quality, but half the cost.” The bill fell further, by 62.5% ($50 ÷ $80).

The old router sent complex work most often to Opus 5. I think a test on simple tasks may not see that, which could explain the gap.

Warp also says it “might have guessed that Grok 4.6 and Sol would be more cost-effective than Opus 5.” I think guessing a cheaper default needs no platform. Knowing it did no harm is harder.

Measuring is what answers that. Warp says merge rate “stayed constant” and a task score rose from 69% to 87% on that page. The score comes from Warp’s own AI judge.

Others on the same show found the same lever from another side. Tenex helps large companies adopt AI, so it’s a seller too. In October, its Alex Lieberman said “recently our costs have been driven just way down from Opus 5.5”.

So Warp saved by moving off Opus 5, and Tenex by moving onto Opus 5.5 from pricier models. Models age fast, as Lloyd said. I think part of this saving was open to any team willing to switch.

Warp has given the 40% as both a one-month and three-month gain

Set Warp’s statements side by side and the 40% keeps changing its window. On Oct. 15 it was “40% higher PR throughput than in the last month” in the rollout post. Five days later, Warp on X said “40% more PRs merged” three months after adopting factories.

In the episode posted Nov. 3, Lloyd said throughput was up 40% “over the last month” 24:35. A one-month gain and a three-month gain can both be 40% only if the other two months were flat. That assumes they measure the same thing, which none of them says.

Reverts moved too, from “a lower revert rate” on Oct. 15 to “hasn’t gone up” in the episode. If the Oct. 20 base was $80, 67% lower is about $26.40 ($80 × 0.33). That’s above the $20 given five days earlier.

Warp’s blog list still titles its Sept. 2 post “How we reduced our cost-per-PR 75%.” The figures in it work out to 62.5% ($50 ÷ $80), or about 72% with its forecast ($57.50 ÷ $80).

Spoken numbers round and posts get edited, but a budget needs a figure with a fixed base.

None of these statements gives a PR count. A How I AI episode from Sept. 21 is titled “How Warp ships 2,000 PRs a month.”

Warp’s pages cited here don’t say how many of those PRs the cost figure covers. Its rollout post tells teams to track “percent of merged PRs that went through the factory.” Ask Warp for that share.

Google offered a fuller pair in October. Sundar Pichai cited a 30% rise in code changes and a fall of one-third in rollbacks over four weeks. That came with no method, from a seller too, but it put a number on the quality side.

Cost per PR leaves out PR size and the people reviewing

A PR isn’t a fixed unit. Flo Crivello of Lindy, which also sells agents, said weekly PRs and lines per PR both tripled in three months. That’s nine times the lines a week (3 × 3), if both held.

So cost per PR can rise with no waste when PRs grow. It can fall with no gain when PRs shrink or chores join the count.

Warp’s factory also generates its weekly changelog and updates its public docs. Nothing cited here says if those count.

Reverts need the same care. If 40% more PRs come with a flat revert rate, the count of reverts also rises 40%. The rate can hold while the cleanup grows.

Then there’s what the $20 includes. Warp’s price list says “Factory usage is billed separately based on inference, compute, and agent usage,” and lists seats apart. Lloyd didn’t say in these minutes which of those his figure counts 24:29.

Review time sits outside all of them. Lloyd’s How I AI episode lists “Why human review is still the bottleneck.” Uber’s Uday Kiran Medisetty said “I think reviews become a big bottleneck same as like any other company”.

At Uber, “we still require an engineer to review every PR,” he said. Lindy sits at the other end, where Crivello said “we hardly review prs anymore”.

Where people do review, my guess is that one engineer-hour costs more than the whole $20. Check that against your payroll, because if it does, review is the bigger cost per PR.

On Lloyd’s chart, two points of correctness cost three times as much

Lloyd showed one chart worth copying. On one Warp task set, GPT 5.6 Sol was “like 95% correct” and cost “six bucks on average” 17:34. GLM 5.3 Flash was 93% and cost “two bucks”.

Warp’s AI judges grade these runs, its page says. So 95% may be an average grade, not a count of right answers.

If correct means pass or fail, the sum is simple. Per 100 runs at $4 more each, Sol costs $600 against $200 and gets five wrong against seven. Two fewer wrong runs would cost $400, or about $200 each.

Rounding alone moves that line between $133 and $400 ($400 ÷ 3, $400 ÷ 1). The 95% and 93% could sit one to three points apart. So treat it as rough.

It still gives you a rule the chart doesn’t state. If a wrong run costs you well under that to catch and redo, the cheaper model wins on these figures. Well above it, the pricier one pays for itself.

OpenRouter’s Alex Atallah, whose company sells access to many models, uses the same lens. What matters is “how much it costs to complete the task, not price per token,” he said. He also rated GLM 5.3 flash highly on cost per task.

Keep the limits in view. Lloyd said these models are “now slightly older” and that the chart covers “a certain set of things”. He didn’t define correct in these minutes.

Five to ten engineers is a seller’s read, so check your bill

Lloyd’s floor carries two maybes. He said “maybe around like five to ten engineers would maybe be the floor where this makes sense” 27:39. For a few people he called a factory “overkill” and pointed to a Claude Code or Codex subscription.

He said some fast-growing startups of maybe 10 to 100 engineers are “pretty self-serve” on tools like Slack. Larger companies need more implementation work, he said. That’s what a seller sees “when we talk to companies,” and these minutes give no sum behind it.

Here’s a sum. Warp’s factory puts you on a meter, since its price list says “Execution is usage-based.” Its Business seat is $50 per user per month.

If Warp’s figures were all usage, 100 PRs would cost $8,000, $2,000 or $1,500 (100 × each). Ten seats are $500 a month.

If you pay flat subscriptions now, set them beside those lines. The gap is what a factory must earn back in output. I read headcount as a rough stand-in for that gap.

Heavy users can reach payroll scale. Tenex’s Lieberman said “I actually don’t have a number off the top of my head”. His estimate of raw tokens against employee spend was “somewhere between 1 to 1 and maybe like 2 to 1”.

At that level, I think tuning pays at almost any headcount.

Output is the stronger argument. Cognition’s Scott Wu, another agent seller, called the spend “very clearly worth it” if engineers ship three times more.

On a team of 10, a real 40% gain is like four more engineers (10 × 0.4). That holds only if PRs keep their size. So I’d judge a factory on that number, and it’s the one with the weakest support above.

Start with your own cost per merged PR and one model test

This week, if your agents bill by usage, divide last month’s agent spend by last month’s merged PRs. That’s your base, and Warp’s $80 can’t stand in for it. Put median lines per PR beside it, given Lindy’s tripling.

Then test one default on past work. Warp says WarpBench used 30 tasks and took “less than 30 minutes of setup work” on its platform.

That run cost $2,130.57. Warp’s $50 came from an earlier test the page doesn’t price.

The costed run reads as the one behind “about another 25%.” If that means cost, it’s $7.50 a PR, so payback takes about 284 PRs ($2,130.57 ÷ $7.50). Your numbers will differ.

Warp says building the replay yourself would take “months of engineering work,” which is a seller’s estimate. A hand test of a few past tasks needs much less, though it proves less.

Track what the $20 leaves out. Count reverts as well as the rate, and count human interactions per PR, a measure Lloyd’s episode lists. Track the share of PRs your agents carry, which Uber puts at about 15% for its background agent.

If you’re a few engineers on subscriptions, stay there, as Lloyd himself said 27:19. If you’re more, try one low-risk part of your system first. Warp’s rollout post says to start with low-risk chores too.

Sam Newman, who writes on software architecture, said “we don’t have to make a one-size-fits-all approach”. Of factories run with no one reading the code, he added, “Not everyone should be doing it whatsoever”.

Lloyd allows for either path, “whether they actually use warp factories or they build their own”. You don’t need to settle that this quarter. If your own number is small beside payroll, the question can wait.

Lloyd says replay checks are next, and Warp was exploring A/B tests

The self-improving loop is younger than it sounds. Lloyd said Warp’s agents can propose factory changes by pull request, and cited 54 failure cases in one triage step. Replaying runs to check such a change is next, since “we don’t have this yet” 15:37.

The WarpBench page had a similar limit on Sept. 2. Its benchmarks “only support retroactive measurements of quality,” so you assume future tasks look like past ones. A/B testing was still something Warp was exploring.

Warp holds a live session on Nov. 10 about the code review bottleneck. Watch there for the PR count behind the cost, a before figure and any count of human time.

Five questions for whoever pitches you a factory

1. What a PR cost before your factory, and who measured it

2. What your cost per PR includes, from compute and seats to human review

3. How many PRs sit behind the average, and whether chores count

4. How PR size has moved over the same dates

5. How many reverts happened, as a count as well as a rate

Forward this to whoever approves your agent bill.

Play the 3 minutes where Lloyd gives his cost per PR.

Where it comes from

The moments cut from this episode

• Agents can improve the factory that runs them, from 12:59

• Factory tuning cut Warp’s cost per PR by about 75%, from 23:51

• Factories start to make sense around five to ten engineers, from 26:54

Other people’s moments

• Lindy uses its AI teammate to optimize its own CI, Flo Crivello on The Cognitive Revolution, Aug. 10, 2026

• Tenex’s token spend is roughly one to two times employee spend, Alex Lieberman and CJ Hess on Latent Space, Oct. 21, 2026

• Real usage and task costs are better signals than model branding, Alex Atallah on Sabrina Halper Show, Sept. 29, 2026

• Uber still requires human review on every PR and Minion handles about 15% of Uber’s production PRs, Uday Kiran Medisetty on The Pragmatic Engineer, Oct. 14, 2026

• Measure AI spend by output, not token counts, Scott Wu on Founders, June 28, 2026

• Run an AI software factory on one module, not the whole system, Sam Newman on The Pragmatic Engineer, Oct. 7, 2026

• Sundar Pichai on Big Technology Podcast, Oct. 20, 2026

The pages outside the episode

• WarpBench, Warp, Sept. 2, 2026

• Tactics for rolling out software factories, Warp, Oct. 15, 2026

• Warp’s post on X, Oct. 20, 2026

• Warp’s price list, read Nov. 4, 2026

• How Warp ships 2,000 PRs a month with AI factories, How I AI, Sept. 21, 2026

Real Alpha sends the best minutes from the week’s long AI interviews, twice a week, free. Get it here.