Product-led Growth (PLG) to Agent-led Growth (ALG).

Developer Experience (DX) to Agent Experience (AX).

These shifts have been under way for some time, but we’re clearly accelerating.

Proxies of agent traffic across infrastructure and security software earnings calls underlines just how much.

Infrastructure Software

DigitalOcean

AI customer ARR reached $234 million, growing over 200% year-over-year… 85% of AI customer ARR… came from inference services and core cloud, not from bare metal. Inference services… now represent over 70% of our total AI customer ARR.

…over 6,000 customers have leveraged the inference engine, while customer count grew an average of close to 60% month-over-month, and the token volume increased 30x over the last 60 days.

This token growth is driven by strong demand from AI natives… As workloads shifted from human prompted to agent-driven, token consumption and cost exploded.

Datadog

As of Q2, over 750 AI customers use Datadog… all 10 of the top 10 AI leaders are Datadog customers… We are also seeing signs of rapid growth in agentic activity with a number of MCP tool calls quadrupling again quarter-over-quarter and growing more than 22x when compared to Q4 2025.

The second thing we see is a very rapid increase in the usage of all of our AI-first surfaces. So that would be the products that measure agents and LLMs, we see an explosion of traffic in terms of the LLM and tool calls we’re getting.

That would be the amount of calls we’re getting to our MCP endpoints. So we see that explode completely over the past 2 quarters.

Dynatrace

Our unified architecture becomes more valuable as AI increases complexity, and that growing value is reflected in higher consumption, broader platform adoption and the following 3 new monetization opportunities. First, AI workloads are similar to core observability workloads in that they leverage the same types of data such as logs, traces and metrics. But AI workloads generate dramatically more telemetry than the systems that came before them.

This is one of the reasons why log management remains our fastest-growing product category, with consumption nearly doubling since surpassing the $100 million milestone just 2 quarters ago

Every time a customer uses Dynatrace Intelligence to get answers through AI function calls or MCP integrations or when one of our agents like the SRE or Assist agent takes autonomous action to resolve an issue, it drives DPS usage. As agents increasingly become consumers of observability, this represents a growing opportunity that didn’t exist 2 years ago.

Atlassian

MCP calls were up 400% in the quarter with barely a blip in people actually using the applications through the web or through a mobile client.

The Teamwork Graph, importantly, we opened up at Team ‘26 US, which is a really important step with our MCP server, with our command-line interface, or CLI, the ability to put the Teamwork Graph to access its information in any Agentic harness that you’re using either chat applications, agents running on other platforms is really important. This is about being an interconnected part of a customer’s technology estate. It’s about the Atlassian platform being a strategic piece in the puzzle of how it is that they run their business. And I think that integration is a really important point. We’re seeing that the reason we talk about the MCP server, MCP and CLI users in the quarter, passing 1 million MAU. We don’t know where that puts us in a world scale, but I think it’s 1 of the biggest MCP servers that exist, more than doubling in the quarter is because of people getting to the Teamwork Graph getting to that context from their agents wherever they’re deployed.

GitLab

For more than a decade, GitLab has brought together the context of how software is built, secured and shipped across source code, issues, merge requests, pipelines, vulnerabilities, policies, approvals and deployments.

As AI becomes more capable, we believe that connected context becomes even more valuable, and we’re seeing early evidence of that with GitLab Orbit, our context graph for the software life cycle. Since opening the beta in June, more than 2,200 organizations have enabled orbit indexing, an increase of 70% in 4 weeks. Customers have generated more than 170,000 queries and roughly 80% of customer query volume comes from customers connecting Orbit to external agents such as Claude Code and Codex.

Year-over-year, secure repositories grew 60%, code pushes grew 50%, CI/CD pipelines grew 40%. Among some customers moving aggressively into AI-assisted development, we’ve seen code bases grow as much as 500%.

Security Software

Palo Alto Networks

The impact on our SASE platform is already evident, where agentic traffic has surged 9x over the last 9 months. Defending at this scale requires machine speed inspection or competence, a core competence we have refined for 2 decades, enabling us to block more than 30 billion attacks in a single day.

Prisma AIRS achieved a significant milestone, surpassing $100 million in ARR within 4 quarters of general availability. This represents the fastest scaling product in the history of Palo Alto Networks.

Fastly

We shared in Q2 that AI-generated traffic is growing at roughly 6.5x the rate of human traffic. Machines don’t browse the way people do. They query, scrape or act on someone else’s behalf, and that makes every request more complicated. This means every request requires an immediate decision. Is this an authorized agent, should it be cached, throttled, monetized and/or blocked? That’s why we see our security and compute products accelerating right alongside this machine traffic.

Akamai

Yes. And we have seen real tailwinds from AI and sort of the post-Mythos world for our more longer-term security products. I think roughly because of AI, the attackers have assembled much larger bot armies to launch attacks. And we’ve seen the scale of the attacks grow by maybe a factor of 10 over the last year.

CrowdStrike

In Q2, our endpoint business accelerated for the fourth consecutive quarter as customers look to secure their growing AI attack surface. We’re seeing dramatic expansion in agentic work on the endpoint, both through fast-growing tools such as Codex and Claude as well as custom agentic applications.

In sampling our customer base, we’ve seen more than 400% growth in Claude usage and more than 100% growth in custom agent usage on endpoints in recent months.

Okta

In terms of like where the products stack up in the quarter, we had, as I mentioned, the quarter was strong across almost every dimension, particularly strong was the 30% of the new bookings were from new products. Okta for AI agents inside of that bucket, there were dozens of deals in the quarter, including several million dollar-plus deals, which is super exciting. But the reality is it’s still very early. We do thousands of transactions every quarter.

And as exciting as that is, Okta for AI agents, it’s still -- it’s too small to show up in the numbers right now. But going forward, especially over the next couple of years, we’re super optimistic. We think this being the system of record for Agentic for agents in the enterprise and being the system of record for agent identity, in the fullness of time, it could be the biggest category of cyber.

Data from Ramp revealing the degree of revenue concentration at the labs was a Rorschach test.

The bullish take is that the top 1% are simply early adopters and the rest would reach similar spending levels, whilst others saw that same 1% plausibly switching to open models at the expense of the labs’ margins.

This degree of concentration is not unique to the models. A small number of buyers recur throughout the AI stack. But does a concentrated customer base necessarily give those customers power?

Let’s walk through a couple of examples each across different layers - the ratings below assess vulnerability to large customers switching suppliers, extracting concessions or reducing spending.

Models

Anthropic

Anthropic’s S-1 should drop in the next few weeks, which will give much more colour on customer concentration. Here’s what we know:

• 80% of revenue from B2B customers

• 6,000 accounts spend $100,000 or more annually

• Grew from 500 to 1,000 accounts spending above $1 million in under

  two months

• Estimated NDR of 140%

Whilst we have the rough data point for the top 1%, we don’t have the share of the top 10 customers. Meta, rumoured to be a large customer of Anthropic (SemiAnalysis estimates 3-5%), has finally caught up to the frontier, posing a real risk of churning.

We’ll have to see the attach rate on Anthropic’s products beyond coding once the S-1 lands.

Concentration Risk: Medium

OpenAI

OpenAI has pushed out its IPO and therefore the data is more sparse:

• >50% of revenue from enterprise, officially surpassed consumer, ahead of schedule

• 9 million business seats as of Februrary 2026

• Codex is said to have somewhere between 15-20m WAUs

• Advertising business now at $1bn run-rate

• Q3 growth faster than Anthropic according to Ramp, albeit from a smaller base

Relative to Anthropic, OpenAI has a broader product portfolio and a more diverse revenue base.

Concentration Risk: Low

Fable 5’s limited adoption and increasing open source adoption suggest we’ve already past the threshold of intelligence needed for most tasks, which poses a real threat to the frontier model premium historically enjoyed by the labs.

On the other hand, Anthropic has a chance to expand beyond code and make the majority of knowledge work as legible as code has been to the scaling laws and RLVR (how else can you justify a rumoured $30 trillion TAM). Jevons Paradox will absolutely apply as token prices fall and adoption goes stratospheric.

Who will capture that incremental token spend remains unclear.

In a stable era like the one we’re in now, the enterprise and application-layer post-training market is maturing to a point where the majority of enterprise token spend could be reallocated towards an ecosystem of specialised models that offer a far lower cost per task than the models on offer from the frontier labs.

Chips

Nvidia

As long as the world is fragmented Nvidia wins; the only way its margins come under pressure is if there are a limited number of buyers who have the means to take only part of Nvidia’s stack or replace them completely.

Ben Thompson

Nvidia’s had a busy couple of weeks buying Huggingface and licensing Poolside’s model development factory. A natural consequence of concentration both ways for the labs and Nvidia is an effort to commoditise their complements.

For Nvidia, a thriving open source ecosystem is existential to their future, lest they end up with only 2 customers who are investing heavily in replacing them. It’s also why Nvidia started breaking out their Data Center revenue into 2 segments: hyperscale and ACIE, which incorporates AI Clouds, Industrial and Enterprise. ACIE customers such as neoclouds are Nvidia shops, whilst the hyperscale customers have the resources and incentives for custom silicon endeavours.

Nvidia’s grip on the inference market is much weaker than training. The Jalapeño results demonstrate the viability of custom silicon built around inference workloads, whilst AI-assisted EDA has already started to compress timelines to tape-out.

Google’s TPUs and Amazon’s Trainium are already serving the frontier labs, even if that says as much about how compute constrained the market is as about the quality of the chips. Microsoft’s Maia and Meta’s MTIA are further behind, but remain priorities.

Despite these efforts, Nvidia’s moat remains formidable, echoing the dominance Intel held during the days of its close integration with Windows, the Wintel duopoly.

The arrival of mobile was Intel’s undoing. Arm’s ISA was superior to CISC for low power mobile computing, and Intel failed to recognise this in time.

Will AI inference be Nvidia’s undoing?

It depends. Is AI inference that different to the deep learning workloads Nvidia has been serving since it released CUDA in 2006? In many ways it is, but is the gulf as large as the one between PCs and mobile that felled Intel?

The evolution of the revenue split between hyperscale and ACIE will be telling. Nvidia’s reference design for ‘synthetic hyperscalers’ is unlocking hundreds of billions of dollars in growth for ACIE segment.

Concentration Risk: Medium

Cerebras

Cerebras is one of many ASICs competing with Nvidia for inference share, and has a much greater concentration. G42 was its biggest customer by far leading up to its IPO, and in 2026 OpenAI and G42 will represent 94% of its revenue, with OpenAI expected to be the overwhelming majority in 2027 and 2028.

Compared to Nvidia, Cerebras is more concentrated in every way. Focused on inference more than training, serving 2 (soon 1) customer with the means and incentive to develop their own silicon, and limited evidence of a new customer cohort like ACIE. Much more reason to be worried.

Concentration Risk: High

Broadcom

Broadcom has a broader product portfolio including VMware and is not designing chips as co-designing and implementing hyperscalers’ custom silicon.

• ⁠Semiconductors: $15.01B = 68% of revenue (up from 56% a year earlier)

• ⁠Infrastructure software (VMware): $7.18B = ~32%

• ⁠Within semis, AI is $10.80B = 72% of the chip segment, ~49% of the whole company. Networking is ~40% of that AI number ($4B+/quarter), custom XPUs the other ~60%.

Broadcom’s moat is process power.

Broadcom’s core competencies are non-core to hyperscalers’ custom silicon efforts.

Concentration Risk: Low

Cloud

CoreWeave

CoreWeave’s top 2 customers are 62% of revenue as of Q2. CoreWeave’s dependence on Microsoft and Meta was underlined when Microsoft was rumoured to be scaling back some of its commitments in the weeks leading to the IPO, causing widespread concern. OpenAI announced a $12bn commitment shortly afterwards, pacifying investors.

A consistent theme in recent Microsoft earnings calls is the notion of ‘fungibility’ of compute. The premise is that if the AI capex fails to deliver the expected ROI, the compute can be repurposed for other workloads. Microsoft itself can be a first-party customer for Azure, and in fact Azure’s growth is partially slowed down by Microsoft being a customer of its own compute. Microsoft also prefers to own rather than lease data centers, given their expertise in constructing data centers.

If compute is largely fungible, what are the implications for neoclouds like CoreWeave, which mostly run Nvidia fleets?

CoreWeave received a Platinum ranking on SemiAnalysis’ ClusterMAX leaderboard of neoclouds, higher than Nebius, OCI, Crusoe, Fluidstack and others. Beyond speed to power and the Nvidia relationship conferring early access to new silicon, this ranking ascribes value to CoreWeave’s PaaS offering for inference (also evidenced by MLPerf benchmarks).

However, relative to other neoclouds, CoreWeave is far more concentrated and exposed to hyperscaler capital allocation. This risk is shared by all neoclouds, but CoreWeave especially so. Microsoft

Concentration Risk: High

Nebius

Like CoreWeave, Nebius has Microsoft and Meta as customers as well, but also counts enterprise like Shopify, Cloudflare and others as customers too.

Nebius has a similarly compelling inference offering for AI workload, but given it’s origins from Yandex has even tighter optimisations across in-house-designed servers/racks.

The evolution of Microsoft/Meta share will be key.

Concentration Risk: Medium

Networking

Lumentum

The growth in model size and sparsity increase demand for scale up and scale out networking.

Lumentum is a key player in optical networking, supplying lasers used in optical transceivers and co-packaged optics. Google accounts for around 26% of revenue, with Microsoft and Amazon contributing roughly 8–15% each. Nvidia, Cisco and Ciena are also among its customers.

Like Broadcom, Lumentum’s moat is partly process power. Manufacturing advanced lasers requires specialised semiconductor facilities, accumulated manufacturing knowledge and consistently high yields. Designing your own accelerator does not automatically give you the capability to manufacture its optical components.

Lumentum has 50–60% share in leading-edge 200G electro-absorption modulated lasers, or EMLs. Customers put these components through reliability testing spanning multiple quarters.

Lumentum also has exposure across competing networking architectures. Conventional pluggable transceivers need lasers, as do emerging co-packaged optics, which bring optical connections closer to the processor or switch. The architecture can change without eliminating the need for Lumentum’s underlying capabilities.

The more immediate concentration risk is therefore spending, not substitution. Google does not need to learn how to manufacture lasers to hurt Lumentum’s growth; it only needs to delay deployments. And selling through equipment manufacturers does not necessarily diversify the underlying demand if those manufacturers ultimately serve the same hyperscalers.

Lumentum illustrates the distinction between customer concentration and customer bargaining power. Its buyers are concentrated, but so are the capabilities they need.

Concentration Risk: Medium

Arista

Microsoft and Meta accounted for 42% of Arista’s revenue in 2025, at 26% and 16% respectively. But unlike Cerebras, Arista has a credible route to adding another major customer.

Anthropic is the clearest candidate to cross 10% of revenue.

Anthropic uses Nvidia GPUs, Google TPUs and Amazon Trainium. Whilst those platforms have their own tightly integrated accelerator interconnects, operating across multiple platforms creates demand for a common networking approach elsewhere.

Custom silicon can therefore work in Arista’s favour. A customer moving away from Nvidia’s compute does not necessarily move away from Arista’s networking.

The moat is not Ethernet itself, which is an open standard. It is Arista’s EOS operating system and the operational consistency it provides across switching and routing hardware.

Arista’s concentration looks more manageable because it has both a defensible software layer and a plausible new customer cohort. The question is whether Anthropic becomes the first of many substantial new buyers, or simply the third name in another concentrated customer list.

Concentration Risk: Medium

Foundry

TSMC

TSMC is perhaps the clearest example of why customer concentration and customer power are not the same thing.

Its two largest customers accounted for 36% of revenue in 2025.

But the custom silicon efforts threatening Nvidia’s share can still generate business for TSMC. Google’s TPUs, Amazon’s Trainium and Nvidia’s GPUs compete for compute budgets whilst relying on the same foundry.

For Nvidia, fragmentation helps prevent a handful of customers from controlling its future. TSMC has a different advantage: it can manufacture for both the incumbent and the companies trying to replace it.

The moat is manufacturing process power at extraordinary scale. A customer needs more than a competing foundry with a similarly named process node. It needs sufficient capacity, competitive yields and a production process that can reliably deliver its design. Samsung and Intel have not yet matched TSMC’s combined leading-edge manufacturing and packaging capabilities.

TSMC does not necessarily need to capture every part of the manufacturing bill to benefit - TSMC welcomes competing packaging capacity because it removes constraints on its core wafer business.

Nvidia alone accounts for around 60% of CoWoS advanced-packaging capacity, with the top three customers exceeding 85%. Those are capacity estimates, not shares of total TSMC revenue.

If spending shifts from Nvidia GPUs to custom accelerators manufactured at TSMC, the foundry can retain the business. If spending on AI infrastructure falls altogether, there is no equivalent protection. Its customers’ exposure to the same downstream buyers still matters.

TSMC is better protected against a change in who wins the AI market than against a contraction in the market itself.

Concentration Risk: Low

Global Foundries

Global Foundries’ two largest customers account for 30% of wafer revenue.

Unlike TSMC, Global Foundries’ argument is not leadership at the smallest process node. It is specialised manufacturing for optical networking, power management, wireless connectivity and other functions where shrinking transistors is not the only measure of performance.

Whichever accelerator wins, the infrastructure still needs to move data and deliver power. GlobalFoundries can participate through silicon photonics and related technologies without having to manufacture the winning GPU or custom ASIC.

There are diversification opportunities in automotive, IoT, and communications infrastructure.

Concentration Risk: Medium

What happens when your customers succeed?

We usually think about customer concentration as the risk that a large customer fails, cuts spending or churns.

The hyperscalers and the frontier labs are the recurring actors throughout the AI stack. A lot is riding on their continued capex and top-line growth.

Ben Thompson’s Aggregation Theory offers part of the explanation for why the same buyers recur throughout the supply chain.

The platforms that won the customer relationship in the internet era enter this one with distribution, cash flows and workloads that support enormous infrastructure investments.

But the suppliers beneath them are not all interchangeable.

AI brings concentrated demand into contact with concentrated capabilities.

Inevitably, as the largest buyers scale, they gain both the resources and the incentive to develop alternatives to their suppliers. A customer can become more valuable to Nvidia whilst simultaneously becoming more capable of replacing it. Growth in the relationship does not necessarily result in retention.

Exposure to the hyperscalers and labs’ continued growth has unequal consequences. Custom silicon that challenges Nvidia can generate work for Broadcom and wafer demand for TSMC. The customer’s attempt to reduce one supplier’s pricing power becomes another supplier’s growth opportunity.

This is why concentration risk differs so much even between companies in the same layer. The important distinction is whether a supplier benefits from its customers’ growth, or becomes the target of vertical integration.

Everyone is exposed to aggregate demand slowing down, of course.

Long-term, I think we're going to end this year at maybe 50/50 background and real-time workloads, but I see this going to 90/10 in favor of background.

Neil Movva, CEO Sail Research

There have been three eras in the post-ChatGPT world: pre-training, reasoning, and agentic inference.

We’re firmly in the era of agentic inference. OpenAI’s recent agent-driven reacceleration, Grok Bot, Instinct and many recent releases give us a glimpse of a future of fleets of long-horizon background agents across the economy.

Last week I listened to Neil Movva, CEO of Sail Research, articulate how the workload properties of long horizon agents fundamentally differ from chat or training. The differences in properties have bearings on all of compute, memory and power, and if long horizon agents soon became the dominant workload, the ripple effects will reach far and wide.

We’ll walk through:

• Compute: Impact on CPU:GPU ratios

• Memory: HBM’s role (or lack thereof)

• Power: SMRs

Compute: CPU:GPU Reach Parity, Cores Increase

This is one of the most consensus takes at the moment.

Long-horizon agents use GPUs to think and CPUs to execute. The ratio of thinking to execution is expected to eventually result in CPU:GPU parity in datacentres, a steep rise from the 1:8 ratio in the first generations of AI factories.

With the number of cores directly driving agent performance, as Rene Haas described on Arm’s earnings call.

Unlike embedded CPUs inside GPU nodes, standalone CPU servers will run isolated sandbox environments to execute code, crawl the web, and fetch real-time APIs. For example, Microsoft's Fairwater Atlanta campus features a ~300MW GPU building supported by a separate ~48MW CPU and storage building dedicated to running these execution environments.

SemiAnalysis published a great essay on the various CPU efforts by x86 vendors, Accelerator companies with CPUs, Hyperscaler-designed Arm CPUs, Merchant Arm vendors and Chinese CPU suppliers.

Sail Research itself is attacking long horizon agent workloads as a systems problem.

As agent runs extend to hours and days, the onus will be on compute providers to optimise for the nature of this workload, oscillating between thinking and executing, suspending and resuming compute as state accumulates.

Memory: HBM & The Capacity Trade

Fundamentally, inference is all about dataflow.

GPU tensor cores perform matrix multiplications on weights and KV caches stored in memory - the attention mechanism in transformers relies on a lookup across all KV pairs computed during prefill for decode.

Transformers offer much better accuracy over larger context windows than RNNs but necessarily consumes more memory - decode grows the KV cache and needs to read the model weights with every new token. This is why decode is memory-bound and prefill is compute-bound - with agents, utilisation of GPUs drops significantly due to intensive memory consumption.

Model sizes and approximate layers:

• 1-3B parameters ~24-32 layers

• 7-13B parameters ~32-40 layers

• 70B parameters ~80 layers

• 100B+ parameters ~100+ layers

Of course, frontier models are trillions of parameters nowadays, with Grok 5 estimated to be more than 6 trillion parameters in size. These weights are easily 1 TB+ in file size. Kimi K3, at 2.8T parameters, is c. 1.4 TB at MXFP4.

Memory bandwidth is the limiting factor for decode, which is why Nvidia has become the biggest customer of HBM, and why HBM is sold out for the next few years across all three major providers (Samsung, SK Hynix, Micron).

HBM uses 3D stacking to increase bandwidth with each generation, which is ultimately a packaging innovation.

Scaling HBM through 3D stacking presents two challenges.

Firstly, HBM achieves high bandwidth largely through brute-force parallelism: thousands of relatively slow electrical connections running through stacked dies, rather than a smaller number of genuinely high-speed links. Those through-silicon vias and micro-bumps consume scarce chip “shoreline,” introduce capacitance that degrades signals, and become increasingly costly in power, heat and packaging complexity as speeds rise

Secondly, the fabs producing HBM are not inclined to ramp up production to meet future demand, having endured the cyclicality of the memory market in the past.

This is why memory bandwidth hasn’t kept up with compute.

It's about sidestepping the HBM shortage and focusing on more extreme offload to other forms of memory such as flash. I'm quite passionate about that idea. Everyone on my team knows that I keep banging the drum around what would we have to change about the model architecture to make offloading KV cache to flash work at a much greater level.

Neil Movva

The underlying premise for using HBM is to prioritise latency over capacity.

If long horizon agents become the dominant workload, what would the memory hierarchy look like? SRAM or HBM for active decoding, DDR for warmer state, and flash for the much larger body of inactive KV caches, checkpoints and agent history.

Instead of attacking the memory wall through reducing distance or compression, long horizon agents would suit pooling, i.e. leveraging another memory tier.

Whereas chat inference keeps almost the entire live request in accelerator memory, a mature long-horizon system could invert the ratio: perhaps 5% of agent state in HBM and 95% on flash. HBM would still carry nearly all active decode traffic, but it would no longer need to hold the accumulated state of every agent waiting to run.

A long-running agent accumulates tool outputs, documents, code, browser state, messages from subagents and checkpoints needed for recovery. This state continues growing throughout the task. Keeping it in HBM while an agent waits on an API, another agent or human approval would amount to reserving the datacentre’s most expensive memory for an idle process. Flash becomes the parking layer: HBM holds the state required for the current burst of decoding, while the much larger record needed to resume dormant agents sits in cold storage.

Chinese fabs like YMTC and CXMT have started shipping competitive NAND and DRAM products respectively, and will eventually commoditise this market, in turn making storage of state for long horizon agents orders of magnitude cheaper.

Power: Trading Concentration For Aggregate Capacity

Just as latency is traded for capacity when it comes to memory, uptime is traded for capacity when it comes to power.

Training requires hyper concentration of compute. Chat inference is the next rung down on the ladder, still favouring compute density to deliver fast decode.

Long horizon agents can leverage a fleet of small compute pools, as Neil describes:

The role of a long horizon agent runtime is to route agents across this distributed compute to make a thousand one-megawatt pools match the performance of a gigawatt campus.

Furthermore, once an agent’s state lives outside the accelerator, the agent is no longer tied to a particular server. An agent can be suspended at one site, its state persisted to storage, and resumed wherever power and accelerators become available. Flash does more than reduce HBM demand: it converts fragmented compute pools into a fungible fleet.

Gigawatt-scale campuses remain extremely difficult to build, and it’s very likely that OpenAI and Anthropic capture the majority of new incremental compute built over the next few years.

Whilst the labs compete for larger pools, smaller brownfield and greenfield pools will be up for grabs.

To that end:

• Neoclouds like Crusoe are building modular data centers scalable from hundreds of kilowatts to tens or hundreds of megawatts, whilst others like Megaport build distributed inference capacity in increments of single-digital MW building blocks.

• Small Modular Reactors (<300 MW per module) moving from demos to production, e.g. Valar Atomics

• Growth in GPU marketplaces like SF Compute, Akash Network, Prime Intellect, Andromeda

• Bloom Energy packages its fuel cells in compact energy servers, where 4 to 6 server modules are combined to form 200-300 kW modules, whilst FuelCell Energy launches 12.50 MW utility-grade power blocks that can run on pipeline natural gas today and transition to green hydrogen or biogas later

Conclusion

The common thread is a change in the unit of optimisation.

Training optimised for FLOPs and chat for time-to-token; long-horizon agents will increasingly optimise for cost per completed task. Resources previously suboptimal become economically useful: CPUs execute, flash warehouses state, and small pools of compute and power are stitched together by an agent runtime. HBM and gigawatt campuses will continue to serve the fast inference market, but that may no longer be the dominant workload.

Model Labs and Agent Labs are converging.

Anthropic continues to develop first-party applications that compete with their customers..

Whilst Harvey comes full-circle and starts training models..

Workload-Harness Fit provides a framework to assess which workloads are best optimised through either pure agent engineering (e.g. a harness that doesn’t touch the weights) on the one end through to end-to-end (pre)training runs on the other end.

Legal AI workloads have the following properties: modest volumes, high value per task, medium on verifiability, medium-duration time horizon per task. In sum, legal workloads would sit somewhere in the middle on this spectrum: the case for model training isn’t clear cut.

But the training imperative has never been clearer.

The enterprise AI market has stabilised and settled on intelligence per $ as the cleanest ROI metric that resolves disparities in headline numbers like input/output token costs, token-intensity, and more broadly benchmark maxing on tasks that don’t reflect the ‘untrainable’ work inside enterprises.

A Pareto Frontier has been established, with buyers assessing vendors’ ability to continue pushing it further out through optimising model-harness combinations.

What is a harness, anyway? There’s a running joke that the pejorative ‘wrapper’ term was just rebranded to ‘harness’, but the reality is it’s simply an evolution of what infrastructure needed to be built to realise the full potential of raw intelligence.

This anatomy from Langchain helps:

That was the premise of agent labs: build the best harness around base models for a given domain.

That co-existence is under threat.

Anthropic is co-designing the harness for their models.

Yang Zhilin, co-founder of Moonshot AI, the lab behind the Kimi family of models, sees this as the next natural evolution of the company’s push into first-party apps:

Poolside’s Eiso Kant has focused its harness development on coding and long-horizon software engineering tasks, eschewing tool registries and MCPs and effectively reframing the definition of a harness to be container, filesystem, provisioned credentials, and codebases:

Whilst conceding there is room for companies to build bespoke harnesses for specific capabilities:

Until now, most model labs had only built harnesses for specific domains, primarily coding. The intent to expand surface area is clear from feature releases and M&A activity.

The gains from co-designing models with harnesses is why agent labs need to move into model training, or risk obsolescence when the model labs concentrate their resources on your domain.

As I’ve said before, the economics of training have changed dramatically over the last four years, which means that becoming a ‘lab’ no longer has the same connotations of capital-intensive, open-ended R&D that it once did.

Pushing out the Pareto Frontier for a given domain is the most important mission for agent labs to work on.

That might end up being the most durable differentiator versus model labs.

Anthropic’s inference gross margins on its API business are reported at over 70% currently, up from roughly 38–40% in 2025.

The challenge is for agent labs to offer the best intelligence per $ at margins that the model labs can’t match.

When the model labs are sitting on lucrative API or advertising businesses and a hundred competing priorities, their ability to concentrate the resources needed to match those margins is slim. Agent Labs should be the ultimate winners in many vertical markets, for these reasons and others (e.g. regulation).

To offer the Pareto optimal frontier for legal, finance, or other domains, agent labs need to co-design models and harnesses too, which is what’s now unfolding.

Last week I had the pleasure of interviewing Jeremie Eliahou Ontiveros from SemiAnalysis. Eliahou Ontiveros is Managing Director for Datacenter, Energy, Industrials Research at SemiAnalysis.

We discuss the state of energy in Europe, bottlenecks on the AI buildout, and more.

To start, could you give a quick intro on yourself and how you got to SemiAnalysis?

I joined SemiAnalysis about two and a half years ago. Before that I was in the finance industry as a buy-side analyst for around four years, and I basically fell into the semis rabbit hole. I was covering some semiconductor stocks in Europe, STMicro, Infineon and so on, wanted to learn more, and discovered SemiAnalysis as an extremely good resource.

It was quite shocking to me, because as a buy-side analyst part of your job is to judge analysts. You don’t have time to go super deep on every sector, so you talk to the sell-side and try to figure out who actually knows the industry. When I started reading Dylan’s content, I thought: this is deeper than anything we have access to as a buy-side firm. I became a subscriber, then saw a Twitter post from him in August 2023 saying he wanted to build a real business out of SemiAnalysis. We started chatting. There were just seven people when I joined in February 2024. We’re 90-plus now, so it’s been an interesting ride.

Energy: Where Europe Stands

SemiAnalysis does very granular monitoring of data centre build-outs, tracking construction starts and leasing data centre by data centre. What does that data reveal about hyperscaler intentions in Europe? Are they underweighting the region, and if so, why?

In Europe we saw signs of activity in 2023 and 2024, demand started to pop up, but then it essentially cooled down. If you look at data centre capacity additions in 2025 relative to 2024, it’s actually stable if not down in Europe. The market cooled quite a bit.

If you want to build at the speed and scale these companies want, you need large sites that are expandable, that can be built fast, where you can get started typically 12 months after the lease is signed. In Europe the biggest markets are near metro areas, and it’s tough to build fast at scale near Paris, for example. So all of the good supply got taken up in 2023 and 2024, and that left an air pocket.

The gap between the US and Europe has widened as a result. The US gained share of global megawatts even in 2023 and 2024 when Europe was relatively stronger, and in 2025 that gap widened further. The market simply wasn’t able to react to the surge in demand.

Part of it is that many European operators weren’t able to anticipate the changes in the industry. In the US, builders understood earlier that AI customers want something different: lower redundancy, faster builds, much higher density. European data centre builders took longer to adapt. They would tell you they weren’t seeing AI demand, but to some extent demand wasn’t showing up because there was no supply adequate to meet it. A chicken-and-egg problem.

The good news is that since the beginning of the year we’re seeing Europe come back. For inference you do want a local presence, and what that typically means is fairly large hubs that can serve a broad range of workloads, including training. Inference drives the need, but what an Anthropic or an OpenAI wants is a large data centre that can do multiple things, so they can increase utilisation and scale faster. Some markets are positioning to be those hubs. In the Nordics you’re seeing quite a bit of activity, multi-hundred-megawatt buildings going up. Not US scale, but catching up.

France looks like Europe’s one real energy advantage: nuclear baseload, EDF, and now Mistral’s planned 1.4GW campus with Nvidia. Do you expect France to translate that into becoming Europe’s leading AI hub?

I think it’s going to be quite complicated. Energy is a key input and France scores well there. We have a lot of nuclear power in France, and it runs at a fairly low utilisation rate relative to the US; France exports a lot of power. There’s plenty of energy that could be tapped for large-scale data centres.

But building a project like that takes more than energy.

You need the supply chain, the labour, and especially permitting, which is a big hurdle in France. A gigawatt-scale project is a massive effort, with up to 10,000 people on site every day, and if you add diesel backup generation permitting the project gets even more complex. Then there’s local pushback. France is a much denser country than the US, so anywhere you build, you’re likely to have communities nearby who really don’t want a gigantic data centre next to their home.

That happens everywhere, including the US. But in Texas there’s plenty of land: you take your 3,000 acres in the middle of nowhere, the first house is miles away, it’s all ranches, and you can burn gas. So France has an energy advantage but a permitting problem. It’s just tougher to build very large-scale projects there.

Your Colossus coverage documented the xAI playbook: bridge power behind the meter, grid connection later. That reset expectations for time-to-power in the US. Does any version of that playbook work in Europe? You’ve flagged Ireland as an exception.

Ireland is interesting because it already has a history with behind-the-meter. There are fairly sizeable deployments live in Ireland today; that market was something of a pioneer. Some use engines, some use small turbines, some use fuel cells. Bloom Energy has deployments there, for example.

But overall it’s more complicated in Europe. One, gas is much more expensive, so the economics don’t work as well. Two, the permitting regime is more complicated. Even in the US it’s hard to permit, which is why most people go to Texas, and really West Texas, not even near Dallas. In some European markets you also pay carbon prices on top, which essentially doubles the cost of your gas.

It’s structurally more expensive than grid power for the most part.

It’s all about speed.

And that speed advantage fades in Europe because you can’t do it at scale. In very localised situations it makes sense. Ireland is that case: the hyperscalers all have big cloud regions in Dublin they wanted to expand, they figured on-site gas was the only way, and Ireland even passed regulations to that effect. But the scale is tens of megawatts, maybe low hundreds.

In the US you’re talking about something like over 20 gigawatts of behind-the-meter data centres as net additions in a single year. The scale is just not the same.

You’ve made the case that data centres could be the demand anchor that finally justifies grid investment, as has happened in Texas. Could AI demand play the same catalytic role in Europe, or do regulated European utilities lack the incentives to respond?

The first thing is that utilities are not going to invest if there’s no demand. Demand has to show up first. The second thing is that in some countries, especially France, there’s already available capacity before you need massive transmission upgrades. You’d see people build substations and do minor upgrades to get power to specific locations first.

The major upgrades come when the multi-year projections change. If you look back ten years, many European utilities would tell you electricity demand was going down. That’s why they didn’t invest, and why they retired generators and nuclear plants: the outlook was that efficiency gains would outpace electrification.

Once people internalise the idea that long-term there’s going to be load growth because of data centres and AI, that’s when you can build transmission with confidence that it benefits everyone. It’s not a cost to utilities or households, because the off-takers, the hyperscalers, largely carry those upgrades, and the upgrades run at high utilisation.

That future could materialise in Europe, but only if the demand shows up. And to have that demand, you need to fix supply: permitting, local pushback, making these build-outs go smoothly so companies actually want to increase their presence in the market. Then you can start funding massive grid upgrades.

Since Europe can’t run the American gas playbook, what’s the fastest route to firm power? The US is restarting Palisades and Three Mile Island; Oracle is deploying Bloom fuel cells; Google and Meta are contracting geothermal. Germany has closed reactors sitting idle and France has an ageing fleet. What’s a real option for Europe?

You could always bring some of these resources back online, some nuclear plants for example, or off-take new plants; France has said it wants to build new nuclear. The problem is you need an off-taker. Look at the contracts that enabled Three Mile Island to come back: 15-20-year-plus off-take at fairly elevated pricing. That’s a massive deal, billions of dollars over the life of the contract, and it works because the buyer knows they can build a useful data centre with that power.

If you can’t even build data centres in the market because of permitting, what’s the point? You’re not going to sign gigawatt-scale off-takes, and bringing back a nuclear plant, which is always gigawatt scale, only happens with those multi-billion-dollar contracts. So yes, there’s capacity that could come back online, but to get there you need the demand and the off-take. It could also be new plants: if you could underwrite new nuclear, that could work too, but that’s an enormous project that would have to be fully or mostly funded by hyperscaler off-take.

Grid and permitting

US interconnection queues have become something of a prisoner’s dilemma, with speculative requests approaching a terawatt. European queues have their own issues: years-long connection dates, capacity hoarding. How would you reform the European system?

That’s a great one, and it happened in the US too; many markets there have gigantic queues. I think you just look at what happened in the US as a leading indicator of what works.

What happened in 2023 and 2024, when the first signs of power demand growth appeared, is that anyone near a transmission line would call the utility and ask for power. If anyone can submit a request, everyone floods every utility. In the US that has led to a total queue of large data centres looking to interconnect of about a terawatt, which is above US peak load. It’s obviously not going to happen; you don’t double peak load in a click. There are massive double, triple, quadruple requests, and some are completely fake.

So you’ve seen a combination of responses.

1) Financial commitments. If you want gigawatts, you put money up front, you fund the studies, you post a deposit for the substation. The capital intensity goes up, which excludes the more speculative players.

2) Much closer collaboration between the company and the utility. It’s not “I’m near a transmission line, give me power.” You come with a proposal, ideally one that adds your own resources. Oracle got 1.2 gigawatts from DTE Electric in Michigan by going to the utility with a plan to bring a gigawatt of batteries in specific locations, which made the whole build-out make sense for the utility. That also requires transparency from the utility and transmission system side.

Another option is disclosure of your other requests. Texas has introduced this idea: if you want a gigawatt, you have to disclose where else you’ve requested one. If you’ve got 40 other requests within a 20-mile radius, the utility can see you’re not serious. More transparency on both sides is basically what’s needed to distinguish real projects from speculators who just wanted to be first in the queue and might flip the position.

Your research showed how synchronous training jobs can swing from full load to zero in milliseconds. Does Europe’s grid structurally favour one workload over another? Would inference be better suited to Europe than training?

The load fluctuation problem can be solved fairly easily; it’s just another phase of a problem grids have already faced. It’s the same as suddenly cutting generation: if you cut load instantly, the effect is similar, and operators now know how to deal with it. Everyone in Europe saw the Iberian blackout, got freaked out, and did everything possible to make sure it never happens again. People are sophisticated enough to understand the implications, so I wouldn’t see it as a training-versus-inference issue.

The real point is that these labs just need scale and speed.

The example I like giving: at the end of 2025, Anthropic had around one and a half gigawatts. By 2027 they want over ten. That’s essentially building the equivalent of Google in two years, an infrastructure Google spent two decades building and put about $90 billion of capex into last year alone. To do that, your data centres have to be on time.

So the training-versus-inference debate gets abstracted away. What you want is scale, and then you adapt. Some inference workloads genuinely need proximity and uptime, and those will exist in every market, in Europe, in Asia, there’s no way around it. But they’re a small share of megawatts. The bulk of megawatts will go to hubs, and the question for any country that wants a high share of the build-out is: can you accommodate the larger build-out and become the hub that serves multiple workloads and neighbouring countries?

In APAC, Australia is clearly becoming that hub. Many gigawatts are being built there right now; the country has demonstrated it can accommodate the speed and scale, and it will likely serve many other markets in the region, not just Australian demand. The ideal scenario for France would be the same: build large-scale infrastructure that serves Germany, Spain and so on. But the core issue remains that you need to be able to build. There’s a scenario where the Nordics get most of it and continental Europe ends up with a fairly small share of the pie.

Policy and sovereignty

Last question. We’ve talked about permitting reform, grid interconnection reform, and there are ideas like the UK’s special compute zones. If you had to prioritise one thing for Europe to become more competitive, what would it be?

Probably compute zones. It’s always a tough trade-off, because whenever you bring in the government it tends to slow the whole process down; ideally you want more private sector. But on the other hand, these countries are so difficult to build in that you probably want zones that are agreed upon in advance and won’t be troublesome. Open communication and partnership with local communities is perhaps even more important than elsewhere.

The key is making sure it’s not just about energy. France did a good job finding sites with energy, but the idea would be to find sites with the whole package: great to build in, fast, at scale, where the local mayors have been briefed and know what to do when someone comes in with a massive project, and where the whole community understands it. That would be a good outcome.

When Moonshot publishes the Kimi K3 checkpoint on July 27th, it will be the largest open weights model ever released: 2.8 trillion parameters, a 1M token context window, native multimodality, and benchmarks that clear Opus 4.8 and sit roughly one generation behind Fable 5 and GPT-5.6.

The discourse has fixated on 2.8 trillion.

The more important number is 16.

From 28% To 2%

K3 activates 16 of 896 experts per token — under 2% of its expert weights touched per forward pass. That is not an implementation detail. It is the terminal point (so far) of the most consistent architectural trend in open models.

Total parameters have grown ~20x since Mixtral, and ~3x in the last twelve months alone.

Active parameters have barely grown at all: a 17-49B band for 27 months. Moonshot shipped K2, K2.5 and K2.6 across nine months with the identical skeleton — 1T total, 32B active — three releases, zero growth in active parameters.

There is scatter (GLM-5.2 is less sparse than V4-Pro), so this is a trend with variance, not a law. But the direction is unambiguous: hold per-token compute roughly flat, inflate total capacity relentlessly.

Why? Because at a fixed training compute budget, more experts means lower loss. The model learns more from the same FLOPs. And the bill is paid in the one resource that has cheap tiers, storage, rather than the two that are scarce and rationed: compute and memory bandwidth.

The open source labs have discovered that the cheapest way to buy intelligence is to spend capacity.

Sparsity is partly an adaptation: when FLOPs are rationed, you scale the axis outside export controls.

KV Cache Compression

Even though expert weights are getting sparser, longer context windows grow the KV cache, and the cache is hauled from memory on every token, exactly like weights.

This is where the companion trend closes the loophole. Attention compression (DeepSeek’s CSA/HCA hybrid, Multi-head Latent Attention, K3’s new attention architecture) is collapsing the cache. V4-Pro’s KV cache at 1M context is 10% the size of its predecessor’s.

Sparser experts shrink the weight-bytes moved per token. Compressed attention shrinks the cache-bytes moved per token. Nothing on the model roadmap shrinks the bytes stored. That number only goes up.

Every architectural trend points away from peak bandwidth scarcity and toward capacity as the binding constraint.

Constraint moves from compute to storage

Sparsity is what makes a 2.8T model servable at all. As Jamin Ball from Altimeter wrote about the true cost of serving frontier open models:

One force that historically factored into open model pricing (anyone can serve it) is much weaker when “anyone” means “anyone with a supernode and a serving stack tuned for a brand new attention architecture.”

A 2.8T model probably has a structurally higher serving floor than a 1T model no matter is serving it. The K2-era 10x discount existed (partially) because those models were both smaller AND served at thin margins. K3 only gives you the second one. For two years “open” and “cheap” were used interchangeably, but they were never the same thing.

Due to sparsity, per-token inference runs ought to run at the cost of a mid-size dense model.

In that sense, sparsity has democratised the compute of frontier inference.

It has done nothing for capacity. At MXFP4, K3’s weights alone are ~1.4TB. Ten-plus H200s just to load them, before the KV cache on a 1M window - this is why Moonshot recommends supernode configurations “with 64 or more accelerators.”

Ever increasing sparsity moves the cost from compute to storage.

Two Serving Regimes

A plausible scenario in low-batch environments (such as enterprises looking to self-host) would see a tiered memory approach (or memory hierarchies) with most frequently used experts stored in HBM whilst less frequently used experts would be in cheap DRAM. A sort of power law for MoE models.

This cost/efficiency gain disappears in hyperscale environemnts. A production server batches hundreds of concurrent requests, and each token picks its own 16 experts. Collectively, a full batch lights up most of the 896 every forward pass. There are no reliably cold experts. So frontier serving pools everything in rack-level HBM across 64+ accelerators and routes tokens to the chips holding their experts. In this regime, sparsity doesn’t reduce HBM purchased at all, it converts bandwidth demand into HBM capacity demand. More stacks, streamed less hard.

At low utilisation, a single power user, i.e. an enterprise with a handful of concurrent sessions, experts genuinely are cold, and tiering the model into system DRAM while hot-loading experts works. This is the regime that matters for open weights specifically, because the entire point of downloading a checkpoint is running it outside a hyperscaler. Sparsity is the only reason the self-hosting path exists at all for trillion-scale models.

Routing is paramount. Today’s routers spread tokens roughly uniformly across experts, which is why hot/cold tiering fails at scale. If labs train routers with deliberate locality, i.e. popular experts, predictable paths, true tiering becomes viable even at high batch, and the cheap-capacity DRAM camps win much bigger.

What do we need to believe for orbital compute to be viable?

Cost, Time To Power, Performance

The grid was built for 1-2% annual demand growth.

AI is a supercycle that the grid can’t handle.

There could be a 38 GW power shortfall between 2026 and 2028 for data centers.

An estimated~$156 billion of projects were cancelled or delayed in 2025, and $130 billion have already been cancelled or delayed in 1Q26.

Reasons range from supply chain shortages and long lead times on electrical equipment to political pushback.

Since I started writing this post, New York became the first US state to impose a one-year ban on data center development. has been tracking data center moratoriums across the US: 40 states have already been affected.

There’s a real risk that the AI buildout will not be able to keep up with demand driven by agentic inference, particularly in Western economies that are not able to rearrange atoms as fast as bits (i.e. energy, manufacturing): Agentic Reasoning tasks can consume ~50 Wh vs. ~0.3 Wh for chat.

For the better part of two decades, the dominant assumption in technology was that software scaled infinitely while the physical world slowly faded into the background.

AI is reversing that logic.

The next phase of the technology cycle will not simply be determined by better models or smarter algorithms, but by access to scarce physical resources: Chips, memory, electricity, land, cooling, and industrial infrastructure. In other words, the AI race is no longer just a software race. It is an industrial-scale, infrastructure race.

Apollo

When Huang says “iso power” he means that power is the constraint; the question is how many tokens can you generate given a fixed power envelope, and Blackwell can generate a lot more.

China — which actually builds power generation — is not iso power; they are iso compute: Huawei’s goal with the CloudMatrix 384 is to get as much compute as possible, power efficiency be damned.

Ben Thompson

The west has to pursue orbital compute - OpenAI and Anthropic alone will consume 100 GW by 2030. Not just because it might be cheaper one day, but because it may deliver faster time-to-power.

But the economic argument will get stronger over time.

What does orbital compute represent? Racks in space, satellite-sized, at dawn-dusk sun-synchronous orbit (panels always lit, radiator always shaded); racks linked by lasers into a virtual data center.

The cost structure of orbital compute breaks down into launch capex, satellite hardware capex and compute payload capex. Each of those is trending downwards, with estimates that orbital compute capex/W falls from US$166/W in 2028 and US$60/W in 2030 to US$32/W in 2031, US$15/W in 2035 and US$9/W by 2040. By 2031, a five-year useful life nets out at $6.5/W/year, comparable to the current rate of Blackwells on earth.

But time-to-power may be the more decisive argument.

Space provides a scalable power envelope that bypasses grid interconnection queues, which currently stretch for years in the US and even longer in Europe. This is compounded by gas turbine availability, transformer lead times, and skilled labour shortages.

Launch cadence remains the unknown. Estimates for Starship launch cadence in the 2030s range widely, as do estimates for the amount of power and compute in orbit.

Beyond cost and time-to-power, what workloads will go into orbit?

The bull case would see training remain on terrestrial compute whilst inference moves to orbit.

As it happens, the biggest inference market, agents, may well suit orbital compute the most.

The other interesting angle is space: slower chips actually make space data centers more viable for a number of reasons.

First, if memory can be offloaded, chips can be made much simpler and run much cooler.

Second, older nodes, by virtue of being physically larger, will better withstand space radiation.

Third, older nodes require less power, which means there will be less heat to dissipate via radiation.

Fourth, not being on the bleeding edge will mean higher reliability, an important consideration given that satellites won’t be repairable.

Ben Thompson

The Stable Era

In Liu Cixin’s Three Body Problem, the planet Trisolaris alternates between chaotic eras, when three suns move unpredictably and civilisation is impossible, and stable eras, when the orbits settle and everything gets built. The inhabitants cannot control which era they are in. They can only recognise the transition and act on it.

Together AI’s CEO Vipul Ved Prakash wrote a compelling essay arguing that we’re in a stable era in AI, identifying three interfaces that have standardised: the transformer architecture below, the OpenAI-compatible inference API above it, and the agentic harness (with MCP as its tool protocol) on top.

Carliss Baldwin and Kim Clark argued that the most important economic event in technology industries is often not the invention of a new product, but the creation of a modular architecture with stable interfaces.

But what is perhaps less appreciated is that, as with previous platform markets, the appearance of stable interfaces is enabling specialization, independent innovation, and a modular re-organization of the AI ecosystem.

AI has already converged on three powerful, standardized interfaces:

1. The transformer architecture

2. Inference API (OpenAI compatible)

3. Agentic harnesses that operate over these APIs

In much the same way that steam engines, interchangeable parts, and shipping containers transformed fragmented crafts into industrial systems, the transformer has become the common industrial substrate for intelligence.

Every improvement to attention algorithms, optimizers, kernel libraries, inference engines, and training frameworks, advances the frontier for nearly every model at once, and an entire supplier ecosystem has emerged around it, spanning software frameworks, silicon, and models.

Invoking Baldwin and Clark’s theory of modularity, he argues that the most important economic event in a technology industry is rarely a product. It is the arrival of stable interfaces, because stable interfaces are what allow an ecosystem to specialise, disaggregate, and improve along every axis at once.

The PC did not out-innovate vertically integrated computer companies because any single component was better. It won because Intel, Microsoft, and thousands of peripheral vendors could innovate independently against a shared contract. Sun Microsystems, was undone by Linux plus commodity x86 not a better integrated competitor.

Stabilisation’s consequences

Together’s API volume grew nearly 10,000x in nine months, from 30B to 400T tokens a month, most of it open-weights models driving agentic workloads.

More importantly, the open frontier is no longer a distillation artefact of the closed one: the modern transformer block is an assembly of freely diffusing parts (RoPE, RMSNorm, SwiGLU, GQA, MoE), and training methods spread the same way, with DeepSeek’s GRPO propagating across the ecosystem within weeks of publication.

Open tokens now price an order of magnitude below the closed frontier tokens they substitute for, and the moment an open model crosses “good enough” for a workload, the closed premium on that workload collapses toward zero.

Cheap, capable, frictionlessly substitutable open weights are the substrate. What gets interesting is the layer that sits on top of them.

The economics of shaping intelligence

The marginal cost of shaping intelligence continues to fall.

Bridgewater’s AIA Labs recently released results from from their collaboration withThinking Machines Lab The task: filtering and triaging financial documents the way an experienced investor would. Frontier models managed roughly 50% accuracy with naive prompts and plateaued below 80% even after expert prompt engineering, short of the threshold investors would trust in a daily workflow. Notably, the frontier curve was flattening: GPT 5.4 cost 43% more than 5.2 for marginal accuracy gains. The binding constraint isn’t raw frontier intelligence.

Bridgewater trained Qwen3-235B on expert-labeled data via Tinker, Thinking Machines’ training API, and reached 84.7% average accuracy against the best frontier model’s 78.2%, a 29.8% reduction in errors, at 13.8x lower inference cost per task.

The result generalised across many internal tasks beyond the six published. They characterised this as “differentiated intelligence”: custom models tuned to specific organisational needs that outperform frontier models on those needs (Applied Compute calls this ‘specific intelligence’).

The strongest argument against post-training is the lack of transferability when a new SOTA model is dropping every 35 days.

A fine-tune, full or LoRA, is locked to one base model’s weight space. New models now arrive so fast that the state of the art holds the top of the public leaderboard for roughly 35 days on average. Every release forces a choice: forgo the newer, smarter base, or pay to re-tune from scratch. The cost of maintaining a portfolio of fine-tuned capabilities scales inversely with the time between releases. Under that math, adaptation is perishable capex, and “just wait for the next frontier model” is a rational procurement strategy.

PorTAL, published by Ramp Labs, is the first credible attack on the structure of that cost rather than its level. The idea: learn the task adaptation once, in a base-agnostic form, and port it to each new frozen model by refitting only a thin per-base converter on a small calibration set.

The results are impressive.

A task representation learned on Qwen3-1.7B and 4B recovered ~98% of a from-scratch LoRA’s accuracy lift on an unseen Qwen3-8B, and ~94% on Gemma-3-4B, across model families. The comparable prior method, Cross-LoRA, recovered only ~14%. The refit needed roughly half the calibration data of training from scratch.

If task adaptations become portable across bases, a fine-tune stops being an expense you re-incur per model release and becomes a durable asset that rides the frontier. The organisation’s accumulated judgment, encoded once, appreciates as the substrate improves underneath it. The cost-benefit case for owning your adaptation stops depending on frontier progress slowing down, which was always the weakest link in the argument.

A base-agnostic task latent is to fine-tuning what the container was to cloud workloads: the abstraction that decouples the thing you own from the substrate it runs on, and thereby makes the substrate a commodity you ride rather than a dependency you fear.

Recipe expertise, today's platform differentiator, diffuses like everything else in this ecosystem: Bridgewater published its entire recipe, ablations included. Methods are a wasting moat. Proprietary data and captured judgment are not.

Post-training has not had its modularisation event yet, but this might be it.

Owning rather than renting intelligence has never been more important. This was true even before export controls on frontier models kicked in.

Frontier tokens cost 10x as much as the best open models.

It’s a similarly stark picture when assessing cost per task.

The reality is that the majority of enterprise software workflows don’t need frontier intelligence.

The case for owning intelligence rather than renting it isn’t just an economic one, but also one based on performance.

It’s clear to me, then, that intelligence here is a runtime phenomenon. Two deployments of byte-identical weights can give you two wholly different experiences: the precision and quantization they’re served at, how the cache gets managed, the pool your request lands in, the speculative decoder racing ahead, the reasoning effort, the noisy neighbors of multitenancy.

If you don’t own the serving engine, the precision, whether the KV cache is paged or quantized, how requests get batched and scheduled then every one of those choices is a vendor’s right to change without telling you. If you do, then when the output changes you have recourse: you can git blame or bisect instead of flaming your vendor on r/{{whatever}}.

Adam Azzam

Adam Azzam of Modal compares model weights to a master tape: pristine, mixed once, sitting in a vault. But you never hear the master tape directly; you hear it played back through a mixing board and room someone else controls: the labs.

That playback chain is the serving infrastructure: routing, quantisation/precision, caching, batching, reasoning effort. It’s why your product has regressed on your evals even though the model was the same.

In addition to serving infrastructure, Lin Qiao of Fireworks makes the case that the frontier is more than just the best closed models from Anthropic and OpenAI:

There isn’t one frontier. There are many.

• A frontier model is one kind of frontier.

• A model post-trained on years of proprietary company knowledge is another.

• A specialized model that solves one narrow problem better than anything else is another.

• A router mapping request to a collection of models working together to outperform any individual model on many tasks.

Several applied AI companies have stepped up efforts to own their intelligence. The most notable example is Harvey, who have published multiple papers on how they’re post-training models for their domain, prompting cofounder Gabe Pereyra to explain the various research angels they’re exploring:

In post-training Nvidia’s Nemotron family, they’ve pushed out the Pareto frontier.

Harvey’s Legal Agent Benchmark, Cursor’s CursorBench, Rogo’s Big Finance Bench, and Cognition’s Frontier Code are examples of what Sarah Guo characterised as the untrainable corner of enterprise work.

The prize is the last corner, the untrainable one: frontier work whose correctness exists only in private. You can see it in the inference clouds hosting the AI-native pioneers, where the vast majority of tokens are generated by custom models, not generic open ones.

The evaluation that decides real money is private and per-firm: what this firm, on this kind of matter, will accept as good work, and it is nowhere near finished, because the depth of the law dwarfs any public test. OpenEvidence is settling what a safe clinical answer looks like. None of this is really measurement, it is judgment about what is true and what is good, written down until it becomes the standard everyone else is measured against, and a foundation lab can’t author it however smart it gets, because that standing only exists inside the field.

Sarah Guo

If applied AI companies can hill-climb on domain-specific evals, Satya Nadella and Microsoft AI are betting that enterprises can too:

Companies need to turn their workflows, domain knowledge, and accumulated judgment into AI systems that improve with each use. Private evals should capture whether a model is actually improving against outcomes that matter to the business (not just external benchmarks!). Private reinforcement learning environments should let models grow stronger on real traces from inside the organization. Its knowledge base makes institutional memory queryable and use of tokens more efficient.

Satya Nadella

The technical report on MAI-Thinking-1 breaks down the components needed for enterprises to build their own hill-climbing machines:

The integrated process of building data pipelines, training infrastructure, reinforcement learning environments and rewards, evaluation suites, and safety tests that turn model development into an empirical optimization loop on a specified domain.

Optimisation doesn’t stop there.

Owning intelligence has clear cost and performance benefits, but there’s more systems engineering to be done.

The empirical optimisation loop Microsoft are advocating for isn’t just to hill-climb on on performance, but across several dimensions.

It’s becoming clearer that energy is the ultimate bottleneck in AI inference. Workload-Harness Fit is narrow form of optimisation and needs to be extended all the way to measuring intelligence per watt.

In pursuing that optimisation, the kind of compound systems engineering that’s needed becomes clearer.

Satya Nadella: Take that example we showed with Land O’Lakes today, which is, here’s an agent, and there is an outcome you care about, I was able to use a model that is using 500B, I was able to use a 5B, and have it really deliver the same outcome, why would I not use that?

Ben Thompson: That does seem to be a very different thing about this period. It seems clear that’s going to be a huge thing in enterprise going forward, using the right model, optimizing, it’s like we didn’t get to the optimization stage of the PC era.

OpenRouter’s compound Fusion model is the best example of this to date.

A combination of Gemini 3 Flash, Kimi K2.6 and DeepSeek V4 Pro costs c. 20% of the Fable 5 + GPT-5.5 combination whilst only being 4% worse on a deep research benchmark.

For all of these reasons, I see scenarios 1 and 3 have the strongest tailwinds, with significant second-order effects on frontier lab economics, cloud inference and margins across the stack.

• June 11th: Building AI-Native Engineering Teams with Ethos CTO Dan Mankowitz

The levers to drive down cost-per-token

Inference is now the dominant cost line in most production AI businesses, and the shape of that cost is changing faster than most operators are pricing in. The software techniques that defined the last eighteen months (quantisation, speculative decoding, prefill/decode disaggregation, smarter KV caching) are real and still being mined, but they are first-order improvements on a stack whose deeper constraints are physical: gigawatts of available power, lead times on Infiniband, the maturity of chip ecosystems, and the time between signing a Nvidia order and seeing tokens come out the other end. Operators who treat inference as a pure software optimisation problem are quietly losing ground to ones who are integrating downward into the data centre and energy stack.

The discussion this week with Erwan Menard, SVP Product from Crusoe, surfaced the bets Crusoe is making as an AI cloud.

1. Vertical integration

The four-layer stack (energy sourcing, data centre construction, GPU cloud, managed AI services) has to be owned end-to-end to be defensible by 2027. Today, customers don’t ask whether their inference provider built its own power, but by 2028, when the US is projected to face a double-digit gigawatt gap between data centre demand and grid supply, that question becomes the buying criteria.

Crusoe has publicly committed $1.5 billion in turbine purchases for 2027 deployment, built with Boom Supersonic, operating at temperatures roughly 20°C above standard GE or Siemens designs. Even if those turbines are not consumed by Crusoe’s own cloud, they can be resold at a higher price than purchased. The downside is bounded; the upside is preferential access to power when peers are waiting in the grid interconnect queue (currently four years in the US).

The tension is the 1990s telecoms parallel: massive infrastructure buildout against speculative demand led to a decade of overcapacity. The 1996-era fibre overbuild preceded the demand wave. The current GPU buildout is happening against an H100 cluster price index that is up 35% year-to-date, on hardware introduced in 2022. That is the inverse of an overcapacity signal.

2. The Pareto curve is the unit of analysis, not the model

The mental model Crusoe pushes customers toward is the throughput-versus-latency Pareto frontier: tokens per second per user (latency) on one axis, tokens per second per megawatt (fleet throughput) on the other. Every inference deployment is a point on that curve, and every GPU generation defines a new curve that sits above and to the right of the last one.

Customers graduate along the curve as the business matures. Early-stage customers cluster at the throughput end because they’re optimising GPU spend during proof-of-concept. Customers with human-facing products migrate towards latency as it starts to bind on user experience.

The complication is that the newest GPU generation always offers the best Pareto curve in principle but not always the best dollar economics in practice, because it’s earliest in its depreciation cycle. A B300 outperforms a Hopper on every axis, and the Hoppers are the workhorses of AI infrastructure globally, presenting a great economical fit to inference small models as an example. Crusoe routinely help their customers navigating these trade-offs.

3. Three product tiers, mapped to customer maturity

Crusoe’s serverless inference is structured as a three-step ladder rather than a single product:

1. Off-the-shelf open models on shared infrastructure, paid per token. DeepSeek, Qwen, Kimi K2, Minimax, GPT-OSS, Nemotron. Available through Crusoe’s own Intelligence Foundry or on OpenRouter and other brokerages. This is the proof-of-concept tier.

2. Bring-your-own open model on dedicated infrastructure, with throughput-optimised or latency-optimised “recipes” selectable by the customer. Self-serve, on-demand GPU-hour billing. Releasing in June.

3. Custom inference engine optimisation by a forward-deployed engineering team. Customers at this tier typically commit at the dollar level (tens of millions over twelve months) rather than at the GPU level, and revisit parameters quarterly as the workload shifts.

The same inference engine sits under all three tiers.

4. The KV cache as proprietary IP

Most of Crusoe’s inference engine is assembled from open-source components (vLLM, SGLang, others) with the value-add being curation: picking which open-source advances to integrate and when. The exception is the KV cache layer, which Crusoe has built in-house and calls “memory alloy”. The mechanism pools the memory available across all GPUs in a serving group and treats it as a unified address space. The payoff is on long-context workloads, where KV cache becomes the bottleneck before compute does. Crusoe published a white paper on the technique earlier this year.

This matters strategically because long-context coding workloads are now a significant fraction of the inference business, and the per-token economics on those workloads are dominated by cache behaviour rather than raw FLOP throughput.

The tokeniser Crusoe open-sourced at GTC, reportedly 40x faster than the Hugging Face baseline, is a related but distinct optimisation.

If the inference engine is proprietary, how does a customer compare outputs against vLLM or a different provider’s stack? Crusoe’s upcoming serverless fine-tuning service (launching end of June) will include an eval harness, and customers operating at billions of tokens per day already maintain their own eval infrastructure across multiple inference providers.

5. Supply chain tensions are plenty, not only on chips

SSD, memory, Infiniband networking are increasing lead times that exceed both GPU lead times. A neocloud’s effective time-to-market is determined by its supply commitments, made nearly a year before the corresponding revenue is recognised. This is the engineering rationale for Crusoe’s modular data centre approach: 20 megawatt sites assembled from ten 2-megawatt prefabricated units called Spark, manufactured in-house using designs inherited from the company’s bitcoin mining origins. These can be brought online in weeks rather than the year a gigawatt site requires, with equivalent per-megawatt economics for a given GPU SKU. This end-to-end industrial approach allows to smoothen supply chain tensions and deliver capacity programmatically and reliably. This modular strategy is complementing the gigawatt campuses Crusoe is known for: They developed the first Stargate campus in the US, which capacity will reach 2.1GW.

The first Spark deployment is a 16 megawatt campus in Nevada, co-located with a solar farm and a battery bank built from retired EV cells. The grid is present as backup, not primary supply. The first 2 Spark units are powering Crusoe Managed Inference service. This is the template Crusoe expects to scale through 2026 and 2027.

6. Inference-specific chips: when and which

Crusoe expects to deploy inference-specialised silicon in 2027 alongside continued GPU procurement. Crusoe will aim to serve 2 chips:

• LPU (Groq) will likely be one of them, primarily because the Nvidia acquisition gives it the financing and ecosystem support that other inference-specific architectures lack.

There are a few dimensions that matter. The first is current LLM inference performance, which is measurable today. The second is architectural adaptability to whatever post-transformer architecture might emerge, which is irreducibly speculative.

The financing point is worth flagging separately. Nvidia actively its chips entry to market: CoreWeave, Nebius, and Crusoe itself benefited from this. A challenger chip vendor that cannot offer similar support places the cost of ecosystem development on the neocloud, which is a meaningful drag on adoption regardless of per-token economics.

7. The last-mile network constraint

If inference moves to the edge in 20-megawatt increments, the constraint shifts from data centre supply to last-mile bandwidth. For example, a single user running five concurrent cloud-based coding agents pulls roughly a terabyte per month of inference traffic. Multiply by agent population and the network buildout starts to look undersized.

However, the last mile is part of the network that has been most consistently overbuilt, with 5G machine-to-machine traffic projections that never materialised. Edge inference deployments fit comfortably within existing last-mile capacity in most markets. The bottleneck, again, is the data centre side.

For those interested in learning more about Cruose, check out CEO Chase Lochmiller’s talk at Stanford which dives deep into the economics of the AI stack from power to compute.

• May 20th: Pydantic x Glyphic Engineering Night

• May 28th: Inference Stack Innovation with Crusoe

• June 4th: Forward Deployed Engineering with Anjor Kanekar

We recently hosted Ellen Moeller McCormack and Archie Moore from Clay’s London office for a discussion on GTM engineering. We covered a ton of ground and I’m excited to share the notes with you!

GTM is converging on engineering as a discipline, not a metaphor

The opening argument from Archie set the frame for the rest of the conversation. Product organisations have largely figured out how to ship quickly. Claude Code and similar tools have collapsed the cycle time for shipping features, but go-to-market has not kept pace. Most GTM stacks remain spaghetti: a sales engagement tool here, an enablement platform there, a CRM stitched in with brittle integrations.

The teams getting an edge are the ones treating GTM the way engineering treats product: sprint-based, iterative, with a roadmap of “plays” rather than a static playbook. Clay runs an internal Kanban board for GTM ideas, sourced from anyone in the business (co-founders to SDRs), with sprint cycles ranging from two days to several months depending on the play’s life cycle.

The role enabling this is the GTM engineer, closer in DNA to rev ops than to a seller but with a tighter pulse on what’s happening at the revenue front line. Ellen noted it’s currently among the fastest-growing roles being hired under the GTM umbrella. The point Archie made repeatedly: you don’t need to be a GTM engineer to do GTM engineering, but you do need to create the organisational space for it.

Every play has a half-life, and “GTM alpha” is the portfolio response

Borrowed from finance, GTM alpha is the framing Clay uses to describe permanent edge in the market. The claim is that any individual play degrades over time, whether competitors copy it, the market gets noisier, or the play’s underlying signal goes stale. Two examples Ellen offered:

• HubSpot’s inbound content marketing was a multi-year alpha generator, but the effectiveness curve has been declining for the last two years. Their response has been to start running outbound (something they’d never done before) using Clay.

• Cold outbound email, which worked at volume a decade ago with generic copy, is now near-dead in its undifferentiated form. The replacement isn’t to abandon outbound but to compose it differently: right person, right content, right time, with the time component increasingly driven by external signals.

The implication for founders is uncomfortable but useful. A successful play is not a permanent asset; it’s a depreciating one. The portfolio question (how many plays should be running simultaneously) got a practical answer from Archie. Clay runs roughly one new play per week , with some plays kept evergreen in the background (job changes was the example) and others run as time-limited sprints. For matrixed product lines, that count multiplies across business units.

The upstream constraint is unique data, not channel mechanics

One of the more analytically useful sections concerned what makes a play actually distinctive. Generic third-party data (firmographics, technographics from standard providers) produces generic outreach. The differentiated plays come from layering unique data sources:

• Internal data: closed-lost accounts, previously DQ’d opportunities, contacts who’ve changed jobs to new companies that might now be in-ICP.

• Composite third-party signals: A customer support platform found that fertility clinics over-indexed on their ICP combination of “large FAQ + high support ticket volume.” Another example was of a design platform where its best SDRs were scouring LinkedIn for posts that broke brand guidelines, screenshotting them to CMOs.

• Custom-built data points: A waste management provider used Google Maps imagery to detect dumpster colour outside commercial buildings (green = theirs, blue = competitor). A healthcare seller stopped using employee count as a hospital-size proxy and started scraping bed counts from hospital websites because research hospitals distorted the headline metric.

The pattern across these is that the data point itself encodes a piece of domain insight that competitors haven’t yet operationalised. The half-life can be short (these are not durable moats) but the cumulative effect of running a portfolio of these is what produces alpha.

Maturity is a three-stage curve from individual hacks to deployed infrastructure

Clay’s framing of how organisations grow into GTM engineering followed a three-level structure:

Level 1: individual pockets. Individuals across the company are already doing creative things, often with tools they pay for themselves. One attendee described building custom one-pagers in 15 minutes plus a sub-45-second Loom, and breaking into five enterprise accounts in ten days under outbound pressure from their manager. Another described sending physical co-branded SIM cards to fintech prospects. The prospect installs it, sees the seller’s brand on their phone, and the seller has a tangible artefact in front of the buyer.

The actionable point Ellen made for founders: most teams run product brainstorms but almost never run GTM brainstorms. The first move at Level 1 is harvesting, going to the best individual performers and asking what’s actually working, then asking the second-order question: what is it about this that’s working, and can we systematise it?

Level 2: standardisation. Once a play works at the individual level, the question becomes which parts can be lifted into infrastructure that runs across the team. The examples discussed:

• CRM hygiene: at high-velocity rep counts (30 to 40 meetings/week), manual CRM updates are a losing battle. Clay’s approach is to auto-populate fields from call transcripts but keep humans in the loop for stage progression, and route denials back to the agent builders with explanations, creating a feedback loop.

• Inbound speed: shortening form fields and using de-anonymisation plus routing logic to hit inbound leads within minutes rather than hours. The conversion drop-off in the first hour is steep enough that there’s no excuse for the latency anymore.

• Closed-lost analysis: most teams do this manually at end-of-quarter, by which point insights are stale. Automating transcript analysis on every lost deal, piping product gaps into engineering’s roadmap intake, and then closing the loop by re-engaging lost opportunities when the gap gets shipped.

• Competitive mention extraction: mining calls for competitor mentions and product gaps, then feeding the knowledge base that downstream agents draw on. The knowledge base improves automatically as more calls accumulate.

Level 3: GTM alpha. This is where the culture of shipping plays is ingrained and the infrastructure is mature enough to deploy them broadly. Notably, even at this level Clay does not run fully autonomous AI SDRs on outbound; humans still check outgoing emails. The genuinely automated workflows tend to be inward-facing: closed-lost analysis, QBR generation, knowledge base maintenance.

Two operational tensions that scale worse than founders expect

Two threads in the discussion are worth pulling out separately because they’re under-discussed and likely to matter more as teams scale.

Versioning and propagation of skills and agents. One attendee asked how a non-engineering team should handle skill versioning. When five people build their own variant of “call analysis,” whose version becomes canonical? Archie’s answer was honest: both happen, ideally sequentially. Encourage individual experimentation early because it preserves creativity, but move proven agents into a centralised library where updates propagate to every workflow that calls them. Notion as a shared knowledge base, plus a designated owner (a GTM engineer or ops lead) whose job is precisely to harvest and centralise, is how Clay handles it.

Org readiness outranks tooling as the binding constraint. This came up most sharply from an attendee at a fast-growing fintech with senior, high-performing reps who refuse to update CRM and use Granola instead of the company-standard call tool. The reps hit quota, the company is winning, and so the lever to enforce process discipline is weak. The room’s collective answer was that the time to invest in operational maturity is before growth decelerates. Public-market analogues are instructive here: waiting until growth slows means the market punishes you before you can course-correct. Pre-IPO investors increasingly discern between AI-native and non-AI-native operations, and GTM rigour is one of the measures.

Seven takeaways for founders

1. Run GTM brainstorms with the same cadence as product brainstorms. Your engineer may have your next outbound play. The cultural permission to experiment has to come from the founder.

2. Treat plays as a portfolio with deliberate cadence. Roughly one new play per week per person doing outbound, with a small number of evergreen plays running in the background. More than that and the cognitive load destroys signal.

3. Score leads on attributes, not on play attribution. Lead scoring at the account level (size, funding, location, fundamentals of fit) is the baseline infrastructure that lets you detect when a play is degrading. If scores drop from 8 to 10 down to 3 to 5, the play is telling you something.

4. Invest in the boring infrastructure early. CRM hygiene, transcript pipelines, knowledge base maintenance. None of these are alpha-generating in themselves, but they are the substrate on which alpha-generating plays run. Skip them and your creative plays sit on broken data.

5. Hire, or designate, someone whose job is harvesting and systematising. Without this role, individual creativity stays trapped at Level 1. The role doesn’t need to be called “GTM engineer,” but the function needs an owner.

6. Buying committees are shifting fast. ICP analyses run 18 months ago are likely already stale. The committee composition for tech purchases has been shifting toward broader executive involvement, and any “definitive” buyer map is a snapshot, not a fixed asset.

7. Be honest about where humans still belong in the loop. Closed-lost analysis and QBR generation are good candidates for high automation. Outbound emails to named accounts are not, yet. The distinction is whether the failure mode is recoverable.

The framing that probably matters most: in a world where product alpha is compressed by AI-driven shipping velocity, GTM alpha is where durable execution edge increasingly lives. For founders, the consequence is that GTM rigour is no longer a Series B problem to defer. It’s a discipline to build into the company culturally from day one.

Wedge is Clay's Startup GTM Accelerator. The two-week async program helps the top early-stage GTM leaders learn Clay as their GTME infrastructure, build workflows directly with our team, and earn thousands of credits as they progress and graduate. Learn more and apply here. Explore Clay's full Startup Program here.

• May 20th: Pydantic x Glyphic Engineering Night

• May 28th: Inference Stack Innovation with Crusoe

• June 4th: Forward Deployed Engineering with Anjor Kanekar

Two weeks ago, we hosted Eric Rodriguez (Harmonic), Brendan Regan and Dennj Osele (Latinum) for a discussion on foundation models for mathematics, formal verification and its applications.

Formal verification is the practice of expressing software programs and mathematical proofs in a language that a machine can check mechanically. Lean has become the most important proof assistant in this ecosystem, with Mathlib serving as its core knowledge base: a formalised library of mathematics now comprising roughly 2.25 million lines of human-curated code. Historically, however, the cost of formalisation has kept the field relatively narrow, concentrated in academic mathematics and high-assurance domains such as hardware, cryptography, and security-critical software.

That constraint is beginning to change. LLMs are making it cheaper to generate Lean code and proofs, reducing the human effort required to formalise mathematics or verify software. If the trend continues, formal verification could move from a specialised discipline to a more widely used layer for checking mathematical, software, and eventually AI-generated outputs.

Models can now generate Lean proofs at a scale and quality that human reviewers struggle to distinguish from human work, opening up a credible path to formalising large parts of mathematics, verifying production software, and producing AI systems whose outputs can be checked rather than merely trusted. The deeper bet is that mathematics is uniquely well-suited as a training environment for general reasoning — the compiler provides a perfect reward signal, which is the cleanest reinforcement-learning setup currently available.

On to the discussion.

1. The frontier has moved past the IMO

Olympiad-level mathematics is now treated as a solved training objective, with new postgraduate mathematics tests emerging, like First Proof. The active research problems are library construction and longer time-horizon research-level proofs, not individual theorem generation.

This changes how capability should be measured. A theorem that takes ~10 hours to prove against a mature library may take ~100 hours without it (Metr time-horizon framing). Library growth is the rate-limiter. Systems that amortise library construction, not just prove harder theorems, are what unlock the next step function.

2. New libraries emerging

LLM submissions are increasing pressure on Mathlib. The underlying constraint is maintenance cost. Lean does not guarantee backward compatibility, so each release requires Mathlib to be updated.

The risk is that Mathlib’s current model, which was designed for human-scale throughput. Parallel libraries may grow alongside it rather than feeding into it.

Multiple parallel AI-generated Lean libraries are emerging outside Mathlib, each aiming to be larger than Mathlib itself (but still nascent). There’s a risk that these will be mutually incompatible, with divergent formalisations of the same objects.

3. Latinum’s approach

Denny spoke about Latinum’s model: recursive latent reasoning. A large encoder ingests context, produces a reasoning step in latent space, feeds it back, and cycles until convergence; a decoder tokenises only the final output.

Lineage: JEPA-style world models, Universal Transformer.

Key claim: the recursion itself is trained, with latent states supervised at each step, permitting parallel internal chains that converge externally. Stated rationale: chain-of-thought burns scarce context window; internal recursion is unconstrained because nothing is printed.

4. Harmonic’s model

Eric spoke about hybrid tree search. Three points on a spectrum:

• Whole-proof generation — simple, no granular feedback, one-line breakage forces full regeneration

• Tree search — tactic-level policy model, parallel state exploration, heavy infrastructure burden (Lean execution at scale)

• Hybrid decomposition — break the problem, use whole-proof generation on pieces, assemble via high-level strategist

Why frontier LLMs don’t natively do tree search: base models are trained for full answers, not single tactics. Tree-search policy models are trained differently and have produced proofs not previously known to humanity (Erdős-class problems), so this is not regurgitation.

5. Action-space trade-off: fundamental, can’t be engineered away

Lean offers two obvious cheating mechanisms for an AI prover. sorry is a placeholder that lets the model skip a proof and still compile — equivalent to writing “trust me.” Custom axioms let the model assert something is true without proof. Both are legitimate Lean primitives with valid uses, but both are also the path of least resistance for a model that can’t actually prove what’s being asked.

Earlier prover systems were built so the model literally could not emit sorrys or introduce axioms — the only allowed operations were adding new content and proving it from existing primitives. This made cheating structurally impossible. Users rejected it: pure-additive systems cannot refactor existing proofs, fix incorrect user-provided definitions, or scaffold a proof structure with intentional placeholders. None of these are exotic workflows — they are how Lean is actually used.

Current approach used by Harmonic: open the action space, constrain harmful actions case-by-case. Recent literature is consistent: over-restriction degrades model performance analogously to constrained attention.

The hard part is that sorry is not purely a failure signal. Mathematicians structure large proofs by writing skeletons with deliberate sorry-marked gaps and filling them in later (the “Lean blueprint” workflow). Models doing this are working correctly. Models emitting sorry because they gave up are working incorrectly. Token-level the two are identical. The reward signal has to distinguish legitimate placeholder use from give-up behaviour without surface-level cues — a non-trivial supervision problem.

Read-across to agent infrastructure: the capability-vs-constraint trade-off is the same problem every agent company faces. Give the model file-write, shell access, or arbitrary API calls and it can do useful work — and also cause damage. Restrict the surface and the agent cannot do real work. Harness engineering — the discipline of designing this trade-off — is now a real engineering function. Formal maths is the cleanest version of the problem because cheating is well-defined; the same logic applies across general agent design.

6. Why maths? Two justifications, in tension

Generalisation bet. Humans good at Fermat-class problems tend to transfer to code and quant reasoning. Train models on maths, expect transfer. Counter: the causal direction may not hold for models — selection effects in humans don’t imply training effects in models. Risk case is a system that plateaus as a domain-specialist.

Data argument. Training data is the binding constraint. Maths plus a compiler is infinite, perfectly-labelled data — the compiler is 100% correct.

Best pushback: mathematical data is not infinite in the relevant sense. 1+1+1+... is trivially generable and useless. The hard problem is deriving which abstractions matter — which theorems are worth proving. Same failure mode as code-focused AGI approaches using compiler-pass or test-pass as reward: clean signal, no direction.

7. The underlying reason labs started with maths: RL signal quality

RL on formal maths works despite being a single-player game with no native win condition. The compiler is the reward signal — making formal maths among the cleanest RL environments available.

This explains the path-dependence of frontier lab strategy: they started where reward signals were cleanest (maths, reasoning), then migrated as signals degraded. The migration targets — Office, Excel, accounting — are domains where partial reward is recoverable (value reconciliation, keyword presence).

8. Mathematics occupies a privileged data position

Sharper framing of the data argument: maths has structure from which semantics can be inferred. Raw Python or JSON does not. A proof is recognisably a proof; a type signature encodes intent. Arbitrary code on the internet does not train the same way.

Lean 4 was designed as a programming language first and proof assistant second. In principle: translate conventional code to Lean, execute and verify simultaneously. The runtime supports full applications.

Commercial threads

Chip verification

Industry ratio: ~2–3 verification engineers per designer. Silicon errata economics justify this. Years of chip timelines are verification-bound.

Under-automated, and hardware description languages have small training corpora relative to mainstream languages — creating specialist-friendly data wedges. Self-play approaches (AlphaGo-analogous) are in commercial development but unsolved.

Software verification

Recent large rounds in math-to-software-verification reflect the thesis shift. The operative question is not whether every SaaS app needs formal verification, but whether verification becomes cheap enough that cost-benefit flips for mid-criticality systems. Current state: hire specialists, give them unlimited runway. Target state: run the verifier, price the failure mode.

Contract-first languages

Dafny is the reference case — contract first, implementation second. Never broke through because it required a mathematician and a software engineer. AI collapses this: model writes the contract, then the code; humans only review the contract. Code trends toward “the new assembly language.”

Destination is unclear — Dafny, extended Rust, Lean 4. Rust already moves most runtime bugs to compile time, so Python → Rust → contract-first is a plausible migration path.

Interconnection problem

Individually-verified components (CPU, router, crypto library) compose into systems with emergent vulnerabilities via the OS and network layer. Local verification is necessary, not sufficient. Concurrency proofs are harder than sequential proofs by another order. The gap between “verified component” and “verified system” is open.

• May 6th: AI-Native GTM with Clay

• May 20th: Pydantic x Glyphic Engineering Night

• May 28th: Inference Stack Innovation with Crusoe

• June 3rd: Forward Deployed Engineering with Anjor Kanekar

Intercom is arguably one of the most successful examples of a SaaS company becoming AI-native. They declared Code Red early after ChatGPT launched and went all in on Fin, their AI agent for customer support. Since then, the company has started publishing a blog charting their journey to become as AI-pilled as they come.

Brian Scanlan, Senior Principal Systems Engineer, went viral when he gave a peek into Intercom’s Claude Code tooling.

I had the chance to speak to Brian about how Intercom is changing hiring practices, instrumentation to measure the right metrics, consciously investing in Anthropic as a platform, org design, code quality and graduating from tokenmaxxing to seeing high quality PRs per engineer inflect. You can read some of the Intercom writing referred to in this interview here and here.

Talent and the interview process

You built a console where Claude can execute arbitrary Ruby against production data, and you mentioned the top-5 users of the read-only Rails production console aren’t engineers. They’re design managers, support engineers, PMs. How has that shift changed how you think about what you’re looking for in talent, and how has your interview process changed?

We’ve always looked for people who are flexible and product-oriented. Intercom is a product-oriented company, the founders are product people, and the name of our engineering role is product engineer. Until recently we even had a product component on our engineering interview. The same is true of designers and PMs: we want them close to reality, whether that’s users, code, or whatever else. There’s a great tweet from our head of product where he basically states that nothing else matters apart from code in production. I love that coming from a head of product, because you might expect them to be focused on plans, roadmaps, Gantt charts, and designs. Instead, what he cares about is how fast we’re getting real features and real code into production.

So this is more of a slight evolution than a reinvention. We’ve always been tech-agnostic. We’ve never just hired Ruby on Rails engineers, and we’ve been pretty liberal about hiring people from diverse backgrounds. I came from Amazon, for example. What we interview for is the ability to teach, learn, and change, the ability to be open-minded, more than five years of any specific stack or the aura of having worked at Google.

The tweaks we’ve made are around openness to AI. Most people showing up in our loops these days have seen what Intercom is doing and want to be part of it, so it’s not usually a problem. But we still get people from environments that have had zero AI exposure, or from more traditional backgrounds. Their enthusiasm and willingness to change is what matters. We look for openness rather than five years of cloud experience.

Looking forward, we expect design, product, and engineering roles to continue to converge. There’s still incredible value in depth across all these areas, but we now have product engineers doing full product management of features, designers shipping real code, and PMs shipping code as well. It’s not just box-checking. It’s about reframing those roles to look more like a builder role rather than a member-of-technical-staff role, rather than “I’m a designer and I hand stuff off to engineers.” We’re already well set up for this; the trajectory is just more and more towards everyone being able to do everything.

Measuring what matters in Claude Code adoption

You’ve instrumented every Claude Code lifecycle event with OpenTelemetry. This essentially treats Claude Code like a product you’re iterating on internally. What metrics do you actually look at to decide whether a skill is working, and how do you avoid the trap of measuring activity instead of impact, e.g. tokens spent, which can be a very poor proxy for value creation?

There’s no shame in going through a journey of adoption where, at the start, you’re just looking to force people to try things out, to token-max. The opportunity cost is so big, and the amount of change you’re trying to drive through people and the organization is so large, that it’s perfectly fine to have a slightly immature initial phase where everyone has to try this and use it as much as possible. Yes, you’ll waste a bit of time, but what you’re really getting is people becoming comfortable with the tools, figuring out where they work and where they don’t, and where you need to invest. You’re putting pressure on the system. If you’re too careful, or chase the perfect measurements too early, you end up learning lessons later than you should have.

One of the ways we’ve historically built at Intercom is to do the harder stuff early. In a lot of project plans, you load the start with easy work to manufacture the illusion of momentum, and the gnarliest, hardest parts get pushed to the end, which is also when you find out whether the thing actually works. We flip that. We want you to hit the hardest part first. We want you to be hungry for the learning of “does this truly solve the problem” rather than burying it.

So with token-maxing or output measurement, you have to first open people up to change. That means, “just go use it for everything, doesn’t matter how good it is.” You need to encourage that initial use: race people against each other, run a crappy leaderboard, gamify it. But you don’t confuse that with outcomes, which is what you ultimately care about. You go through a juvenile phase deliberately, because gamifying the trial is the easiest way to get people to test it across everything and figure out where it works and where it doesn’t.

Build vs. buy in an AI-native infrastructure stack

You’ve got 13 plugins, 100+ skills, and a JAMF-managed marketplace. That’s a lot of internal infrastructure to maintain. I’m sure you’ve been able to rip out some SaaS apps as a result. How do you make the build-vs-maintain tradeoff as a company?

Historically, we’ve had an engineering principle that we are technically conservative; we bias towards buying over building. We want people focused on shipping product rather than everything else, and we lean heavily on vendors rather than building expertise across the board.

That has changed somewhat. One of the biggest things to take off at Intercom, arguably bigger than Claude in software development, has been the adoption of Claude Code with a well-set-up suite of MCPs and access controls across the rest of the company. What people have ended up using it for is building their own sales dashboards, customer health dashboards, and highly granular customer-specific reporting, usually someone in sales or someone driving these things directly. What this has effectively replaced is business intelligence tooling and the likes of Tableau.

We didn’t deliberately set out to replace Tableau. We’ve just had a kind of bottoms-up product-market fit for this way of working, and we couldn’t go back to a hosted platform or a more traditional system. Once people have great, well-controlled data, good guardrails, and good places to host reports or small applications, the need for a centralized purchase of something like Tableau diminishes.

So I wouldn’t say we’re consciously trying to rip out big pieces of software, but the need for smaller purchases (role-specific workflows, reporting tools, that kind of thing) is shrinking. We’re probably purchasing far fewer of those, and we’re likely on a path to replacing things like Tableau over time, but very much bottoms-up. We’re not doing top-down “let’s vibe-code a Salesforce replacement today.”

The harness around Claude Code, and the role of the human engineer

The Claude Code hooks you’ve imposed to enforce adherence to your PR workflow and approved tools are great examples of how you’re building your own harness around Claude Code to get maximum value. What other best practices have you had to follow to make Claude Code more reliable over time, especially as we enter the age of background agents? And how do you see the role of human engineers evolving to best enable coding agents?

We’ve had to strengthen the guardrails substantially and provide a lot of guidance, and we’ve socialized this problem heavily. We tell people: we are all part of the flywheel, and we expect everyone to be part of it. I’ve written an engineering AI principle that says all technical work is becoming agent-first. In practice, that means you should be using an agent for all technical work, and when it doesn’t get things right perfectly every single time, your job is to notice and do something about it.

What that has meant in practice: more linters everywhere, making patterns in our codebase more standardized, and writing down explicit guidance for Claude Code along the lines of “if you’re going to do X, use these paths.” We have the headwind of being a 15-year-old SaaS with all sorts of legacy in our codebase, patterns we’ve long since moved past but that still exist. We have to explicitly tell Claude about the modern way. What we do with humans is essentially the same: we onboard people by showing them how to get stuff done, what good changes look like, and which parts of the code to ignore as the old way. The pairing and unblocking process for Claude Code mirrors human onboarding.

Beyond socialization and guardrails, we’ve found that skills, currently the main unit of execution, need to be extremely small. Small, testable, provably good. Skills need to do the full job, because it’s very easy to build a skill that looks like it’s doing a good job but isn’t, and a bad skill is almost indistinguishable from a great one at first glance. The hard part is really pushing on what great looks like: fast, accurate, gets it right first time, tested against all the use cases and edge cases, something you can stand over.

We have a continual flywheel built into pretty much every skill. If something happens that suggests the skill should be updated, the instruction is to update the skill. Skills are self-improving as they’re used. It’s easy to go expansive and build opinionated, monolithic skills that try to do a hundred different things, but I think it’s similar to writing code: you want small, testable units that do one thing well and are understandable.

The quality bar has to be extremely high, because the worst case is you spend a lot of time building automation, everyone is very enthusiastic, but the skills aren’t reliable, and then people stop using them, ignore them, bypass them. People just want to get their job done. So we run a small number of extremely high-quality skills rather than a large volume trying to automate complex, sprawling work.

Model lock-in and platform optionality

In terms of building infrastructure around Claude Code, are you building in a way that allows you to port your harness onto Codex or other providers easily, or are you developing switching costs as you invest more into enablement of Claude models specifically?

We knew this when we decided to go all-in on Claude Code. It was a slightly easier decision in December because Opus was ahead, but we knew the frontier wouldn’t always be in one place. Maybe Opus isn’t the best for Ruby, maybe not the best for security, maybe Codex overtakes it on cost. We accepted that.

But this is a platform play. We don’t choose multiple cloud providers and say “Google has the best blob storage, Oracle the best MySQL hosting.” You get the most benefit from a single platform and the compounding benefits of a system that works together. Vibes-wise, Opus may not be the best right now, and you could argue benchmarks back and forth, but I’m not too worried, because we get more benefit from being on one platform and being able to prove we can reproduce work at extremely high quality.

If Anthropic shut down tomorrow, I don’t think it would be a gigantic task to move to Codex, or even to roll our own to a degree. It’s likely a single Codex session away. We do already use Codex in our environment, not as a general-purpose agent but for things like code review and reviewing the output of another agent. Using two different models for meta-analysis is a perfectly good approach to gaining more confidence in code quality.

So we’re being pragmatic, but we put a lot of weight on the fact that it’s hard enough to get one system to work well. It’s not worth trying to get everything working at the level of quality and repeatability we want across multiple providers. That doesn’t mean we won’t change providers in the future. Capabilities are converging, and the providers are all moving towards the same shapes. I’m not too worried about lock-in, but I’d be worried if we were trying to be agnostic for the sake of it. It’s like multi-cloud: you need really strong reasons to go multi-cloud, because you miss out on the benefits of going deep on one.

Org design in an AI-native company

How has your org chart changed in the last 2 years, in light of how many AI-native companies are operating as very flat structures?

In the last three years, yes, a lot. We went through a phase, like many other companies, of fewer middle managers, a higher ratio of engineers to managers, and updated expectations of managers to be more hands-on, more leading rather than just facilitating. That was happening in the post-ZIRP era at Intercom.

Then, through the building of Fin, Intercom went through dramatic change. We put 80% of R&D on Fin, moved away from our traditional help desk (we’ve since invested back in it, which is great) but we made very dramatic decisions about shifting people off the help desk into effectively greenfield areas. We dropped our tried-and-tested product team format and moved to what we internally called experience teams, more like factory project teams, way more execution-oriented, none of the baggage. We staffed them almost exclusively with tenured people who knew how to get things done inside Intercom, leaving traditional onboarding to other teams. Those have been really successful, both for focus and because it’s the right model when you’re building with AI: you need to be able to remove baggage, get people out of the way, give them a mission, and let them go.

We’re not as advanced as Anthropic, where there’s more of a research-lab flavor. We’re still a product company that wants to ship features on the roadmap. Sometimes we do experimental work, sometimes we don’t know if a feature is buildable, but we haven’t fully melted into a managerless setup.

The role of planning has changed significantly, though. We’re not doing big roadmaps anywhere now; you can barely plan six weeks out. We used to set high-level roadmap goals every three or six months and then figure out what to build in six-week chunks. Now we set goals every six weeks and figure out what to build, and that can be replanned within days. So planning cycles have shifted dramatically. The teams still exist, but the format, expectations, and scope of what people work on have changed significantly. We just haven’t completely broken up the org.

Measuring code quality

In your recent write-up reflecting on how far you’ve come, you mentioned using static analysis and various heuristics to measure code quality. Can you elaborate on that?

We’ve been working with a group at Stanford doing research on the impact of AI on engineering. We gave them access to our GitHub and they’ve been pulling data. It’s something we care about. We’ve always had a strong testing culture and well-understood patterns. We write code in well-understood monolith applications, so we know how to get most things done. I worry more about software architecture than the code itself.

We initially saw code quality start to decline, despite having more linters written this year than ever before and more tests written than ever before. The fundamentals had improved. We’d raised the bar on multiple quality metrics by putting AI on those problems and getting more repeatable software production by guiding Claude. But the Stanford research did indicate that quality was slipping in a few directions: greater complexity, some rework.

Then something turned a corner, and this was only in the last six weeks or so, when we happened to look back and check in. To a degree we’d already accepted that doubling engineering throughput might come with a small dip in quality; we wanted the acceleration. But today, code quality is increasing every single day. The overall quality of the codebase is going up. That’s a testament to the detailed work we’ve been doing on inspecting output quality, but also to things like our automatic approval process.

The automatic approval process basically says: if you produce a pull request that’s what you probably should have been creating anyway (small, using feature flags, with code observability, single-purpose, not in dangerous code paths) it gets auto-approved. People are bending their work towards the path of least friction, which is exactly where we want them to go. We use an LLM judge with all of our domain knowledge about what a great Intercom pull request looks like, and we’re now producing more of these than ever.

All those years of telling people what great looks like (the blog posts, the code quality books) that was great. But what has actually transformed code production is giving people the incentive to do the right thing through automatic approvals.

The unsolved problem I worry about is that we may be moving so fast that software architecture drifts into incoherency; you end up with something hard to move, manipulate, or reason about. I’m hedging on the assumption that we’ll soon be able to use agents to identify and solve those drift areas. But honestly, it’s nice to be worrying about higher-level issues rather than whether the code makes sense, or whether we’re producing something below what I’d expect from a senior engineer. That’s a nice problem to have.

The fully loaded cost of a PR

You recently made the point that enterprises thinking narrowly about token spend are underestimating the salary cost per PR. How would you advise other companies to think about the fully loaded cost of a PR?

Think about fixed and variable costs, and the cost of the work being done. There’s a strong parallel to what we saw with Fin. When we launched Fin at $0.99 per resolution, we got pushback and claims from companies that this wasn’t much cheaper than what they could achieve with their own customer support team. But when we measured the all-in cost of a conversation in our own business, accounting for software licenses, management overhead, office space, all the real costs, it was over $60. And that wasn’t even maxed out.

It’s hard to think about this because people don’t often do these full-cost comparisons. But the economics work out so strongly in favor of whatever can produce the highest-quality output without human intervention. If you have a CS agent that resolves at 70% and you’re competing against one at 55%, given the all-in cost of human resolution, the 55% vendor probably can’t give it away for free.

The same is true in our engineering environment. Remove as much human work as possible so you can focus on higher-quality work. People need to compare the throughput, quality, ambition, and lack of rework against the new state with token spend factored in. Token spend matters; we do worry about it, mostly to make sure we’re not being sloppy and to find inefficiency. But it’s incredible: we have an awful and ever-growing bill, and yet the cost per change is dropping. As a business, that’s exactly what you want. A black box you put money into and get features out the other side. We’re getting more features out per dollar, with no signs of that changing. We’re going to keep that flywheel turning so we can put in even more money and get even greater multiples of features out the other side.

Brian is hosting a meetup on Wednesday in London, join here!

• May 1st: basedcollective x unicorn mafia: demo night

• May 6th: AI-Native GTM with Clay

Last week, we hosted Alec Barber from OpenAI for a discussion on Evals, building on the first discussion we ran with Wulfie Bain last August.

The central thesis: build your own eval harness

Codex and Claude Code are powerful enough that you should build your eval harness yourself, tightly coupled to your AI harness, rather than adopting a generic evals platform.

The primitives problem. Every evals platform Alec has worked on has struggled to define a base “test case” that generalises across single-turn, multi-agent, and decision-tree architectures. His old startup built around a single-turn schema; users immediately wanted multi-agent, and the abstraction didn’t extend.

Where platforms do make sense. Observability tools (Langfuse etc.) are worth using. Don’t reinvent Grafana. Hyper-focused platforms (e.g. coding-agent-specific) can deliver real value because the primitive problem is bounded. And a platform that genuinely closes the loop end-to-end would be an exception.

The architectural implication. Design your AI harness for testability from day one: decomposable, inspectable, unit-test ready. Too many teams build scrappily and realise later they want evals. Alec’s recommendation: feed good eval-writing skills to Codex or Claude Code and have the agent build a bespoke harness.

Defining what a harness is. The harness is the layer around the model, e.g. the CLI for Codex or Claude Code are concrete examples. The CLI is the harness sitting on top of the model. Another example in insurance: documents go in, the system makes 20 sophisticated calls, applies regex at various steps, mixes deterministic and AI components, and eventually produces a pricing judgment. That whole system is the harness. The inference step is just one implementation detail inside it.

Evaluating the evals

Entropy as a signal. Run a single test case 10 times against the same model and grader. 10/10 pass or fail = low entropy, good signal. 5/5 split = high entropy, meaning the grader or test case is ambiguous. Costs 10x per case but the diagnostic value is significant.

Log-prob confidence scoring. An attendee spoke about a vertical AI product that uses token-level probabilities from the OpenAI Responses API to compute a heuristic confidence score. Low-confidence outputs route to annotators who improve the dataset on the fly. The confidence-vs-performance curve isn’t perfect but is statistically validated.

Dataset hygiene

A useful split:

• Regression set. Stable, broad coverage. Every change is tested against this.

• Iteration set. Small, focused on a current failure mode.

Fixes migrate from iteration into regression. Critically, prune the regression set over time: when a new model generation drops, some cases become trivially easy and just waste compute.

Binary benchmarks and saturation

Trajectory detail gets hidden in one number. SWE-bench is binary (solved or not), masking partial correctness. One approach is using LLM-as-judge to score each step of a trajectory, aggregate, and correlate with the final outcome.

Saturation. There’s a pattern where you can spend months building an eval for it to be saturated in weeks. Causes include genuine capability gains, contamination, and reward hacking.

The consensus was that the next frontier is production and real-world use, though this blurs the line between scientific benchmark and lab demo. Alec floated an economics analogy: maybe evals need to become an interpretive discipline, educated judgments from people who’ve looked at a lot of data.

Benchmark scores in SWE bench correlate heavily with other domains, raising the question of why you’d run 20,000 benchmarks when three capture the signal.

The domain-expert bottleneck

Building good evals for a domain (legal, aerospace, insurance) requires someone who is simultaneously:

• A software engineer (to build the infrastructure)

• A domain expert (to know what “good” means)

• A product/design thinker (to surface the right UX for evaluation)

These unicorns are rare.

One participant working in vertical AI found that domain experts kept saying “this is wrong” without articulating why. He built a small vibe-coded UI that mirrored the platform lawyers already used, letting them highlight right and wrong passages in their native context. The UI extracts tacit judgment in a form the eval pipeline can use.

Alec strongly endorsed this. The answer is bespoke domain UX — not generic eval platforms. If you’re building for legal, your eval UI should look like what lawyers already work in. This makes the problem a design problem as much as engineering.

Enterprise adoption and liability

Many enterprises still make eval decisions on vibes. Adoption was framed as friction-to-set-up vs. value-delivered. In regulated industries, value is high enough (cost of a legal or insurance error is enormous) that adoption is better. Compliance departments effectively force evals as audit artefacts.

Practical recommendations for founders

1. Design your AI harness for testability from day one. Decomposable, unit-testable, each component inspectable. Don’t build a blob and retrofit evals.

2. Build your eval harness yourself using Codex or Claude Code. The coupling to your AI harness is too tight to delegate to a generic platform.

3. Use observability and dashboards off the shelf. Langfuse, Grafana — don’t reinvent these.

4. Find good eval-writing skills online (he named Hamel Husain) and feed them to the agent as context.

5. Invest in the domain-expert UX. Build bespoke interfaces that mirror how experts already work, so you can extract their tacit judgment without training them on eval frameworks.

6. Maintain the regression/iteration split and prune stale tests over time.

7. Use entropy diagnostics to identify test cases where your grader is ambiguous.

• April 1st: Tool Use: A breakfast discussion

• April 22nd: Evals with OpenAI

• April 23rd: Mathematical Superintelligence with Harmonic

Subscribe now

A range of agent labs (Cursor, Intercom, Cognition and Decagon) released frontier vertical models in recent months, manifesting one of the many plausible end states for app-layer companies: vertically integrate through model training to reduce dependence on the big lans and protect margins.

The direction of travel was clear once companies like Thinking Machine Labs started abstracting infra primitives, as we covered last October:

Fine-tuning and RL are being productised and abstracted for agent labs to capitalise on their data and distribution advantages.

Just recently Thinking Machine Labs’ Tinker was the latest abstraction for fine-tuning, whilst OpenPipe (acquired by CoreWeave) announced a serverless RL product.

These abstractions pave the way for agent labs to start as API consumers of model labs → capture traces/evals → train narrow models (embeddings, autocomplete, router, policy) before attempting larger training runs with open-source models.

More infra providers have set out to catalyse this market - here’s how Prime Intellect described the vision for their recently released model training offering:

If every company had the same access to frontier training infrastructure, the collective creativity of the market would unlock far more breakthroughs.

We’re already starting to see enterprises and application-layer AI startups realize this. Cursor is beginning to post-train their own models optimized directly for Cursor itself as the RL environment, to gain more sovereignty over their product stack.

We want more application-layer companies entering the training game for every vertical of the economy.

There are two camps at the moment when it comes to how agent labs will focus their technical resources.

One argues that the examples set by coding and customer support will be followed by market leaders in other verticals, with research effort focused on training and updating model weights.

The second camp rails against any training, instead focusing research talent on agent engineering (e.g. context management, tool use, long-horizon tasks and everything else that makes a model production-grade for a task) or building the best ‘harness’.

It depends on the workload.

Workload-Harness Fit

Workloads or tasks vary by volume, value, verification properties and horizon, among other dimensions.

Volume — How many times per day/week does this task execute? This determines both the economic incentive to reduce per-query cost and the rate at which your data flywheel generates training signal. Intercom's 2M weekly conversations sit at one extreme; a boutique consulting firm's quarterly strategic analyses sit at the other.

Value per execution — What's the economic impact of each individual task completion? A customer service deflection might save $3-5. A correct medical diagnosis or a successful code deployment to production might be worth thousands. This determines how much you can invest in model quality per query and how much tolerance there is for error.

Verification Properties - Domains differ fundamentally in how verifiable their reward signals are. Sean Cai outlined three verification properties: veracity (how confident are you the signal is correct?), proliferation (how widely is the signal tracked and available?), and asymmetry (how rare and expensive is the expertise needed to judge correctness?).

Time horizon - How many sequential decisions, tool interactions, and context switches does the task require? A single-turn code completion is low horizon. A multi-file refactoring session with debugging is medium. A multi-week scientific experiment design is extremely high horizon. Longer horizons make both reward attribution and training rollout generation exponentially harder.

Just by looking at workloads through these dimensions, we can start to appreciate how agent labs will focus their research efforts.

• Cursor and Cognition’s workloads are high volume, high value (especially as enterprise penetration increases), moderate on verification, and increasingly long time horizon with cloud/background agents.

  • Given these qualities, pre and post-training with sophisticated multi-dimensional reward engineering was justified. The moderate verification means you need to invest heavily in constructing composite reward signals (correctness + style + efficiency + behavioural penalties). The medium horizons require infrastructure for long rollouts and self-summarisation. LoRA or more efficient forms of fine-tuning is probably insufficient here — Cursor explicitly does full-parameter updates because the bar is just that high when you’re trying to raise the ceiling of software engineering, not the floor.

• Intercom and Decagon's workloads are high volume, low to moderate value, clean verification, short to medium horizon. Each execution is low value individually but the aggregate volume is massive, creating strong economic incentive to reduce per-query cost. Verification is clean — the ticket was resolved or it wasn't. Horizons are short — typically single-turn or a few exchanges. The data flywheel spins fast because you get millions of labelled outcomes per month.

  • The volume justifies the training investment in a full post-training pipeline on a strong base open-source model through inference cost savings alone, let alone the performance improvements that Decagon and Intercom are showing. The clean verification signal makes RL tractable. The short horizons mean rollouts are cheap to generate. This is where the Intercom/Apex playbook works best, and where even LoRA-scale fine-tuning on platforms like Prime Intellect could deliver meaningful gains for smaller players. It’s worth noting Intercom was early to recruit a research team of scientists, ahead of companies like Decagon and Sierra.

• Harvey and Legora’s workloads are moderate volume, high value, moderate on verification, medium horizon. Volume is meaningful but not massive (thousands rather than millions of executions per week). Value per execution is high. Verification is moderate — an expert can evaluate quality, but it's expensive and slow. Horizons involve multi-step reasoning but typically within a bounded scope.

  • The jury is out on what approach is best. Harvey allegedly attempted training runs in the past, whilst Legora has stayed clear of any training and fully focused on building the best harness for Anthropic’s models.

The taxonomy of workloads governs which end markets justify training versus agent engineering. But knowing where you sit on the spectrum is only part of the evaluation, the other half is what it actually costs to execute. Cursor's Composer 2 technical report is the first detailed look at the infrastructure overhead required to operate at the rightmost end of this spectrum. .

The Training Imperative

The training pipeline

Composer 2’s development follows a two-stage process. The first stage of continued pretraining took an existing open-weight model (Kimi K2.5, a 1 trillion parameter model from Moonshot AI) and trained it further on a massive code-dominated dataset, progressively extending its ability to process longer sequences of code. This stage builds the model’s foundational understanding of programming languages, APIs, and software patterns. The second stage of reinforcement learning is where the model learns how to work as a developer. The model is dropped into realistic coding environments, given tasks derived from actual developer workflows (feature iteration, debugging, refactoring, code review, documentation), and scored on the quality of its end-to-end solutions. Cursor demonstrated empirically that the first stage reliably predicts the success of the second: lower pretraining loss consistently translated to higher RL reward, justifying the investment in pre-taining.

The infrastructure overhead

Composer 2’s training run spanned three GPU regions and four CPU regions globally. The RL phase alone required hundreds of thousands of isolated virtual machines (Firecracker VMs on Cursor’s internal compute platform, Anyrun) to simulate realistic coding environments, each capable of running a full development setup including a browser. These environments needed to be spun up at a rate exceeding 500 pods per second to keep pace with the bursty nature of RL training workloads. Cursor partnered with Fireworks AI for inference during training, running distributed clusters across the US and Europe. The team also wrote custom GPU kernels targeting NVIDIA’s latest Blackwell hardware, including a modified low-precision number format that they found necessary to prevent training instability. The contributor list runs to approximately 50 people! This resembles a global ML infrastructure operation.

Reward engineering and behavioural shaping

Several of Cursor’s most consequential design choices concern how the model is rewarded during RL training, and these reveal the depth of product-informed thinking embedded in the pipeline. Beyond the primary correctness signal, Cursor applies a nonlinear length penalty that penalises the model more steeply for unnecessary effort on simple tasks while tolerating extended reasoning on complex ones, incentivising the model to be fast when it can and thorough when it must. They layer in auxiliary rewards for coding style, communication quality, and product-specific behaviours, including explicit penalties for patterns like creating TODO items and leaving them unfinished. The team actively monitors for emergent problematic behaviours during training and introduces corrective rewards reactively — for instance, when the model began leaving chains-of-thought in code comments or collapsed to using only the terminal tool. The self-summarisation technique, inherited from Composer 1.5, allows the model to chain multiple generations with intermediate summaries within a single training rollout, with the final reward flowing back to upweight good summaries and downweight lossy ones.

Benchmarking philosophy: why CursorBench matters more than public scores Existing public benchmarks are increasingly unreliable indicators of real-world coding agent quality, with four structural problems: domain mismatch (SWE-bench focuses narrowly on bug-fixing), prompt over-specification (public benchmarks assume unnaturally explicit instructions), data contamination (OpenAI suspended SWE-bench Verified reporting after finding models could generate solutions from memory), and narrow evaluation scope (only functional correctness, ignoring code quality, latency, cost, and interactive behaviour). Cursor’s response is CursorBench, an internal evaluation suite built from real coding sessions of their engineering team. The structural contrast is stark: CursorBench tasks require a median of 181 lines of code changes versus 7–10 for SWE-bench, while task prompts are significantly shorter and more ambiguous — mirroring the underspecified nature of real developer requests. The benchmark is continuously refreshed as developer workflows evolve, with each iteration increasing in complexity.

Despite the enormous effort, training Composer 2 was imperative for Cursor.

Developers will pick the best harness for their workloads and Composer 2, a SOTA model at a fraction of the cost, has a chance at winning back both old and new workloads.

Short AGI?

There’s a real risk of obsolescence from large models, but that logic is not too different to saying two companies (or one, if you ask most people) will capture all of the value created by AI in the coming decades.

John Collison asked Bret Taylor half-jokingly whether Sierra is itself premised on a world without AGI:

John: Now we get to it, which is you described building stuff that you know you’re going to throw away because the model capabilities will get there and you’re occasionally... They are developing capabilities that you developed yourself. Isn’t Sierra itself “short AGI?” Sorry, I said I couldn’t resist.

Bret: The short answer is I don’t know. I mean, the fog of war in the software industry is pretty thick right now. I really believe in the applied AI market, though. I think most companies don’t want to buy models or buy software. They want to buy solutions to their problem.

I think there’s so much nuance in how these companies align themselves with different departments at these companies, solve their very unique problems in very specific ways. That is a mix of product, not technology, but product, go-to-market. It’s an ecosystem around it. I think a lot of that still exists because I’m not sure coding the software was necessarily the hard part.

I think the reason for it is most business users want actual solutions to their problems, and they want a company that serves their unique problems in a very specific and bespoke way. I actually am extremely bullish on applied AI. I actually think we could accelerate. I’ll make one statement, which is I think if we paused model development, we’d still have trillions of dollars of economic value.

I think not only am I somewhat skeptical that there will only be two companies in the world, I actually think one of the main things impeding adoption of AI is the lack of existence of all those other companies.

The models are getting better faster than any of us could have ever imagined, with the leaked details of Anthropic’s Mythos suggesting we’re about to experience another step-change improvement in capabilities.

But focus matters, as OpenAI has now admitted.

Startups will create enduring value and technical differentiation in some shape or form will play a role.

Sivesh and I recently hosted a roundtable on the future of software engineering with Anthropic’s Ash Prabaker and we were joined by engineering leaders from Stripe, NVIDIA, Microsoft, Google DeepMind, xAI, Apple, Scale AI, as well as the legend Peter Steinberger of OpenClaw/OpenAI.

Origins of Claude Code

The session opened with a retelling of the Claude Code origin story, much of which has been covered in public interviews. It began as a simple terminal UI in late 2024, was rough at first, and was built against a guiding principle of designing for where models would be in six to twelve months rather than where they were that day. Adoption was organic — an IC-driven project that scaled through demonstrated value rather than mandate.

The Recursive Improvement Thesis

A major thread throughout the discussion was “closed-loop” development. One participant described a setup at their company where bug reports are automatically triaged by an agent, bucketed by severity, checked against an eval set, and then a fix PR is opened — much of it running with minimal human touch. The room broadly agreed that this kind of loop is where compounding gains actually come from: better coding tools improve the models, better models improve the coding tools. Several people noted their companies are prioritizing coding specifically because of this dynamic.

How Workflows Are Changing

Participants compared notes on what’s shifting in their engineering practice:

1. Test-first has become the default. Multiple people said they now define test cases first and let the agent build against them — described as the only sane way to handle the volume of PRs being generated.

2. Two tiers of evals. One participant outlined their team’s approach: regression evals that must stay at 100% and run on every PR, plus frontier evals for new capabilities. Others in the room recognized the pattern.

3. Don’t mandate adoption. There was strong consensus here. One attendee described using competitions, hackathons, and casual incentives instead of top-down requirements — arguing that forced usage breeds resentment, whereas letting people see early adopters’ results drives proliferation naturally.

4. Code review is in flux. One participant admitted that human reviewers at their company often just click approve within minutes because the AI review layer has gotten good enough. When pushed on where this ends up, they acknowledged the mandatory-human-review model will eventually become inefficient — and suggested they may already be past that point for some repos. This landed with a mix of recognition and discomfort around the table.

5. Comments are back. A cultural reversal several people found amusing: engineers initially hated the verbose comments agents generated, but the consensus is now swinging toward leaving them in, because the next agent session finds them useful. One person put it as “we’re writing code for AI readability as much as human readability now.”

6. Life in the terminal. One participant described their personal workflow as: plan, verify the plan, implement via the agent, move on — without reading generated code line by line. This prompted some debate about when that’s safe and when it isn’t.

What Still Gets Scrutiny

Not all code is treated the same. Participants generally agreed that anything involving destructive actions (data loss, permission escalation) or core infrastructure deserves higher human review, while internal prototypes don’t need the same bar as public-facing code. Where exactly to draw the line varied by company.

The Bottleneck: Long-Horizon Tasks

The room converged on long-horizon tasks as the real frontier problem. One participant noted that product engineering has started to go exponential for them, but closing the loop on more complex research workflows isn’t there yet. The open questions everyone shared: what do you actually assign an agent for a four- or five-hour run? How do you observe it? How do you keep a human in the loop without babysitting? Nobody had a clean answer.

Infrastructure and Sandboxing

Discussion of how the industry has swung on sandboxing — first toward it for safety, then away from it for convenience, now back toward it with more nuance (remote coding agents, sandbox-per-session). The practical pain points people raised were compute for long-running sessions, permissioning, and enterprise deployment.

Observability and On-Call

One participant described early-stage internal prototypes where agents with access to logs, source control, and chat systems handle incident triage and debugging — reducing the on-call burden even though the systems aren’t production-grade yet. A side effect several people found interesting: engineers without infra backgrounds can now contribute to infra work because agents fill the knowledge gaps.

Context Management

Someone asked how you manage context at scale when thousands of people are changing things every minute. The honest answer from the room was that nobody has this figured out. One participant admitted their approach is basically unstructured — ad hoc chat threads that agents get MCP access to read, plus a strong writing culture but no formal documentation process.

A study was mentioned suggesting that pre-loaded markdown context files can sometimes underperform versus letting agents traverse the codebase from first principles. The counter offered was that this probably reflects stale or agent-generated context. The takeaway people seemed to agree on: human-authored context files help, agent-authored or stale ones can actively hurt. Humans have to supply the insight.

Hiring

When hiring came up, the most striking claim was that the trait one participant now screens hardest for isn’t raw engineering skill — it’s willingness to experiment constantly at the bleeding edge. Their best performers are the ones who understand model limits deeply enough to know when to trust the output and when to intervene. Another attendee noted their core infrastructure teams have stayed lean because AI-assisted cross-pollination lets product engineers contribute outside their usual domain.

SaaS Under Pressure

This got lively.

Participants traded stories about which tool categories they’ve replaced internally:

• Incident management — one person said their team ripped out their vendor because it was too complicated for how people actually worked.

• Auth layers — one participant claimed to have migrated auth systems several times in six months, each migration taking hours not weeks.

• Project tracking — someone is building custom UIs on top of their coding agent for managing engineering work, and floated that this whole category might be next.

• Internal micro-tools — link shorteners and similar utilities were the easy wins several people mentioned.

The pattern everyone noticed: it’s all developer tooling so far, because that’s where engineers have agency and speed. Business-facing software (CRMs, etc.) is stickier. One view was that incumbent business tools survive not because they’re good but because nobody has shipped a compelling AI-native replacement — just incremental add-ins.

A counterpoint from the room: the opportunity-cost argument (”we should focus on what we’re best at”) may always hold, which means labs might never prioritise building SaaS replacements over improving models.

The “Everything Is an Option” Problem

A startup founder in the room raised the flip side: because AI makes everything feasible, prioritisation is harder, not easier. Six months ago, rebuilding a tool internally was obviously not worth it. Now it takes a night. Teams get overloaded by the sheer volume of things they could do. Nobody had a great answer beyond defining clear swim lanes and giving individuals ownership of mini-companies inside the org.

Code Quality

When someone asked about code quality standards, the response was that the definition is shifting. “Good code” used to mean human-centric things — simple, easy to maintain, easy to contribute to. Now it has to account for AI readability too. The practical view from the room: strong regression evals and test-first discipline matter more than clean-code aesthetics.

Design Taste and Slop

“The purple gradient vibe” got a laugh — everyone recognized the AI-generated-UI aesthetic. The catch-22 someone identified: if you update a model’s taste profile, everyone uses it, and the new aesthetic just becomes the next generation of slop. Someone also noted that some models actively steer users toward particular frameworks, which functions as a form of lock-in.

Convergence Risk

One attendee raised a concern that everyone coding with the same models making the same suggestions will collapse the industry onto the same tools and patterns. The pushback was that this was a bigger risk with earlier model generations, which were much stronger at popular web stacks than at legacy or niche languages — and that gap is closing. Code modernisation of legacy systems came up as an area improving fast.

Background Agents

General agreement that the direction of travel is toward asynchronous background agents — remote sandboxes, monitorable from a phone, persisting across hours or days. One person noted that multi-hour autonomous runs are only recently becoming routine for them, having been experimental until not long ago.

Model vs. Harness

Asked how much recent improvement is model weights versus harness, one participant’s view was that both matter but on different cadences — big leaps come from model steps, and the harness philosophy should be “get out of the way of the model.” They described a stripped-down prototype — basically a current-gen model with a system prompt and bash access — that performs surprisingly well, which wouldn’t have worked a few generations ago.

Regulated Industries

Someone from a fintech background asked about regulated deployment. The room’s read: the most successful AI startups in regulated industries (legal tech was the example) are still fundamentally human-in-the-loop chat-with-document products. Nobody has made the jump to autonomous agents in regulated workflows. The bar is asymmetric — analogous to self-driving cars, where AI has to be dramatically better than a human to be accepted. Better explainability and structured audit trails were floated as the unlock.

Orchestrating Multiple Agents

Refreshingly low-tech answer: git worktrees and ten terminal tabs. More sophisticated orchestration is being built by third parties, but nobody in the room claimed to have it solved.

The Digital Transformation Irony

Someone observed that getting engineers to adopt AI tools — finding champions, overcoming resistance, managing change — is exactly the digital transformation problem other industries have faced for years. The irony of applying it to the engineers who built those transformation tools was not lost on the room.

The Arc of Programming Languages

Closing question: will agents start writing closer to the metal, bypassing the abstraction layers that exist for human convenience? The view was yes, eventually, but only when the model decides it serves performance — not because lower-level code is easier for models. Current models still benefit from well-structured, well-commented, human-readable code. Someone noted a trend toward Rust in startups, driven partly by AI flattening the learning curve.

Key Takeaways

• The recursive loop is real. Better coding tools produce better models, which produce better coding tools. Multiple participants said this is why their companies are prioritising coding.

• The bottleneck has moved from writing code to managing long-horizon tasks and deploying agents in regulated settings.

• Developer tooling is being displaced first. Business-facing software with network effects is holding.

• The human role is shifting from writing and reviewing to planning, evaluating, and steering — and the best performers are the ones who stay at the bleeding edge.

• Enterprise adoption is gated by permissioning, sandboxing, and regulatory caution more than by model capability.

• Slop and convergence are real concerns when millions of people use the same models to make the same choices.

• Context remains unsolved. Human-authored context helps; stale or agent-generated context can hurt.

Have any feedback? Reach out on LinkedIn or X.

Every public software CEO fielded the same set of questions about their right to exist in the age of agents. The discourse on the end of software rages on as startups substitute app software with custom tooling developed internally and labs productise aggressively in all directions.

This AI-pilled worldview would see most established enterprise software companies get wiped out and replaced by a combination of AI vendors and in-house builds.

I’m of course talking my book when I argue for the rotation of market share from incumbents to disruptors, but there’s more nuance to this than my X feed cares for.

Let’s look through recent earnings commentary from application software companies to unpack how they’re faring. I chose MNDY, HUBS, INTU, KVYO, BOX, and SAP as a broad basket covering SMB to enterprise apps.

Application Software

MNDY

FY2027 forecasts were scrapped as a result of higher uncertainty around MDNY’s historical strength in the self-serve SMB segment:

Given the evolving nature of the AI landscape and the choppiness in the no-touch demand environment, we believe it is responsible to keep our near-term communication focused on what we can execute and deliver with high confidence.

MNDY can’t reliably model its own self-serve funnel, which historically represented the majority of new customer acquisition.

The ZIRP-fuelled PLG days ended a while ago, but it’s still striking that MNDY deemed the no/low-touch channel structurally choppy.

Disaggregating the drivers of this choppiness is tricky, but commoditisation of the SMB offering by the AI labs is one plausible driver.

On the other end, $50k+ customers now represent 41% of total ARR and have 91% gross retention, whilst $100K ARR saw record net adds, and the $500K+ cohort grew 74% - the move upmarket seems to be working. It just might not be working enough to offset deceleration downmarket.

Margins came down a bit, reflecting investment in the upmarket GTM and AI, but pricing is also adapting:

So our AI capabilities are foundational in our platform. They’re embedded in our workflows. And based on the feedback we’ve gotten from customers, they really enjoy the predictability of PPU pricing, and they like to consume those capabilities that way. With that said, some of the more compute-intensive workloads that drive outputs with these workloads, we are charging and monetizing that through credits. and our customers like the mix of both of those.

The AI SKUs are: AI agents for workflow automation, Monday Vibe for application development, and Sidekick for information retrieval. Traction is still early.

HUBS

On whether HUBS’ data will be consumed by third-party agents and undermine the role of system of record, CEO Yamini Rangan’s answer focused on business logic and domain context:

Our strategy, as I just articulated, is to be that intelligent system of customer content and we have the data, but more importantly, we have the business context, the industry context and the domain context to deliver it. And that’s why customers come to us. They rely on us for that context. They want to use more of our APIs and partners want to customize and build on top of us. And as AI adoption accelerates, the value of our agentic platform increases.

Followed by (in my view, the weaker argument) determinism:

SaaS platforms are more than data. It is the logic, right? You can certainly get a nondeterministic output for a sales e-mail, but try taking a nondeterministic output for your sales forecast. That is not possible. It’s workflows like forecasting, routing, approvals, permissions, that is logic. It’s not data to be sucked away. And ownership accountability and governance, all of those lives inside applications.

Like MNDY, HUBS prices on seats and credits, with the customer support agent driving ~60% of credits consumed, with prospecting agent, data agent, and intent monitoring each contributing 10-15%.. Intriguingly, the support agent’s stated resolution rate of mid-60s is in the ballpark of private darlings like Sierra and Decagon.

Credits are not yet a material contributor to reported revenue, but the trajectory is clear: this is a FY2026 emerging tailwind that could become a FY2027 growth driver.

Management reiterated the core ICP of companies with 2 to 2,000 employees, with strong traction at the upper and multi-product penetration: deals over $5K MRR grew 33%, deals over $10K MRR grew 41%, and customers with 500+ seats grew fivefold. Multi-hub adoption among new Pro Plus customers hit 62%, with 40% of the installed Pro Plus base (by ARR) now owning 4+ hubs, up 6 points year-over-year.

As a sign of yet another AI-native buying third-party SaaS instead of building in-house, Lovable is a customer of HUBS - here’s Dharmesh Shah’s take:

So just because it’s possible. So we have, as Yamini mentioned, a large engineering team knows what they’re doing, spending 97% of their kind of calories using agent coding tools, they’re not doing it to replace internal platforms. So we think the best companies, both AI and non-AI will not be using Vibe coding to replace core systems. They’ll be doing it to add value to their customers. That’s what Lovable doing. I think that’s what some of the best companies in the world will continue to do.

HUBS is seeing the same declines in organic search as a channel as other software companies and has leaned into AEO.

INTU

Having announced apps on ChatGPT’s app store and a partnership with Anthropic to access Intuit products inside Claude, CEO Sasan Goodarzi addressed the elephant in the room:

And it’s also -- the thing I would point out is it’s why companies like OpenAI, companies like Anthropic look to the partnership with us because at the end of the day, they see and understand that this is a business that comes with a lot of liability and LLMs can’t just create the platform that we’ve created overnight.

Time will tell if relinquishing the interface to the end customer to the labs is a wise decision. The question of the value of incumbent apps where the product is primarily consumed by agents will only get louder in the future.

Taking liability is a recurring argument being touted in defence of vertical AI lately, a point INTU reiterated.

In our category, accuracy, compliance, security, reliability of financial decisions, and the liability that comes with it are critical to our customers. It’s our advantage and it’s why we win.

Management presented a narrative of AI enhancing Human Intelligence, i.e. AI-enabled services.

Our success rests on our powerful combination of proprietary data, domain-specific AI platform capabilities and AI-powered human intelligence, which we’ll refer to as HI.

Our system of intelligence combines AI and HI to deliver done-for-you experiences with accuracy, compliance, security, reliability and data privacy that create a durable competitive advantage. This foundation delivers what matters most to customers when it comes to financial insights, money management, taxes, bookkeeping and accounting, leading to complete confidence in their high stakes financial decisions.

AI-enabled services firms have raised boatloads of funding across law, insurance, accounting, banking and healthcare. The AI+HI traction INTU is seeing is a strong data point for the services-as-software thesis..

INTU’s agents are being rolled out at scale, with 3 million customers using them and repeat engagement of more than 85%.

INTU sees 3 monetisation levers: agents drive higher willingness to pay for subscriptions, agents drive cross-sell, agents drive higher services adoption (HI) - no credit-based consumption as less likely to see compute-intensive workloads relative to MDNY and HUBS.

Mailchimp continues to struggle, speaking to broader SME weakness that the MNDY no-touch performance already hinted at. INTU’s push into mid-market ERP is going much better, with new IES contracts growing nearly 50% quarter-over-quarter.

KVYO

KVYO’s business model is more naturally aligned with consumption.

Revenue scales with active consumer profiles and message/interaction volume, not with the number of humans operating the platform. When AI agents generate more campaigns, send more messages, handle more customer service conversations, and drive more transactions, Klaviyo’s revenue grows automatically because interaction volume increases.

When Marketing Agent enables a small team to run more campaigns without adding headcount, Klaviyo captures the increased message volume regardless of how many humans were involved in creation.

KVYO’s AI uptake might be one of the most underrated stories in public markets right now: for Marketing Agent adopters, more than half of campaigns are now AI-generated, performing as well as or better than manually created campaigns while taking significantly less time. Customer Agent resolution rates increased 20 points since launch, with monthly resolution volume up 50%+ since Black Friday/Cyber Monday. 85%+ all-time repeat engagement across 3 million+ customers using agents.

In keeping with this growth, the company is reframing its category - marketing automation is out, ‘autonomous B2C CRM’ is in.

Our technology marries the customer database we founded Klaviyo on and our robust marketing messaging infrastructure with agents for marketing and customer service that will autonomously create, deliver and optimize customer experiences on behalf of a business. And this agent layer that is designing and delivering experiences to billions of consumers is trained from our deep expertise and the trillions of experiences we’ve delivered for businesses already.

Enterprise traction accelerated meaningfully in FY2025. Customers with $50K+ ARR grew 37% to 3,912, with 349 net new additions in Q4 alone — beating the previous record by 25%. Customers generating $1M+ ARR doubled year-over-year. It’s no surprise KVYO hired ex-WDAY co-CEO Chano Fernandez as their new co-CEO. Another sign of the move upmarket is an Accenture partnership who are building practices around “marketing reinvention and service reinvention” powered by Klaviyo, targeting the largest consumer brands globally.

FY2026 revenue guidance of $1.501-$1.509B (21.5-22.5% growth) represents a meaningful deceleration from 32% in FY2025, but this assumes minimal contribution from AI.

KVYO’s answers on long-term durability were grounded in technical differentation rather than business context:

Our ability to be the agent and the platform of choice finds its roots in how our platform was built. This is our durable advantage. At the core is the database and data infrastructure, specifically built to handle the scale of consumer data and indexing and enriching through machine learning and serving hundreds of thousands of requests per second with millisecond latencies.

It allows Klaviyo to ingest, aggregate and govern first-party data in real time, so every consumer behavior, transaction, preference and consent is available to our users and now to our agents to deliver the best possible consumer experience. That foundation is coupled with our marketing platform, which not only provides high throughput, scalable systems to render messages, deliver them with excellent deliverability and enforce compliance, but also integrates with our customer data infrastructure to make last-mile personalization decisions on content, incentives, timing and channels at the moment we deliver a message or experience to an end consumer. Speed and scale matter.

Like INTU, KVYO supports Claude via MCP and has an app for ChatGPT, but remains confident in its infrastructure ensuring durability.

Lastly, the company is one of the best example of operating leverage and margin expansion through AI, with operating margins expansion 900bps YoY in Q4.

BOX

Aaron Levie is one of the most AI-pilled enterprise software CEOs and he deftly managed to tie file-based systems as the winning agent tool-calling paradigm to Box’s strength in enterprise content management:

Files are quite simply the native unit of work for agents. Agents use files to keep track of their work…

Thus, to have an effective AI agent strategy, companies fundamentally need a content strategy. They need a secure platform to manage critical content and ensure it can connect to all of their people, agents and applications. This is what we’re building at Box with our Intelligent Content Management Platform.

The narrative being proposed is that agent proliferation = inflection in volume of enterprise content that needs to be managed, governed, and secured. Every Claude Cowork session, every OpenClaw task, every custom agent workflow generates files that need to persist beyond the stateless agent session. The agent’s compute environment may disappear, but the documents it produced, the contracts it reviewed, the research it synthesised — all of that must be retained, governed, and discoverable.

The strategic implication is that BOX’s TAM grows with agent adoption rather than being threatened by it. More agents = more files = more content management demand. This is why Levie described Claude Cowork and OpenClaw as “universally good things” for BOX — every agentic knowledge work session creates content that needs a secure home.

BOX is seeing its AI SKUs drive enterprise penetration and expansion. Enterprise Advanced launched approximately one year ago and already accounts for 10% of Box’s revenue. The pricing uplift from Enterprise Plus to Enterprise Advanced has been 30-40% on a per-seat basis — at the high end of the 20-40% range initially anticipated. Total Suites customers now represent 66% of revenue, up from 60% a year ago.

Even so, top-line growth was still in the single-digits at 9%.

On monetisation, BOX sees themselves as serving agent demand through API consumption in a headless-first world, or more seats if human-in-the-loop use cases grow. The consumption model is still immaterial overall, but might be the future when the agent-first economy takes off.

SAP

SAP has leaned into the ‘Business Data Cloud’ with a framing that foundation models lack sufficient business context to be reliable. Citing the example of recently won customers like H&M, CEO Christian Klein emphasised SAP’s position as embedded across mission-critical workflows.

And so when I think about the future of AI and SAP, I’m super happy that I have our ERP. I’m super happy that I have our apps because without those apps, I wouldn’t have the data. And without the data, I wouldn’t have an AI.

Management also stressed their right to win against a backdrop of geopolitical turbulence:

While geopolitical and trade tensions have taken a certain toll on our top line performance in 2025, the growing need for sovereignty and resilience also offers unique opportunities for those vendors that could offer technologies and tools to reduce dependencies from dominant offering. As largest non-U.S. software, SaaS and PaaS vendor, there is no company better positioned than SAP to satisfy this rapidly growing demand.

Two-thirds of deals closed with AI in some shape or form.

Looking across MNDY, HUBS and SAP, there’s an argument for SAP’s position being strongest because of the criticality of the data they sit on: ERP, supply chain, manufacturing.

Conclusions

• AI impact on revenue still early, particularly the premise of agentic consumption of APIs or credits

• SMB segment showing weakness, indicating both limits of PLG and competitive threat from labs’ prosumer AI assistants

• Companies are either leaning on accumulated business logic as a durable asset in a future agent economy where software is headless, or on sheer infrastructure differentiation

• Enterprises still value governance and security of AI, as well as counterparties owning liabilities; will incumbents win on this vector?

• Partnerships with labs via connectors and MCPs not seen as major threat to enterprise value, even if surface area of product and customer interactions shrinks; no rev-share or data being transferred. How will these lines evolve?

Have any feedback? Reach out on LinkedIn or X.

Roundtables

March 5: The Future of Software Engineering with Anthropic

Last week, we hosted a roundtable on the future of AI compute with Michael, CEO of Spectral Compute, who have built a compiler that compiles CUDA source code directly to native machine instructions for non-NVIDIA GPUs. We were joined by attendees from Graphcore, Cerebras, DeepMind and more.

1. Nvidia’s Monopoly Position

Michael opened by framing the current state of AI compute as a “monopoly moment.” Key points:

• Nvidia controls 80%+ of datacenter GPU market share and briefly touched $5 trillion in market cap.

• Their dominance stems from a head start in accelerated computing that dates back to CUDA’s first release in 2006–2007. Nearly two decades of consistent innovation in parallel compute has created an entrenched ecosystem.

• Although open-source alternatives exist (e.g., Kronos Group standards, which Nvidia itself co-chairs), none approach CUDA’s adoption or defaultness.

• There are now hundreds of millions of lines of CUDA code in the wild. Developers have built deep expertise around CUDA over years, creating massive switching costs. Michael’s analogy: convincing the world to drop CUDA is like convincing everyone to adopt the UK power plug — technically superior, but practically impossible.

2. Why Vendor Optionality Matters

Michael argued that having alternatives to Nvidia is critical for multiple reasons beyond just cost.

Cost Leverage

• Nvidia was selling H100s at roughly 10x manufacturing cost, enabled by their monopolistic position.

• These margins are unsustainable long-term, but the market hasn’t corrected because of ecosystem lock-in.

Supply Chain Resilience

• Single-vendor dependency carries severe supply risk. When Nvidia misses production targets or its suppliers slip, customers are stuck.

• Worst-case wait times for latest Nvidia compute are 18+ months.

• Geographic prioritisation compounds the problem: North American customers get served first, APAC gets deprioritised, and Middle Eastern customers wait until policy deals are brokered.

Geopolitics

• Compute allocation has become a geopolitical instrument. Where you sit geographically determines your access to cutting-edge chips.

3. Current Challengers and the Competitive Landscape

Michael surveyed the emerging alternatives to Nvidia:

AMD Instinct Series

• Architecturally the most similar chips to Nvidia’s GPUs. The Instinct MI300 series is shipping, with the MI4XX series announced.

• Meta signed a deal with AMD to deploy 6 GW of Instinct compute by end of 2027 — a massive commitment, though execution remains to be seen.

• AMD GPUs are generally ~30% cheaper than equivalent Nvidia hardware, but people still aren’t switching en masse due to software incompatibility.

Cerebras

• Cerebras’ wafer-scale engine draws ~27 kilowatts but is more efficient on a per-token basis at scale.

• Cerebras announced a 500–750 MW deployment partnership.

• Crucially, Michael noted that Cerebras’s chip is not an ASIC — it is programmable and general purpose, which he considers a wise design choice nowadays.

Google TPUs

• TPUs are not purchasable outside of Google Cloud Platform; they are fully vertically integrated. You access them via GCP subscriptions.

• Programming model is fundamentally different: systolic arrays with data-flow architecture rather than traditional threads/warps.

• Writing kernels for TPUs works differently — there aren’t traditional kernels.

ASIC-Based Startups

• There are ASICs focused on transformers like Etched, Positron and even Taalas who baked the transformer architecture into chip-level design, claiming speeds like 17,000 tokens per second.

• Michael was critical of this approach: baking in transformer-specific logic creates the same depreciation cliff seen with Bitcoin miners (90%+ depreciation within 16–18 months).

• He questioned the practical utility of extreme token speeds: “Your coding agent is going to ask you dumb questions a lot faster” — the model quality running on these chips matters more than raw throughput.

• Current speeds on 8B models might scale to 250B+ models in the future, making specialisation worthwhile.

• Michael’s response: AI innovation is outpacing chip tape-out timelines. The field hasn’t settled on a final architecture — the transformer itself may be disrupted. Betting silicon on a fixed architecture is premature.

Other Notable Mentions

• Next Silicon (Israeli) — Michael was most enthusiastic about their approach: they’ve etched the selection DAG part of a compiler into the chip, a novel architectural choice.

• Axelera AI (European, embedded/small-scale inference)

• Michael referenced a list of ~50 startups globally attempting to build AI chips.

Microsoft Maia

• Microsoft launched Maia with an SDK access request form. Michael noted he still hadn’t received his SDK access, suggesting the chip may have incurred into delays.

4. The Depreciation Problem and Programmability

A substantial debate emerged around chip depreciation and what it means for investment decisions.

The Financial Reality

• H100 chips launched at $40k per chip; a single node costs $400k+.

• These are not disposable assets — buyers need confidence the hardware retains value beyond a one-year cadence.

• Bitcoin miners depreciate 90%+ within 16–18 months, creating a cliff that makes planning impossible.

Cerebras

If CUDA is truly so powerful and versatile, why does hardware depreciate so fast? In theory, CUDA’s universality should let you keep running older chips (e.g., multiple A100s instead of one H100).

Michael’s response acknowledged the tension:

• Nvidia has a financial incentive to drive new GPU purchases — they make money on new silicon, not on customers reusing old chips.

• There’s deliberate market segmentation: Nvidia explicitly wants H100s in certain deployments and not others. They don’t even use the term “heterogeneous” for their own chip generations.

• Additionally, hardware constraints make simple substitution impossible: precision (FP) differences between A100 and H100, and capped InfiniBand on A100s mean you can’t just swap older interconnects with newer ones for large-scale training.

The Core Argument

Michael’s central thesis: programmability and general-purpose capability are the most important chip design attributes because they preserve long-term value as workloads evolve. Application-specific chips face existential risk from architectural shifts in AI.

With tape-out costs at $20–30M minimum and 2–3 year chip lifecycles, a startup raising $100M gets essentially one generation of silicon. If the AI landscape shifts (e.g., mixture-of-experts changing distributed compute patterns), an architecture-specific chip is stranded.

5. Spectral Compute

The CPU Analogy

Michael explained the problem by contrasting CPUs and GPUs:

• In the CPU world, you write code once and it runs on AMD, Intel, or ARM without modification. You pick hardware on merits (price, performance, cache) without rewriting your software.

• This freedom does not exist for GPUs. Switching from Nvidia to AMD requires porting code, despite the chips being architecturally very similar. AMD calls thread groups “waves” where Nvidia calls them “warps” — substantively the same thing, but different enough to break compatibility.

The Spectral Approach

• Embrace CUDA as the de facto standard rather than trying to replace it.

• Let developers write CUDA code, pipe it through Spectral’s compiler, and have it compile directly to AMD native instructions (and eventually other architectures).

• This is not transpilation — it’s ahead-of-time compilation that goes all the way down to native hardware instructions, enabling deep optimisation.

• Developers can still hyper-optimise for specific hardware via #ifdef blocks for particular devices, but the default path works across all supported GPUs.

Why Direct Compilation Matters

• AMD’s CUDA equivalent (HIP/ROCm) is frustratingly different in subtle ways that break programs and is also slower in many cases.

• Transpiling to HIP would lose optimisation opportunities. By compiling directly to AMD native instructions, Spectral can reason about the full pipeline from high-level CUDA to transistor-level operations.

• A specific example: AMD uses opposite column/row format for tensor operations compared to Nvidia. Spectral’s compiler can detect this, prove that reordering is safe, and perform the transformation at compile time with zero runtime overhead.

Performance Results

• Spectral showed benchmarks comparing their compiler against AMD’s HIP toolkit. In several cases Spectral outperforms AMD’s own software stack; in others there’s room for improvement on both sides.

• They also compared against OpenCL: on Nvidia, OpenCL is behind; on AMD, it’s roughly on par. But there’s limited OpenCL code in the wild to benchmark against.

Coverage Status

• Michael showed their current CUDA API coverage, prioritised by real-world usage. Core CUDA APIs are substantially covered, including GrowMax operations.

• PyTorch support is their key milestone: early builds expected mid-Q2 2025. This is significant because PyTorch exposes nearly all low-level CUDA APIs to Python, so supporting it effectively means comprehensive CUDA coverage.

Business Traction

• Most traction currently in academia — researchers using some of the world’s biggest supercomputers.

• Working with a consortium of premier motorport teams on safety applications.

• Engagements in HFT/algorithmic trading.

• Goal for next 12 months: serious deployments with large AI labs and adoption by neocloud providers as the standard portability layer.

Team and Hiring

• 22 people, remote-first, clustered around Europe (6 in greater London, 4 in Edinburgh, 3 in the Netherlands, 2 in Greece, 1 in Denmark, 1 in Italy, co-founder in California, 1 in Singapore).

• Talent is extremely scarce: people who are both GPU architecture experts AND compiler specialists number roughly ~200 globally. Spectral recruits from universities and cross-trains.

Open Source Strategy

• Not currently open-sourcing the core technology. The “coup de grâce” is their vendor-neutral compiler optimisations, which they consider proprietary IP — “for the same reason Nvidia isn’t open-sourcing theirs.”

6. AI-Assisted Compiler Optimisation

A notable exchange occurred around AI’s role in GPU code optimisation.

• An attendee noted his team went from two months to two weeks of optimisation using AI-assisted coding tools at low-level GPU programming.

• He questioned whether traditional compilers could ever make the leap from naive attention to Flash Attention — something that took human researchers two years to develop.

• Michael’s view: the compiler “just isn’t smart enough yet” — nobody has written compilers to fully exploit GPU architectural possibilities. That’s a core part of Spectral’s R&D.

• He argued AI-assisted code generation will eventually outperform human hand-optimisation because it can brute-force search a much larger space of possible programs. Combined with fast correctness verification, this creates a direct path to highly optimized code.

• Importantly, that optimised code should be written in CUDA (not HIP or another niche language), for the same reason LLMs are better at English than Danish: the training data corpus is vastly larger.

7. The Cloud Hyperscaler Lock-In Dynamic

Discussion turned to how cloud providers are compounding the vendor lock-in problem:

• Google, Microsoft, and Amazon each want to control the full vertical stack — custom chips, custom SDKs, and proprietary APIs.

• Once customers build on a hyperscaler’s fully integrated stack, switching costs become prohibitive (similar to AWS’s original API lock-in playbook).

• The landscape will fracture further, not consolidate. Unlike the CPU world (Intel, AMD, ARM — three similar architectures), the GPU/accelerator world could see 20–30 different architectures.

• Each hyperscaler’s new chip generation introduces architectural changes, deepening incompatibility.

• This trend is likely to intensify as the industry shifts from training toward inference, where specialised inference chips from each hyperscaler will capture more market share.

8. Training vs. Inference Shift

A significant data point emerged in discussion:

• 60%+ of total compute used to train foundational models is now post-training (reinforcement learning), not pre-training. The pre-training dominant era is over.

• This RL-heavy workload can be disaggregated across data centers but still faces throughput constraints between clusters.

• This shift has implications for chip design and data center architecture — different workloads may favour different hardware configurations.

9. Data Center and Infrastructure Constraints

Cooling Capacity

• Cerebras’s wafer-scale engine draws 27 kW. Regardless of chip vendor, liquid cooling and high-power infrastructure is inevitable.

• However, the buildout is slow: companies like Cloudflare with hundreds of thousands of PoPs globally can’t upgrade all sites simultaneously.

• Meanwhile, the most common PCIe inference card (H100 NVLink) fits standard 600W PCIe slots. Nvidia hasn’t even updated this form factor for Blackwell. There’s an underserved niche for inference in existing data center infrastructure.

Multi-Data-Center Training

• It was noted that if you can link H100s across two data centers via high-bandwidth interconnects, you can approximate the performance of newer GPUs without ripping and replacing.

• Michael agreed this works for some workloads that can be disaggregated, but not all.

10. Key Debates and Open Questions

Will Nvidia’s Dominance Erode Through Interoperability?

• Developers won’t switch because alternatives aren’t faster; alternatives can’t get faster without developer adoption.

• Michael’s answer: some large players (Meta with AMD) are making the leap. Spectral’s role is to be the “universal Babel translation layer” that removes the switching cost entirely. Their roadmap includes supporting AMD’s ROCm libraries on Nvidia hardware — enabling developers to use the best of both ecosystems on either vendor’s chips.

Groq and Nvidia’s Acquisition

• Nvidia acquired a non-exclusive IP license to Groq’s core technology and hired their CEO and senior staff in late 2024. The deal was characterised as more than 3x Groq’s previous valuation, effectively an “acquihire” structure.

GPU Utilization Gaps

• There’s research showing traditional kernels only use 40–50% of Nvidia GPU compute capacity, largely due to memory-bound bottlenecks.

• Michael framed this as an algorithmic optimisation problem with multiple attack vectors: data compression, hardware changes, and compiler improvements. No single approach solves it universally.

• Nvidia’s tensor cores are themselves an example of the hybrid approach: an ASIC-like unit embedded within a general-purpose programmable chip.

The Cisco Networking Analogy

• There’s a parallel to Cisco’s networking dominance: Cisco created an entire certification ecosystem that made switching to Arista painful. However, customers with extreme performance needs (e.g., low-latency trading) will always seek alternatives.

• Cerebras signed with OpenAI specifically because OpenAI couldn’t achieve >1,000 tokens/second inference with other hardware and needed breakthrough performance.

Signals

What I’m Reading

A Level Headed Look at State of Software

The path to ubiquitous AI

Strategies for learning

Earnings Commentary

Have any feedback? Reach out on LinkedIn or X.

Gradient Descending Roundtables

February 25: The Future of AI Compute

March 5: The Future of Software Engineering with Anthropic

Last week, we hosted Matt and Thomas from Scale AI, following a previous roundtable on Rubrics as Rewards. We discussed RL environments, one of the bigger themes in AI this year.

Core Challenges in Building RL Environments

1. The Optimisation/Scale Problem. The ideal RL environment should be extensive and cover many domains, but larger environments are computationally heavier. There’s a fundamental tension between realism/breadth and efficiency. GPU constraints are significant across training, inference, and environment simulation simultaneously. Anyone doing large-scale RL environment training would face GPU, DRAM, and potentially CPU bottlenecks across all three vectors.

2. Verifiability Across Domains. Coding is the gold standard for RL because outputs can be programmatically tested. But many domains lack this property. Matt and Thomas described using “rubrics” — sets of 20-30 rules used as an LLM judge to score outputs in harder-to-verify domains — but acknowledged this still carries reward hacking risk, is compute-intensive (running two sets of inference at scale), and depends heavily on rubric quality, which requires expert curation.

3. Real-World Relevance. Environments must actually drive performance on things end customers care about. Building an environment around vending machines might not justify huge investment. The benchmark and the environment need to be jointly validated — you need confidence that improving the benchmark actually reflects improving real-world performance, not just benchmark hacking.

The Vending Machine Case Study

This was presented as a Scale AI internal hackathon project, partly inspired by an actual Anthropic experiment where Claude was given control of a real office vending machine (stocking items, web searching for suppliers, sending emails to reorder). The hackathon version was scoped down to fit a 2-day format.

Environment Structure

The environment was built around a simple loop:

• Observation (input to model): Current inventory (item slots, quantities, prices), available cash balance, current time, and a delta log of changes since the last step (deliveries, price changes, sales)

• Action (output from model): A JSON object specifying restock orders and/or price changes

• Step function: Takes the action JSON, updates the simulator state, returns the new observation

The model used was GPT OSS (with special tokens visible in the transcript examples). Reasoning tokens were artificially constrained because unconstrained reasoning allowed models to crush benchmarks.

Mechanics of the Simulator

• Demand was simulated based on price (lower price = higher demand)

• Operating costs were baked in so the agent couldn’t simply do nothing — inaction leads to consistent losses

• Expiration and other costs added further realism

• Delivery lag was simulated (restock orders appear in future observations)

Training Run

• Used GRPO (Group Relative Policy Optimization) algorithm

• Only 50 training steps — a very short run

• Eval: 10 independent runs of 4 days each (16 steps per run)

• Base model (pre-training): Lost ~50% of starting capital on average

• Post-training: Generated massive returns (example cited: ~£3,500 from £100 starting capital)

Key Finding — Reward Hacking vs. Environment Realism The team initially suspected reward hacking given the extraordinary returns, but upon inspecting actual generation traces, the model had genuinely learned good business operations. The real problem was environment realism: the simulation had no competition. In the real world, high-margin strategies attract competitors who arbitrage profits away. The model found the globally optimal strategy within the (unrealistic) environment, not a strategy that would work in the real world.

Reward Hacking Example Encountered Early in development, the team found the agent exploiting a loophole: one item had very low supplier cost but very high demand. The agent filled every slot with that single item at maximum margin. This was caught by manually reviewing generation logs — the entire output was just repeated restock orders for the same item.

Reward Design: Sparse vs. Dense Rewards

A significant portion of the discussion covered reward function design:

Sparse Rewards (traditional approach)

• Run the entire simulation, assign a binary 1/0 at the end (success/failure)

• Extremely compute-inefficient — you spend enormous compute for 1 bit of signal

• Poor for scaling training

Dense Rewards (what they used) Three improvements over sparse:

1. Continuous reward instead of binary: Use actual cash balance as reward (e.g., £100 profit → reward of 100, not just 1)

2. Intermediate step rewards: Assign small rewards/penalties at each step, not just at episode end. If the agent makes a small profit in step 3, that’s reflected in the step-3 reward. This provides dramatically more training signal per compute dollar

3. Curriculum/shaping: Adding human prior knowledge through reward structure helps models learn faster, though it introduces the risk of over-constraining behaviour

The tension highlighted: more intermediate rewards guide the model better but risk teaching it to optimise for the reward signal rather than the true objective. The answer given was that compute constraints force you toward denser rewards in practice — in an ideal world with unlimited compute you’d just use a single sparse end reward, but that’s not viable at scale.

Verifiability and Domain Expansion

Domains that work well with RL environments

• Coding (programmatic test execution)

• Computer use / tool interaction (did it navigate to the right page? Binary, immediate)

• Specific financial tasks (binary compliance checks)

Rubric-based approach for harder domains

• Construct 20-30 expert-curated rules

• Use an LLM as judge to score outputs against the rubric

• Scale has “PR Bench” (public) specifically for financial services and legal

• Risks: reward hacking via rubric exploitation, LLM judge biases, compute cost of dual inference

• In enterprise settings, they often build narrow domain-specific benchmarks rather than broad ones

Key insight on domain selection: For RL environments to be valuable, the benchmark must be: (a) something you can agree actually represents desired performance, and (b) tight enough that improving it reflects real capability gains rather than benchmark gaming. “Maximise profit in an undefined scenario” is too broad; “improve Rust coding on this specific task suite” is sufficiently constrained.

Reward Hacking Detection and Prevention

How it’s caught: Primarily by sampling generations and reading the raw model outputs manually. No automated detection was described as reliable.

Why it’s hard to prevent by design: Models find exploits humans don’t anticipate because they run thousands of simulations and inevitably discover edge cases in the environment logic.

Real-world example cited: Chinese labs reportedly hacked SWE-Bench (a coding benchmark) by looking at future git commits to copy solutions rather than actually solving problems — even frontier labs struggle with this.

Dual-benchmark approach: Run training against benchmark A, but also monitor performance on a separate benchmark B that targets the same real-world capability. If B collapses while A improves, strong evidence of benchmark hacking.

Discussion: Multi-Objective and Competing Goals

There are challenges with competing goals — e.g., a shopping agent maximising short-term basket size but damaging long-term customer retention:

• The environment itself doesn’t resolve competing objectives; the agent is free to explore any strategy

• The horizon is fixed (e.g., 4 days) and the agent learns whatever maximizes reward within that horizon

• Competing long-term goals (e.g., a B-corp optimizing for both profit and sustainability) are hard to encode — you’d need to design separate reward components, or accept that some goals require other training methods (RLHF rather than RLVR)

• RLHF (human preference feedback) is the approach for subjective domains (does this slide look good?) while RLVR (verifiable rewards) handles objective ones

Scalability and the Real World vs. Simulation Debate

Can you just use the real world as the environment? Why not use a real corner shop? The answer: you don’t have infinite compute to represent the real world, and the real world is too slow and expensive to run millions of times. The optimization is finding a simulation that is realistic enough to drive genuine performance improvement but constrained enough to be computationally tractable. This is described as an open problem for 2026.

Multiplayer/competitive environments as middle ground OpenAI Five (Dota 2) was cited as the canonical example — a rich, competitive, multi-agent environment that can be run millions of times. Video games were popular for this, but labs have moved away because real-world tasks (coding, office work) drive more economic value. However, for robotics and world models, game-like simulation environments are coming back into favor.

CPU as emerging bottleneck As environments become more complex (computer use, video editing, Linux environments), CPU becomes a constraint alongside GPU. Running heavy OS-level simulations at RL training scale could make CPU the binding constraint. One participant suggested rewriting environments in JAX to run on GPU, though this clearly doesn’t apply to high-fidelity real-world simulations.

Training Pipeline Efficiency

Emphasised as critical because RL is inherently sequential (run simulation → training step → repeat, cannot parallelise easily):

• Any inefficiency in any layer (kernels, model, environment simulator) compounds across the entire training run

• GPU utilisation must be maximised since RL idle time is extremely costly

• Environment step functions must be highly optimised to not block training

Multi-Agent Training Clarification

A point of confusion clarified in Q&A: the 10 evaluation runs shown in graphs were not separate agents learning from each other — they were 10 independent runs of the same model (to account for non-determinism). The training consisted of 50 gradient update steps where all runs contributed to shared model weight updates via GRPO. Post-training, another 10 independent eval runs were conducted.

Signals

What I’m Reading

Founders Should Chase Secrets

Time’s Up for SaaS (Grow Faster or Vanish)

End Game Play

In defense of vertical software

You’ve been kicked out of the arena, you just don’t know it yet

Earnings Commentary

Have any feedback? Reach out on LinkedIn or X.

Gradient Descending Roundtables

February 18: RL Environments with Scale AI

February 25: The Future of AI Compute

March 4: The Future of Software Engineering with Anthropic

Before Moltbook took over the internet this past weekend, the sell-off of public software companies reignited debates around the future of software.

Andrej Karpathy comments on recent advances in AI coding read ominously:

LLM agent capabilities (Claude & Codex especially) have crossed some kind of threshold of coherence around December 2025 and caused a phase shift in software engineering and closely related. The intelligence part suddenly feels quite a bit ahead of all the rest of it - integrations (tools, knowledge), the necessity for new organizational workflows, processes, diffusion more generally. 2026 is going to be a high energy year as the industry metabolizes the new capability.

The markets are spooked even as incumbents are crushing earnings estimates, as fundamental assumptions behind the quality of the software business model have come into question, as Jamin Ball wrote:

Mainly, confidence in the SaaS business model has shattered. SaaS businesses were long thought of as “cash flow annuities.” Loose money early on, flip profitable, and then every year print cash predictably. You could then calculate the “intrinsic value” of a SaaS business by summing the present value of every annual cash flow, with a terminal value assumption. More specifically, calculate the present value of the next 10 years of cash flows (discounted back to today), and make an assumption of the terminal value (ie year 11 onward).

AI is creating huge questions about what the future retention rates of these “stable” software companies will be. Software bears will say this platform shift will lead to deteriorating retention rates as companies leave behind legacy SaaS vendors for modern AI native alternatives. At the same time (and related), this is increasing the probability that the terminal value is in fact 0 for some companies.

Retention has stabilised, having compressed from 2021 levels.

Software companies have become cash flow profitable, but this hasn’t translated to true economic profitability on a GAAP basis given the impact of stock-based compensation.

In light of this, there are three scenarios being debated, all against a backdrop of a potential historic upgrade cycle across the enterprise:

1. Incumbents infuse just enough AI across their product surface area, combined with their distribution advantages, to drive renewals

2. AI-native disruptors win the upgrade cycle and displace incumbents

3. Enterprises build their own applications/agents

The most compelling argument against the latter is simple: focus.

Which leads me to the greatest irony of all: the companies that spend the most time and effort replacing SaaS with internal tools are the most likely to be disrupted. In business, nothing matters more than focus. The second you start expending engineering, product teams, and designers to reinvent the wheel, you’re distracting from your core competency, slowing everything and everyone down. Why do you think Netflix won the streaming wars? Why does Spotify lead in streaming music? Why does TSMC make all the chips in the world and not Intel? Focus. The best thing any business can do is invest time and effort in amplifying its own product offerings with AI. Replacing existing products to save negligible costs is a waste of time.

Followed closely by the delta in capabilities/total cost of ownership delivered by third-party vendors who are themselves singularly focused on their core product, versus software developed in-house.

SaaS is used because you get an exceptional product at extremely compelling financials; building it internally makes ZERO sense. The corporate version of Slack costs $18 per head per month. For a company of 1,000 people, you’re paying approximately $220,000 a year for Slack. $220K/year, which yields a proxy for roughly $75M in R&D effort annually. In other words, you’re earning a 340x amplifier for your money. SaaS has historically been an exceptional business because of how compelling the financials are for everyone. Slack is able to build an exceptional product because it focuses on one thing, their messaging app. Then they offer it at very compelling financial terms because they can leverage scale. It’s cheap for their customers while still highly profitable for them due to margins and scale.

JP Morgan’s internal tech team made similar remarks: AI is a TAM-expanding technology and the best use of incremental dollars is on the core business rather than rebuilding third-party apps.

Recent surveys suggest this sentiment still holds true in the enterprise.

For now, at least, the upgrade cycle is going to be contested by incumbents and disruptors.

Klarna’s narrative in the run up to its IPO is instructive.

Staking its credentials as an AI-native company, the company decided to churn from Salesforce, among other SaaS apps, and consolidate around a graph database built on neo4j.

Sebastian Siematkowski’s explanation nearly one year ago was prescient:

Key to our explorations became the conclusion that the utilization of SaaS to store all forms of knowledge of what Klarna is, why it exists (docs), what it tries to accomplish (slides, tickets, kanban boards), how it is doing (sheets, analytics), who is it dealing with (CRM, supplier management), who works here (ERP, HR) and what it has learnt was fragmented over these SaaS—most of them having their own ideas and concepts and creating an unnavigable web of knowledge that required a tremendous amount of Klarna specific expertise to operate and utilize.

Since then, incumbents have continued to erect walls around their data, and the concept of context graphs to collect company-specific decision traces has gone viral.

Directionally, it’s becoming clearer that enterprises want to remove silos and instrument the telemetry to capture decision traces that will enable agents.

That future, one where stateful agents have freedom to operate across a company’s data estate, will reward vendors who are able to build the best harnesses around foundation models for their domain.

The best harness will translate to the strongest capabilities, and by extension the highest ROI. Support agents with the best harness will deliver the highest resolution rate. Coding agents with the best harness will have the highest throughput of PRs that get merged. Legal agents with the best harness will be faster, cheaper and more accurate at reviewing contracts.

Incumbents have the distribution, ecosystem, long-standing CIO relationships and platforms. The startups taking share away from incumbents are leaning into the future enterprises want and shipping 10x better products. The gulf has to be big enough to win this generational upgrade cycle.

A number of debates will be settled in coming years.

What’s the right UI for knowledge work in an agent-first world?

Will UIs optimise for the experience of agent delegation rather than humans doing the work?

We’re moving at warp speed. Buckle up.

Signals

What I’m Reading

How to Bet on AGI

Lessons from Building AI Agents for Financial Services

eating lobster souls Part III (the finale): Escape the Moltrix

Intel Earnings, The Agentic Opportunity, Intel’s Mistaken Pessimism

A Straussian reading of The Adolescence of Technology

Earnings Commentary

Have any feedback? Email me at akash@earlybird.com.