• May 20th: Pydantic x Glyphic Engineering Night

• May 28th: Inference Stack Innovation with Crusoe

• June 4th: Forward Deployed Engineering with Anjor Kanekar

Two weeks ago, we hosted Eric Rodriguez (Harmonic), Brendan Regan and Dennj Osele (Latinum) for a discussion on foundation models for mathematics, formal verification and its applications.

Formal verification is the practice of expressing software programs and mathematical proofs in a language that a machine can check mechanically. Lean has become the most important proof assistant in this ecosystem, with Mathlib serving as its core knowledge base: a formalised library of mathematics now comprising roughly 2.25 million lines of human-curated code. Historically, however, the cost of formalisation has kept the field relatively narrow, concentrated in academic mathematics and high-assurance domains such as hardware, cryptography, and security-critical software.

That constraint is beginning to change. LLMs are making it cheaper to generate Lean code and proofs, reducing the human effort required to formalise mathematics or verify software. If the trend continues, formal verification could move from a specialised discipline to a more widely used layer for checking mathematical, software, and eventually AI-generated outputs.

Models can now generate Lean proofs at a scale and quality that human reviewers struggle to distinguish from human work, opening up a credible path to formalising large parts of mathematics, verifying production software, and producing AI systems whose outputs can be checked rather than merely trusted. The deeper bet is that mathematics is uniquely well-suited as a training environment for general reasoning — the compiler provides a perfect reward signal, which is the cleanest reinforcement-learning setup currently available.

On to the discussion.

1. The frontier has moved past the IMO

Olympiad-level mathematics is now treated as a solved training objective, with new postgraduate mathematics tests emerging, like First Proof. The active research problems are library construction and longer time-horizon research-level proofs, not individual theorem generation.

This changes how capability should be measured. A theorem that takes ~10 hours to prove against a mature library may take ~100 hours without it (Metr time-horizon framing). Library growth is the rate-limiter. Systems that amortise library construction, not just prove harder theorems, are what unlock the next step function.

2. New libraries emerging

LLM submissions are increasing pressure on Mathlib. The underlying constraint is maintenance cost. Lean does not guarantee backward compatibility, so each release requires Mathlib to be updated.

The risk is that Mathlib’s current model, which was designed for human-scale throughput. Parallel libraries may grow alongside it rather than feeding into it.

Multiple parallel AI-generated Lean libraries are emerging outside Mathlib, each aiming to be larger than Mathlib itself (but still nascent). There’s a risk that these will be mutually incompatible, with divergent formalisations of the same objects.

3. Latinum’s approach

Denny spoke about Latinum’s model: recursive latent reasoning. A large encoder ingests context, produces a reasoning step in latent space, feeds it back, and cycles until convergence; a decoder tokenises only the final output.

Lineage: JEPA-style world models, Universal Transformer.

Key claim: the recursion itself is trained, with latent states supervised at each step, permitting parallel internal chains that converge externally. Stated rationale: chain-of-thought burns scarce context window; internal recursion is unconstrained because nothing is printed.

4. Harmonic’s model

Eric spoke about hybrid tree search. Three points on a spectrum:

• Whole-proof generation — simple, no granular feedback, one-line breakage forces full regeneration

• Tree search — tactic-level policy model, parallel state exploration, heavy infrastructure burden (Lean execution at scale)

• Hybrid decomposition — break the problem, use whole-proof generation on pieces, assemble via high-level strategist

Why frontier LLMs don’t natively do tree search: base models are trained for full answers, not single tactics. Tree-search policy models are trained differently and have produced proofs not previously known to humanity (Erdős-class problems), so this is not regurgitation.

5. Action-space trade-off: fundamental, can’t be engineered away

Lean offers two obvious cheating mechanisms for an AI prover. sorry is a placeholder that lets the model skip a proof and still compile — equivalent to writing “trust me.” Custom axioms let the model assert something is true without proof. Both are legitimate Lean primitives with valid uses, but both are also the path of least resistance for a model that can’t actually prove what’s being asked.

Earlier prover systems were built so the model literally could not emit sorrys or introduce axioms — the only allowed operations were adding new content and proving it from existing primitives. This made cheating structurally impossible. Users rejected it: pure-additive systems cannot refactor existing proofs, fix incorrect user-provided definitions, or scaffold a proof structure with intentional placeholders. None of these are exotic workflows — they are how Lean is actually used.

Current approach used by Harmonic: open the action space, constrain harmful actions case-by-case. Recent literature is consistent: over-restriction degrades model performance analogously to constrained attention.

The hard part is that sorry is not purely a failure signal. Mathematicians structure large proofs by writing skeletons with deliberate sorry-marked gaps and filling them in later (the “Lean blueprint” workflow). Models doing this are working correctly. Models emitting sorry because they gave up are working incorrectly. Token-level the two are identical. The reward signal has to distinguish legitimate placeholder use from give-up behaviour without surface-level cues — a non-trivial supervision problem.

Read-across to agent infrastructure: the capability-vs-constraint trade-off is the same problem every agent company faces. Give the model file-write, shell access, or arbitrary API calls and it can do useful work — and also cause damage. Restrict the surface and the agent cannot do real work. Harness engineering — the discipline of designing this trade-off — is now a real engineering function. Formal maths is the cleanest version of the problem because cheating is well-defined; the same logic applies across general agent design.

6. Why maths? Two justifications, in tension

Generalisation bet. Humans good at Fermat-class problems tend to transfer to code and quant reasoning. Train models on maths, expect transfer. Counter: the causal direction may not hold for models — selection effects in humans don’t imply training effects in models. Risk case is a system that plateaus as a domain-specialist.

Data argument. Training data is the binding constraint. Maths plus a compiler is infinite, perfectly-labelled data — the compiler is 100% correct.

Best pushback: mathematical data is not infinite in the relevant sense. 1+1+1+... is trivially generable and useless. The hard problem is deriving which abstractions matter — which theorems are worth proving. Same failure mode as code-focused AGI approaches using compiler-pass or test-pass as reward: clean signal, no direction.

7. The underlying reason labs started with maths: RL signal quality

RL on formal maths works despite being a single-player game with no native win condition. The compiler is the reward signal — making formal maths among the cleanest RL environments available.

This explains the path-dependence of frontier lab strategy: they started where reward signals were cleanest (maths, reasoning), then migrated as signals degraded. The migration targets — Office, Excel, accounting — are domains where partial reward is recoverable (value reconciliation, keyword presence).

8. Mathematics occupies a privileged data position

Sharper framing of the data argument: maths has structure from which semantics can be inferred. Raw Python or JSON does not. A proof is recognisably a proof; a type signature encodes intent. Arbitrary code on the internet does not train the same way.

Lean 4 was designed as a programming language first and proof assistant second. In principle: translate conventional code to Lean, execute and verify simultaneously. The runtime supports full applications.

Commercial threads

Chip verification

Industry ratio: ~2–3 verification engineers per designer. Silicon errata economics justify this. Years of chip timelines are verification-bound.

Under-automated, and hardware description languages have small training corpora relative to mainstream languages — creating specialist-friendly data wedges. Self-play approaches (AlphaGo-analogous) are in commercial development but unsolved.

Software verification

Recent large rounds in math-to-software-verification reflect the thesis shift. The operative question is not whether every SaaS app needs formal verification, but whether verification becomes cheap enough that cost-benefit flips for mid-criticality systems. Current state: hire specialists, give them unlimited runway. Target state: run the verifier, price the failure mode.

Contract-first languages

Dafny is the reference case — contract first, implementation second. Never broke through because it required a mathematician and a software engineer. AI collapses this: model writes the contract, then the code; humans only review the contract. Code trends toward “the new assembly language.”

Destination is unclear — Dafny, extended Rust, Lean 4. Rust already moves most runtime bugs to compile time, so Python → Rust → contract-first is a plausible migration path.

Interconnection problem

Individually-verified components (CPU, router, crypto library) compose into systems with emergent vulnerabilities via the OS and network layer. Local verification is necessary, not sufficient. Concurrency proofs are harder than sequential proofs by another order. The gap between “verified component” and “verified system” is open.

• May 6th: AI-Native GTM with Clay

• May 20th: Pydantic x Glyphic Engineering Night

• May 28th: Inference Stack Innovation with Crusoe

• June 3rd: Forward Deployed Engineering with Anjor Kanekar

Intercom is arguably one of the most successful examples of a SaaS company becoming AI-native. They declared Code Red early after ChatGPT launched and went all in on Fin, their AI agent for customer support. Since then, the company has started publishing a blog charting their journey to become as AI-pilled as they come.

Brian Scanlan, Senior Principal Systems Engineer, went viral when he gave a peek into Intercom’s Claude Code tooling.

I had the chance to speak to Brian about how Intercom is changing hiring practices, instrumentation to measure the right metrics, consciously investing in Anthropic as a platform, org design, code quality and graduating from tokenmaxxing to seeing high quality PRs per engineer inflect. You can read some of the Intercom writing referred to in this interview here and here.

Talent and the interview process

You built a console where Claude can execute arbitrary Ruby against production data, and you mentioned the top-5 users of the read-only Rails production console aren’t engineers. They’re design managers, support engineers, PMs. How has that shift changed how you think about what you’re looking for in talent, and how has your interview process changed?

We’ve always looked for people who are flexible and product-oriented. Intercom is a product-oriented company, the founders are product people, and the name of our engineering role is product engineer. Until recently we even had a product component on our engineering interview. The same is true of designers and PMs: we want them close to reality, whether that’s users, code, or whatever else. There’s a great tweet from our head of product where he basically states that nothing else matters apart from code in production. I love that coming from a head of product, because you might expect them to be focused on plans, roadmaps, Gantt charts, and designs. Instead, what he cares about is how fast we’re getting real features and real code into production.

So this is more of a slight evolution than a reinvention. We’ve always been tech-agnostic. We’ve never just hired Ruby on Rails engineers, and we’ve been pretty liberal about hiring people from diverse backgrounds. I came from Amazon, for example. What we interview for is the ability to teach, learn, and change, the ability to be open-minded, more than five years of any specific stack or the aura of having worked at Google.

The tweaks we’ve made are around openness to AI. Most people showing up in our loops these days have seen what Intercom is doing and want to be part of it, so it’s not usually a problem. But we still get people from environments that have had zero AI exposure, or from more traditional backgrounds. Their enthusiasm and willingness to change is what matters. We look for openness rather than five years of cloud experience.

Looking forward, we expect design, product, and engineering roles to continue to converge. There’s still incredible value in depth across all these areas, but we now have product engineers doing full product management of features, designers shipping real code, and PMs shipping code as well. It’s not just box-checking. It’s about reframing those roles to look more like a builder role rather than a member-of-technical-staff role, rather than “I’m a designer and I hand stuff off to engineers.” We’re already well set up for this; the trajectory is just more and more towards everyone being able to do everything.

Measuring what matters in Claude Code adoption

You’ve instrumented every Claude Code lifecycle event with OpenTelemetry. This essentially treats Claude Code like a product you’re iterating on internally. What metrics do you actually look at to decide whether a skill is working, and how do you avoid the trap of measuring activity instead of impact, e.g. tokens spent, which can be a very poor proxy for value creation?

There’s no shame in going through a journey of adoption where, at the start, you’re just looking to force people to try things out, to token-max. The opportunity cost is so big, and the amount of change you’re trying to drive through people and the organization is so large, that it’s perfectly fine to have a slightly immature initial phase where everyone has to try this and use it as much as possible. Yes, you’ll waste a bit of time, but what you’re really getting is people becoming comfortable with the tools, figuring out where they work and where they don’t, and where you need to invest. You’re putting pressure on the system. If you’re too careful, or chase the perfect measurements too early, you end up learning lessons later than you should have.

One of the ways we’ve historically built at Intercom is to do the harder stuff early. In a lot of project plans, you load the start with easy work to manufacture the illusion of momentum, and the gnarliest, hardest parts get pushed to the end, which is also when you find out whether the thing actually works. We flip that. We want you to hit the hardest part first. We want you to be hungry for the learning of “does this truly solve the problem” rather than burying it.

So with token-maxing or output measurement, you have to first open people up to change. That means, “just go use it for everything, doesn’t matter how good it is.” You need to encourage that initial use: race people against each other, run a crappy leaderboard, gamify it. But you don’t confuse that with outcomes, which is what you ultimately care about. You go through a juvenile phase deliberately, because gamifying the trial is the easiest way to get people to test it across everything and figure out where it works and where it doesn’t.

Build vs. buy in an AI-native infrastructure stack

You’ve got 13 plugins, 100+ skills, and a JAMF-managed marketplace. That’s a lot of internal infrastructure to maintain. I’m sure you’ve been able to rip out some SaaS apps as a result. How do you make the build-vs-maintain tradeoff as a company?

Historically, we’ve had an engineering principle that we are technically conservative; we bias towards buying over building. We want people focused on shipping product rather than everything else, and we lean heavily on vendors rather than building expertise across the board.

That has changed somewhat. One of the biggest things to take off at Intercom, arguably bigger than Claude in software development, has been the adoption of Claude Code with a well-set-up suite of MCPs and access controls across the rest of the company. What people have ended up using it for is building their own sales dashboards, customer health dashboards, and highly granular customer-specific reporting, usually someone in sales or someone driving these things directly. What this has effectively replaced is business intelligence tooling and the likes of Tableau.

We didn’t deliberately set out to replace Tableau. We’ve just had a kind of bottoms-up product-market fit for this way of working, and we couldn’t go back to a hosted platform or a more traditional system. Once people have great, well-controlled data, good guardrails, and good places to host reports or small applications, the need for a centralized purchase of something like Tableau diminishes.

So I wouldn’t say we’re consciously trying to rip out big pieces of software, but the need for smaller purchases (role-specific workflows, reporting tools, that kind of thing) is shrinking. We’re probably purchasing far fewer of those, and we’re likely on a path to replacing things like Tableau over time, but very much bottoms-up. We’re not doing top-down “let’s vibe-code a Salesforce replacement today.”

The harness around Claude Code, and the role of the human engineer

The Claude Code hooks you’ve imposed to enforce adherence to your PR workflow and approved tools are great examples of how you’re building your own harness around Claude Code to get maximum value. What other best practices have you had to follow to make Claude Code more reliable over time, especially as we enter the age of background agents? And how do you see the role of human engineers evolving to best enable coding agents?

We’ve had to strengthen the guardrails substantially and provide a lot of guidance, and we’ve socialized this problem heavily. We tell people: we are all part of the flywheel, and we expect everyone to be part of it. I’ve written an engineering AI principle that says all technical work is becoming agent-first. In practice, that means you should be using an agent for all technical work, and when it doesn’t get things right perfectly every single time, your job is to notice and do something about it.

What that has meant in practice: more linters everywhere, making patterns in our codebase more standardized, and writing down explicit guidance for Claude Code along the lines of “if you’re going to do X, use these paths.” We have the headwind of being a 15-year-old SaaS with all sorts of legacy in our codebase, patterns we’ve long since moved past but that still exist. We have to explicitly tell Claude about the modern way. What we do with humans is essentially the same: we onboard people by showing them how to get stuff done, what good changes look like, and which parts of the code to ignore as the old way. The pairing and unblocking process for Claude Code mirrors human onboarding.

Beyond socialization and guardrails, we’ve found that skills, currently the main unit of execution, need to be extremely small. Small, testable, provably good. Skills need to do the full job, because it’s very easy to build a skill that looks like it’s doing a good job but isn’t, and a bad skill is almost indistinguishable from a great one at first glance. The hard part is really pushing on what great looks like: fast, accurate, gets it right first time, tested against all the use cases and edge cases, something you can stand over.

We have a continual flywheel built into pretty much every skill. If something happens that suggests the skill should be updated, the instruction is to update the skill. Skills are self-improving as they’re used. It’s easy to go expansive and build opinionated, monolithic skills that try to do a hundred different things, but I think it’s similar to writing code: you want small, testable units that do one thing well and are understandable.

The quality bar has to be extremely high, because the worst case is you spend a lot of time building automation, everyone is very enthusiastic, but the skills aren’t reliable, and then people stop using them, ignore them, bypass them. People just want to get their job done. So we run a small number of extremely high-quality skills rather than a large volume trying to automate complex, sprawling work.

Model lock-in and platform optionality

In terms of building infrastructure around Claude Code, are you building in a way that allows you to port your harness onto Codex or other providers easily, or are you developing switching costs as you invest more into enablement of Claude models specifically?

We knew this when we decided to go all-in on Claude Code. It was a slightly easier decision in December because Opus was ahead, but we knew the frontier wouldn’t always be in one place. Maybe Opus isn’t the best for Ruby, maybe not the best for security, maybe Codex overtakes it on cost. We accepted that.

But this is a platform play. We don’t choose multiple cloud providers and say “Google has the best blob storage, Oracle the best MySQL hosting.” You get the most benefit from a single platform and the compounding benefits of a system that works together. Vibes-wise, Opus may not be the best right now, and you could argue benchmarks back and forth, but I’m not too worried, because we get more benefit from being on one platform and being able to prove we can reproduce work at extremely high quality.

If Anthropic shut down tomorrow, I don’t think it would be a gigantic task to move to Codex, or even to roll our own to a degree. It’s likely a single Codex session away. We do already use Codex in our environment, not as a general-purpose agent but for things like code review and reviewing the output of another agent. Using two different models for meta-analysis is a perfectly good approach to gaining more confidence in code quality.

So we’re being pragmatic, but we put a lot of weight on the fact that it’s hard enough to get one system to work well. It’s not worth trying to get everything working at the level of quality and repeatability we want across multiple providers. That doesn’t mean we won’t change providers in the future. Capabilities are converging, and the providers are all moving towards the same shapes. I’m not too worried about lock-in, but I’d be worried if we were trying to be agnostic for the sake of it. It’s like multi-cloud: you need really strong reasons to go multi-cloud, because you miss out on the benefits of going deep on one.

Org design in an AI-native company

How has your org chart changed in the last 2 years, in light of how many AI-native companies are operating as very flat structures?

In the last three years, yes, a lot. We went through a phase, like many other companies, of fewer middle managers, a higher ratio of engineers to managers, and updated expectations of managers to be more hands-on, more leading rather than just facilitating. That was happening in the post-ZIRP era at Intercom.

Then, through the building of Fin, Intercom went through dramatic change. We put 80% of R&D on Fin, moved away from our traditional help desk (we’ve since invested back in it, which is great) but we made very dramatic decisions about shifting people off the help desk into effectively greenfield areas. We dropped our tried-and-tested product team format and moved to what we internally called experience teams, more like factory project teams, way more execution-oriented, none of the baggage. We staffed them almost exclusively with tenured people who knew how to get things done inside Intercom, leaving traditional onboarding to other teams. Those have been really successful, both for focus and because it’s the right model when you’re building with AI: you need to be able to remove baggage, get people out of the way, give them a mission, and let them go.

We’re not as advanced as Anthropic, where there’s more of a research-lab flavor. We’re still a product company that wants to ship features on the roadmap. Sometimes we do experimental work, sometimes we don’t know if a feature is buildable, but we haven’t fully melted into a managerless setup.

The role of planning has changed significantly, though. We’re not doing big roadmaps anywhere now; you can barely plan six weeks out. We used to set high-level roadmap goals every three or six months and then figure out what to build in six-week chunks. Now we set goals every six weeks and figure out what to build, and that can be replanned within days. So planning cycles have shifted dramatically. The teams still exist, but the format, expectations, and scope of what people work on have changed significantly. We just haven’t completely broken up the org.

Measuring code quality

In your recent write-up reflecting on how far you’ve come, you mentioned using static analysis and various heuristics to measure code quality. Can you elaborate on that?

We’ve been working with a group at Stanford doing research on the impact of AI on engineering. We gave them access to our GitHub and they’ve been pulling data. It’s something we care about. We’ve always had a strong testing culture and well-understood patterns. We write code in well-understood monolith applications, so we know how to get most things done. I worry more about software architecture than the code itself.

We initially saw code quality start to decline, despite having more linters written this year than ever before and more tests written than ever before. The fundamentals had improved. We’d raised the bar on multiple quality metrics by putting AI on those problems and getting more repeatable software production by guiding Claude. But the Stanford research did indicate that quality was slipping in a few directions: greater complexity, some rework.

Then something turned a corner, and this was only in the last six weeks or so, when we happened to look back and check in. To a degree we’d already accepted that doubling engineering throughput might come with a small dip in quality; we wanted the acceleration. But today, code quality is increasing every single day. The overall quality of the codebase is going up. That’s a testament to the detailed work we’ve been doing on inspecting output quality, but also to things like our automatic approval process.

The automatic approval process basically says: if you produce a pull request that’s what you probably should have been creating anyway (small, using feature flags, with code observability, single-purpose, not in dangerous code paths) it gets auto-approved. People are bending their work towards the path of least friction, which is exactly where we want them to go. We use an LLM judge with all of our domain knowledge about what a great Intercom pull request looks like, and we’re now producing more of these than ever.

All those years of telling people what great looks like (the blog posts, the code quality books) that was great. But what has actually transformed code production is giving people the incentive to do the right thing through automatic approvals.

The unsolved problem I worry about is that we may be moving so fast that software architecture drifts into incoherency; you end up with something hard to move, manipulate, or reason about. I’m hedging on the assumption that we’ll soon be able to use agents to identify and solve those drift areas. But honestly, it’s nice to be worrying about higher-level issues rather than whether the code makes sense, or whether we’re producing something below what I’d expect from a senior engineer. That’s a nice problem to have.

The fully loaded cost of a PR

You recently made the point that enterprises thinking narrowly about token spend are underestimating the salary cost per PR. How would you advise other companies to think about the fully loaded cost of a PR?

Think about fixed and variable costs, and the cost of the work being done. There’s a strong parallel to what we saw with Fin. When we launched Fin at $0.99 per resolution, we got pushback and claims from companies that this wasn’t much cheaper than what they could achieve with their own customer support team. But when we measured the all-in cost of a conversation in our own business, accounting for software licenses, management overhead, office space, all the real costs, it was over $60. And that wasn’t even maxed out.

It’s hard to think about this because people don’t often do these full-cost comparisons. But the economics work out so strongly in favor of whatever can produce the highest-quality output without human intervention. If you have a CS agent that resolves at 70% and you’re competing against one at 55%, given the all-in cost of human resolution, the 55% vendor probably can’t give it away for free.

The same is true in our engineering environment. Remove as much human work as possible so you can focus on higher-quality work. People need to compare the throughput, quality, ambition, and lack of rework against the new state with token spend factored in. Token spend matters; we do worry about it, mostly to make sure we’re not being sloppy and to find inefficiency. But it’s incredible: we have an awful and ever-growing bill, and yet the cost per change is dropping. As a business, that’s exactly what you want. A black box you put money into and get features out the other side. We’re getting more features out per dollar, with no signs of that changing. We’re going to keep that flywheel turning so we can put in even more money and get even greater multiples of features out the other side.

Brian is hosting a meetup on Wednesday in London, join here!

• May 1st: basedcollective x unicorn mafia: demo night

• May 6th: AI-Native GTM with Clay

Last week, we hosted Alec Barber from OpenAI for a discussion on Evals, building on the first discussion we ran with Wulfie Bain last August.

The central thesis: build your own eval harness

Codex and Claude Code are powerful enough that you should build your eval harness yourself, tightly coupled to your AI harness, rather than adopting a generic evals platform.

The primitives problem. Every evals platform Alec has worked on has struggled to define a base “test case” that generalises across single-turn, multi-agent, and decision-tree architectures. His old startup built around a single-turn schema; users immediately wanted multi-agent, and the abstraction didn’t extend.

Where platforms do make sense. Observability tools (Langfuse etc.) are worth using. Don’t reinvent Grafana. Hyper-focused platforms (e.g. coding-agent-specific) can deliver real value because the primitive problem is bounded. And a platform that genuinely closes the loop end-to-end would be an exception.

The architectural implication. Design your AI harness for testability from day one: decomposable, inspectable, unit-test ready. Too many teams build scrappily and realise later they want evals. Alec’s recommendation: feed good eval-writing skills to Codex or Claude Code and have the agent build a bespoke harness.

Defining what a harness is. The harness is the layer around the model, e.g. the CLI for Codex or Claude Code are concrete examples. The CLI is the harness sitting on top of the model. Another example in insurance: documents go in, the system makes 20 sophisticated calls, applies regex at various steps, mixes deterministic and AI components, and eventually produces a pricing judgment. That whole system is the harness. The inference step is just one implementation detail inside it.

Evaluating the evals

Entropy as a signal. Run a single test case 10 times against the same model and grader. 10/10 pass or fail = low entropy, good signal. 5/5 split = high entropy, meaning the grader or test case is ambiguous. Costs 10x per case but the diagnostic value is significant.

Log-prob confidence scoring. An attendee spoke about a vertical AI product that uses token-level probabilities from the OpenAI Responses API to compute a heuristic confidence score. Low-confidence outputs route to annotators who improve the dataset on the fly. The confidence-vs-performance curve isn’t perfect but is statistically validated.

Dataset hygiene

A useful split:

• Regression set. Stable, broad coverage. Every change is tested against this.

• Iteration set. Small, focused on a current failure mode.

Fixes migrate from iteration into regression. Critically, prune the regression set over time: when a new model generation drops, some cases become trivially easy and just waste compute.

Binary benchmarks and saturation

Trajectory detail gets hidden in one number. SWE-bench is binary (solved or not), masking partial correctness. One approach is using LLM-as-judge to score each step of a trajectory, aggregate, and correlate with the final outcome.

Saturation. There’s a pattern where you can spend months building an eval for it to be saturated in weeks. Causes include genuine capability gains, contamination, and reward hacking.

The consensus was that the next frontier is production and real-world use, though this blurs the line between scientific benchmark and lab demo. Alec floated an economics analogy: maybe evals need to become an interpretive discipline, educated judgments from people who’ve looked at a lot of data.

Benchmark scores in SWE bench correlate heavily with other domains, raising the question of why you’d run 20,000 benchmarks when three capture the signal.

The domain-expert bottleneck

Building good evals for a domain (legal, aerospace, insurance) requires someone who is simultaneously:

• A software engineer (to build the infrastructure)

• A domain expert (to know what “good” means)

• A product/design thinker (to surface the right UX for evaluation)

These unicorns are rare.

One participant working in vertical AI found that domain experts kept saying “this is wrong” without articulating why. He built a small vibe-coded UI that mirrored the platform lawyers already used, letting them highlight right and wrong passages in their native context. The UI extracts tacit judgment in a form the eval pipeline can use.

Alec strongly endorsed this. The answer is bespoke domain UX — not generic eval platforms. If you’re building for legal, your eval UI should look like what lawyers already work in. This makes the problem a design problem as much as engineering.

Enterprise adoption and liability

Many enterprises still make eval decisions on vibes. Adoption was framed as friction-to-set-up vs. value-delivered. In regulated industries, value is high enough (cost of a legal or insurance error is enormous) that adoption is better. Compliance departments effectively force evals as audit artefacts.

Practical recommendations for founders

1. Design your AI harness for testability from day one. Decomposable, unit-testable, each component inspectable. Don’t build a blob and retrofit evals.

2. Build your eval harness yourself using Codex or Claude Code. The coupling to your AI harness is too tight to delegate to a generic platform.

3. Use observability and dashboards off the shelf. Langfuse, Grafana — don’t reinvent these.

4. Find good eval-writing skills online (he named Hamel Husain) and feed them to the agent as context.

5. Invest in the domain-expert UX. Build bespoke interfaces that mirror how experts already work, so you can extract their tacit judgment without training them on eval frameworks.

6. Maintain the regression/iteration split and prune stale tests over time.

7. Use entropy diagnostics to identify test cases where your grader is ambiguous.

• April 1st: Tool Use: A breakfast discussion

• April 22nd: Evals with OpenAI

• April 23rd: Mathematical Superintelligence with Harmonic

Subscribe now

A range of agent labs (Cursor, Intercom, Cognition and Decagon) released frontier vertical models in recent months, manifesting one of the many plausible end states for app-layer companies: vertically integrate through model training to reduce dependence on the big lans and protect margins.

The direction of travel was clear once companies like Thinking Machine Labs started abstracting infra primitives, as we covered last October:

Fine-tuning and RL are being productised and abstracted for agent labs to capitalise on their data and distribution advantages.

Just recently Thinking Machine Labs’ Tinker was the latest abstraction for fine-tuning, whilst OpenPipe (acquired by CoreWeave) announced a serverless RL product.

These abstractions pave the way for agent labs to start as API consumers of model labs → capture traces/evals → train narrow models (embeddings, autocomplete, router, policy) before attempting larger training runs with open-source models.

More infra providers have set out to catalyse this market - here’s how Prime Intellect described the vision for their recently released model training offering:

If every company had the same access to frontier training infrastructure, the collective creativity of the market would unlock far more breakthroughs.

We’re already starting to see enterprises and application-layer AI startups realize this. Cursor is beginning to post-train their own models optimized directly for Cursor itself as the RL environment, to gain more sovereignty over their product stack.

We want more application-layer companies entering the training game for every vertical of the economy.

There are two camps at the moment when it comes to how agent labs will focus their technical resources.

One argues that the examples set by coding and customer support will be followed by market leaders in other verticals, with research effort focused on training and updating model weights.

The second camp rails against any training, instead focusing research talent on agent engineering (e.g. context management, tool use, long-horizon tasks and everything else that makes a model production-grade for a task) or building the best ‘harness’.

It depends on the workload.

Workload-Harness Fit

Workloads or tasks vary by volume, value, verification properties and horizon, among other dimensions.

Volume — How many times per day/week does this task execute? This determines both the economic incentive to reduce per-query cost and the rate at which your data flywheel generates training signal. Intercom's 2M weekly conversations sit at one extreme; a boutique consulting firm's quarterly strategic analyses sit at the other.

Value per execution — What's the economic impact of each individual task completion? A customer service deflection might save $3-5. A correct medical diagnosis or a successful code deployment to production might be worth thousands. This determines how much you can invest in model quality per query and how much tolerance there is for error.

Verification Properties - Domains differ fundamentally in how verifiable their reward signals are. Sean Cai outlined three verification properties: veracity (how confident are you the signal is correct?), proliferation (how widely is the signal tracked and available?), and asymmetry (how rare and expensive is the expertise needed to judge correctness?).

Time horizon - How many sequential decisions, tool interactions, and context switches does the task require? A single-turn code completion is low horizon. A multi-file refactoring session with debugging is medium. A multi-week scientific experiment design is extremely high horizon. Longer horizons make both reward attribution and training rollout generation exponentially harder.

Just by looking at workloads through these dimensions, we can start to appreciate how agent labs will focus their research efforts.

• Cursor and Cognition’s workloads are high volume, high value (especially as enterprise penetration increases), moderate on verification, and increasingly long time horizon with cloud/background agents.

  • Given these qualities, pre and post-training with sophisticated multi-dimensional reward engineering was justified. The moderate verification means you need to invest heavily in constructing composite reward signals (correctness + style + efficiency + behavioural penalties). The medium horizons require infrastructure for long rollouts and self-summarisation. LoRA or more efficient forms of fine-tuning is probably insufficient here — Cursor explicitly does full-parameter updates because the bar is just that high when you’re trying to raise the ceiling of software engineering, not the floor.

• Intercom and Decagon's workloads are high volume, low to moderate value, clean verification, short to medium horizon. Each execution is low value individually but the aggregate volume is massive, creating strong economic incentive to reduce per-query cost. Verification is clean — the ticket was resolved or it wasn't. Horizons are short — typically single-turn or a few exchanges. The data flywheel spins fast because you get millions of labelled outcomes per month.

  • The volume justifies the training investment in a full post-training pipeline on a strong base open-source model through inference cost savings alone, let alone the performance improvements that Decagon and Intercom are showing. The clean verification signal makes RL tractable. The short horizons mean rollouts are cheap to generate. This is where the Intercom/Apex playbook works best, and where even LoRA-scale fine-tuning on platforms like Prime Intellect could deliver meaningful gains for smaller players. It’s worth noting Intercom was early to recruit a research team of scientists, ahead of companies like Decagon and Sierra.

• Harvey and Legora’s workloads are moderate volume, high value, moderate on verification, medium horizon. Volume is meaningful but not massive (thousands rather than millions of executions per week). Value per execution is high. Verification is moderate — an expert can evaluate quality, but it's expensive and slow. Horizons involve multi-step reasoning but typically within a bounded scope.

  • The jury is out on what approach is best. Harvey allegedly attempted training runs in the past, whilst Legora has stayed clear of any training and fully focused on building the best harness for Anthropic’s models.

The taxonomy of workloads governs which end markets justify training versus agent engineering. But knowing where you sit on the spectrum is only part of the evaluation, the other half is what it actually costs to execute. Cursor's Composer 2 technical report is the first detailed look at the infrastructure overhead required to operate at the rightmost end of this spectrum. .

The Training Imperative

The training pipeline

Composer 2’s development follows a two-stage process. The first stage of continued pretraining took an existing open-weight model (Kimi K2.5, a 1 trillion parameter model from Moonshot AI) and trained it further on a massive code-dominated dataset, progressively extending its ability to process longer sequences of code. This stage builds the model’s foundational understanding of programming languages, APIs, and software patterns. The second stage of reinforcement learning is where the model learns how to work as a developer. The model is dropped into realistic coding environments, given tasks derived from actual developer workflows (feature iteration, debugging, refactoring, code review, documentation), and scored on the quality of its end-to-end solutions. Cursor demonstrated empirically that the first stage reliably predicts the success of the second: lower pretraining loss consistently translated to higher RL reward, justifying the investment in pre-taining.

The infrastructure overhead

Composer 2’s training run spanned three GPU regions and four CPU regions globally. The RL phase alone required hundreds of thousands of isolated virtual machines (Firecracker VMs on Cursor’s internal compute platform, Anyrun) to simulate realistic coding environments, each capable of running a full development setup including a browser. These environments needed to be spun up at a rate exceeding 500 pods per second to keep pace with the bursty nature of RL training workloads. Cursor partnered with Fireworks AI for inference during training, running distributed clusters across the US and Europe. The team also wrote custom GPU kernels targeting NVIDIA’s latest Blackwell hardware, including a modified low-precision number format that they found necessary to prevent training instability. The contributor list runs to approximately 50 people! This resembles a global ML infrastructure operation.

Reward engineering and behavioural shaping

Several of Cursor’s most consequential design choices concern how the model is rewarded during RL training, and these reveal the depth of product-informed thinking embedded in the pipeline. Beyond the primary correctness signal, Cursor applies a nonlinear length penalty that penalises the model more steeply for unnecessary effort on simple tasks while tolerating extended reasoning on complex ones, incentivising the model to be fast when it can and thorough when it must. They layer in auxiliary rewards for coding style, communication quality, and product-specific behaviours, including explicit penalties for patterns like creating TODO items and leaving them unfinished. The team actively monitors for emergent problematic behaviours during training and introduces corrective rewards reactively — for instance, when the model began leaving chains-of-thought in code comments or collapsed to using only the terminal tool. The self-summarisation technique, inherited from Composer 1.5, allows the model to chain multiple generations with intermediate summaries within a single training rollout, with the final reward flowing back to upweight good summaries and downweight lossy ones.

Benchmarking philosophy: why CursorBench matters more than public scores Existing public benchmarks are increasingly unreliable indicators of real-world coding agent quality, with four structural problems: domain mismatch (SWE-bench focuses narrowly on bug-fixing), prompt over-specification (public benchmarks assume unnaturally explicit instructions), data contamination (OpenAI suspended SWE-bench Verified reporting after finding models could generate solutions from memory), and narrow evaluation scope (only functional correctness, ignoring code quality, latency, cost, and interactive behaviour). Cursor’s response is CursorBench, an internal evaluation suite built from real coding sessions of their engineering team. The structural contrast is stark: CursorBench tasks require a median of 181 lines of code changes versus 7–10 for SWE-bench, while task prompts are significantly shorter and more ambiguous — mirroring the underspecified nature of real developer requests. The benchmark is continuously refreshed as developer workflows evolve, with each iteration increasing in complexity.

Despite the enormous effort, training Composer 2 was imperative for Cursor.

Developers will pick the best harness for their workloads and Composer 2, a SOTA model at a fraction of the cost, has a chance at winning back both old and new workloads.

Short AGI?

There’s a real risk of obsolescence from large models, but that logic is not too different to saying two companies (or one, if you ask most people) will capture all of the value created by AI in the coming decades.

John Collison asked Bret Taylor half-jokingly whether Sierra is itself premised on a world without AGI:

John: Now we get to it, which is you described building stuff that you know you’re going to throw away because the model capabilities will get there and you’re occasionally... They are developing capabilities that you developed yourself. Isn’t Sierra itself “short AGI?” Sorry, I said I couldn’t resist.

Bret: The short answer is I don’t know. I mean, the fog of war in the software industry is pretty thick right now. I really believe in the applied AI market, though. I think most companies don’t want to buy models or buy software. They want to buy solutions to their problem.

I think there’s so much nuance in how these companies align themselves with different departments at these companies, solve their very unique problems in very specific ways. That is a mix of product, not technology, but product, go-to-market. It’s an ecosystem around it. I think a lot of that still exists because I’m not sure coding the software was necessarily the hard part.

I think the reason for it is most business users want actual solutions to their problems, and they want a company that serves their unique problems in a very specific and bespoke way. I actually am extremely bullish on applied AI. I actually think we could accelerate. I’ll make one statement, which is I think if we paused model development, we’d still have trillions of dollars of economic value.

I think not only am I somewhat skeptical that there will only be two companies in the world, I actually think one of the main things impeding adoption of AI is the lack of existence of all those other companies.

The models are getting better faster than any of us could have ever imagined, with the leaked details of Anthropic’s Mythos suggesting we’re about to experience another step-change improvement in capabilities.

But focus matters, as OpenAI has now admitted.

Startups will create enduring value and technical differentiation in some shape or form will play a role.

Sivesh and I recently hosted a roundtable on the future of software engineering with Anthropic’s Ash Prabaker and we were joined by engineering leaders from Stripe, NVIDIA, Microsoft, Google DeepMind, xAI, Apple, Scale AI, as well as the legend Peter Steinberger of OpenClaw/OpenAI.

Origins of Claude Code

The session opened with a retelling of the Claude Code origin story, much of which has been covered in public interviews. It began as a simple terminal UI in late 2024, was rough at first, and was built against a guiding principle of designing for where models would be in six to twelve months rather than where they were that day. Adoption was organic — an IC-driven project that scaled through demonstrated value rather than mandate.

The Recursive Improvement Thesis

A major thread throughout the discussion was “closed-loop” development. One participant described a setup at their company where bug reports are automatically triaged by an agent, bucketed by severity, checked against an eval set, and then a fix PR is opened — much of it running with minimal human touch. The room broadly agreed that this kind of loop is where compounding gains actually come from: better coding tools improve the models, better models improve the coding tools. Several people noted their companies are prioritizing coding specifically because of this dynamic.

How Workflows Are Changing

Participants compared notes on what’s shifting in their engineering practice:

1. Test-first has become the default. Multiple people said they now define test cases first and let the agent build against them — described as the only sane way to handle the volume of PRs being generated.

2. Two tiers of evals. One participant outlined their team’s approach: regression evals that must stay at 100% and run on every PR, plus frontier evals for new capabilities. Others in the room recognized the pattern.

3. Don’t mandate adoption. There was strong consensus here. One attendee described using competitions, hackathons, and casual incentives instead of top-down requirements — arguing that forced usage breeds resentment, whereas letting people see early adopters’ results drives proliferation naturally.

4. Code review is in flux. One participant admitted that human reviewers at their company often just click approve within minutes because the AI review layer has gotten good enough. When pushed on where this ends up, they acknowledged the mandatory-human-review model will eventually become inefficient — and suggested they may already be past that point for some repos. This landed with a mix of recognition and discomfort around the table.

5. Comments are back. A cultural reversal several people found amusing: engineers initially hated the verbose comments agents generated, but the consensus is now swinging toward leaving them in, because the next agent session finds them useful. One person put it as “we’re writing code for AI readability as much as human readability now.”

6. Life in the terminal. One participant described their personal workflow as: plan, verify the plan, implement via the agent, move on — without reading generated code line by line. This prompted some debate about when that’s safe and when it isn’t.

What Still Gets Scrutiny

Not all code is treated the same. Participants generally agreed that anything involving destructive actions (data loss, permission escalation) or core infrastructure deserves higher human review, while internal prototypes don’t need the same bar as public-facing code. Where exactly to draw the line varied by company.

The Bottleneck: Long-Horizon Tasks

The room converged on long-horizon tasks as the real frontier problem. One participant noted that product engineering has started to go exponential for them, but closing the loop on more complex research workflows isn’t there yet. The open questions everyone shared: what do you actually assign an agent for a four- or five-hour run? How do you observe it? How do you keep a human in the loop without babysitting? Nobody had a clean answer.

Infrastructure and Sandboxing

Discussion of how the industry has swung on sandboxing — first toward it for safety, then away from it for convenience, now back toward it with more nuance (remote coding agents, sandbox-per-session). The practical pain points people raised were compute for long-running sessions, permissioning, and enterprise deployment.

Observability and On-Call

One participant described early-stage internal prototypes where agents with access to logs, source control, and chat systems handle incident triage and debugging — reducing the on-call burden even though the systems aren’t production-grade yet. A side effect several people found interesting: engineers without infra backgrounds can now contribute to infra work because agents fill the knowledge gaps.

Context Management

Someone asked how you manage context at scale when thousands of people are changing things every minute. The honest answer from the room was that nobody has this figured out. One participant admitted their approach is basically unstructured — ad hoc chat threads that agents get MCP access to read, plus a strong writing culture but no formal documentation process.

A study was mentioned suggesting that pre-loaded markdown context files can sometimes underperform versus letting agents traverse the codebase from first principles. The counter offered was that this probably reflects stale or agent-generated context. The takeaway people seemed to agree on: human-authored context files help, agent-authored or stale ones can actively hurt. Humans have to supply the insight.

Hiring

When hiring came up, the most striking claim was that the trait one participant now screens hardest for isn’t raw engineering skill — it’s willingness to experiment constantly at the bleeding edge. Their best performers are the ones who understand model limits deeply enough to know when to trust the output and when to intervene. Another attendee noted their core infrastructure teams have stayed lean because AI-assisted cross-pollination lets product engineers contribute outside their usual domain.

SaaS Under Pressure

This got lively.

Participants traded stories about which tool categories they’ve replaced internally:

• Incident management — one person said their team ripped out their vendor because it was too complicated for how people actually worked.

• Auth layers — one participant claimed to have migrated auth systems several times in six months, each migration taking hours not weeks.

• Project tracking — someone is building custom UIs on top of their coding agent for managing engineering work, and floated that this whole category might be next.

• Internal micro-tools — link shorteners and similar utilities were the easy wins several people mentioned.

The pattern everyone noticed: it’s all developer tooling so far, because that’s where engineers have agency and speed. Business-facing software (CRMs, etc.) is stickier. One view was that incumbent business tools survive not because they’re good but because nobody has shipped a compelling AI-native replacement — just incremental add-ins.

A counterpoint from the room: the opportunity-cost argument (”we should focus on what we’re best at”) may always hold, which means labs might never prioritise building SaaS replacements over improving models.

The “Everything Is an Option” Problem

A startup founder in the room raised the flip side: because AI makes everything feasible, prioritisation is harder, not easier. Six months ago, rebuilding a tool internally was obviously not worth it. Now it takes a night. Teams get overloaded by the sheer volume of things they could do. Nobody had a great answer beyond defining clear swim lanes and giving individuals ownership of mini-companies inside the org.

Code Quality

When someone asked about code quality standards, the response was that the definition is shifting. “Good code” used to mean human-centric things — simple, easy to maintain, easy to contribute to. Now it has to account for AI readability too. The practical view from the room: strong regression evals and test-first discipline matter more than clean-code aesthetics.

Design Taste and Slop

“The purple gradient vibe” got a laugh — everyone recognized the AI-generated-UI aesthetic. The catch-22 someone identified: if you update a model’s taste profile, everyone uses it, and the new aesthetic just becomes the next generation of slop. Someone also noted that some models actively steer users toward particular frameworks, which functions as a form of lock-in.

Convergence Risk

One attendee raised a concern that everyone coding with the same models making the same suggestions will collapse the industry onto the same tools and patterns. The pushback was that this was a bigger risk with earlier model generations, which were much stronger at popular web stacks than at legacy or niche languages — and that gap is closing. Code modernisation of legacy systems came up as an area improving fast.

Background Agents

General agreement that the direction of travel is toward asynchronous background agents — remote sandboxes, monitorable from a phone, persisting across hours or days. One person noted that multi-hour autonomous runs are only recently becoming routine for them, having been experimental until not long ago.

Model vs. Harness

Asked how much recent improvement is model weights versus harness, one participant’s view was that both matter but on different cadences — big leaps come from model steps, and the harness philosophy should be “get out of the way of the model.” They described a stripped-down prototype — basically a current-gen model with a system prompt and bash access — that performs surprisingly well, which wouldn’t have worked a few generations ago.

Regulated Industries

Someone from a fintech background asked about regulated deployment. The room’s read: the most successful AI startups in regulated industries (legal tech was the example) are still fundamentally human-in-the-loop chat-with-document products. Nobody has made the jump to autonomous agents in regulated workflows. The bar is asymmetric — analogous to self-driving cars, where AI has to be dramatically better than a human to be accepted. Better explainability and structured audit trails were floated as the unlock.

Orchestrating Multiple Agents

Refreshingly low-tech answer: git worktrees and ten terminal tabs. More sophisticated orchestration is being built by third parties, but nobody in the room claimed to have it solved.

The Digital Transformation Irony

Someone observed that getting engineers to adopt AI tools — finding champions, overcoming resistance, managing change — is exactly the digital transformation problem other industries have faced for years. The irony of applying it to the engineers who built those transformation tools was not lost on the room.

The Arc of Programming Languages

Closing question: will agents start writing closer to the metal, bypassing the abstraction layers that exist for human convenience? The view was yes, eventually, but only when the model decides it serves performance — not because lower-level code is easier for models. Current models still benefit from well-structured, well-commented, human-readable code. Someone noted a trend toward Rust in startups, driven partly by AI flattening the learning curve.

Key Takeaways

• The recursive loop is real. Better coding tools produce better models, which produce better coding tools. Multiple participants said this is why their companies are prioritising coding.

• The bottleneck has moved from writing code to managing long-horizon tasks and deploying agents in regulated settings.

• Developer tooling is being displaced first. Business-facing software with network effects is holding.

• The human role is shifting from writing and reviewing to planning, evaluating, and steering — and the best performers are the ones who stay at the bleeding edge.

• Enterprise adoption is gated by permissioning, sandboxing, and regulatory caution more than by model capability.

• Slop and convergence are real concerns when millions of people use the same models to make the same choices.

• Context remains unsolved. Human-authored context helps; stale or agent-generated context can hurt.

Have any feedback? Reach out on LinkedIn or X.

Every public software CEO fielded the same set of questions about their right to exist in the age of agents. The discourse on the end of software rages on as startups substitute app software with custom tooling developed internally and labs productise aggressively in all directions.

This AI-pilled worldview would see most established enterprise software companies get wiped out and replaced by a combination of AI vendors and in-house builds.

I’m of course talking my book when I argue for the rotation of market share from incumbents to disruptors, but there’s more nuance to this than my X feed cares for.

Let’s look through recent earnings commentary from application software companies to unpack how they’re faring. I chose MNDY, HUBS, INTU, KVYO, BOX, and SAP as a broad basket covering SMB to enterprise apps.

Application Software

MNDY

FY2027 forecasts were scrapped as a result of higher uncertainty around MDNY’s historical strength in the self-serve SMB segment:

Given the evolving nature of the AI landscape and the choppiness in the no-touch demand environment, we believe it is responsible to keep our near-term communication focused on what we can execute and deliver with high confidence.

MNDY can’t reliably model its own self-serve funnel, which historically represented the majority of new customer acquisition.

The ZIRP-fuelled PLG days ended a while ago, but it’s still striking that MNDY deemed the no/low-touch channel structurally choppy.

Disaggregating the drivers of this choppiness is tricky, but commoditisation of the SMB offering by the AI labs is one plausible driver.

On the other end, $50k+ customers now represent 41% of total ARR and have 91% gross retention, whilst $100K ARR saw record net adds, and the $500K+ cohort grew 74% - the move upmarket seems to be working. It just might not be working enough to offset deceleration downmarket.

Margins came down a bit, reflecting investment in the upmarket GTM and AI, but pricing is also adapting:

So our AI capabilities are foundational in our platform. They’re embedded in our workflows. And based on the feedback we’ve gotten from customers, they really enjoy the predictability of PPU pricing, and they like to consume those capabilities that way. With that said, some of the more compute-intensive workloads that drive outputs with these workloads, we are charging and monetizing that through credits. and our customers like the mix of both of those.

The AI SKUs are: AI agents for workflow automation, Monday Vibe for application development, and Sidekick for information retrieval. Traction is still early.

HUBS

On whether HUBS’ data will be consumed by third-party agents and undermine the role of system of record, CEO Yamini Rangan’s answer focused on business logic and domain context:

Our strategy, as I just articulated, is to be that intelligent system of customer content and we have the data, but more importantly, we have the business context, the industry context and the domain context to deliver it. And that’s why customers come to us. They rely on us for that context. They want to use more of our APIs and partners want to customize and build on top of us. And as AI adoption accelerates, the value of our agentic platform increases.

Followed by (in my view, the weaker argument) determinism:

SaaS platforms are more than data. It is the logic, right? You can certainly get a nondeterministic output for a sales e-mail, but try taking a nondeterministic output for your sales forecast. That is not possible. It’s workflows like forecasting, routing, approvals, permissions, that is logic. It’s not data to be sucked away. And ownership accountability and governance, all of those lives inside applications.

Like MNDY, HUBS prices on seats and credits, with the customer support agent driving ~60% of credits consumed, with prospecting agent, data agent, and intent monitoring each contributing 10-15%.. Intriguingly, the support agent’s stated resolution rate of mid-60s is in the ballpark of private darlings like Sierra and Decagon.

Credits are not yet a material contributor to reported revenue, but the trajectory is clear: this is a FY2026 emerging tailwind that could become a FY2027 growth driver.

Management reiterated the core ICP of companies with 2 to 2,000 employees, with strong traction at the upper and multi-product penetration: deals over $5K MRR grew 33%, deals over $10K MRR grew 41%, and customers with 500+ seats grew fivefold. Multi-hub adoption among new Pro Plus customers hit 62%, with 40% of the installed Pro Plus base (by ARR) now owning 4+ hubs, up 6 points year-over-year.

As a sign of yet another AI-native buying third-party SaaS instead of building in-house, Lovable is a customer of HUBS - here’s Dharmesh Shah’s take:

So just because it’s possible. So we have, as Yamini mentioned, a large engineering team knows what they’re doing, spending 97% of their kind of calories using agent coding tools, they’re not doing it to replace internal platforms. So we think the best companies, both AI and non-AI will not be using Vibe coding to replace core systems. They’ll be doing it to add value to their customers. That’s what Lovable doing. I think that’s what some of the best companies in the world will continue to do.

HUBS is seeing the same declines in organic search as a channel as other software companies and has leaned into AEO.

INTU

Having announced apps on ChatGPT’s app store and a partnership with Anthropic to access Intuit products inside Claude, CEO Sasan Goodarzi addressed the elephant in the room:

And it’s also -- the thing I would point out is it’s why companies like OpenAI, companies like Anthropic look to the partnership with us because at the end of the day, they see and understand that this is a business that comes with a lot of liability and LLMs can’t just create the platform that we’ve created overnight.

Time will tell if relinquishing the interface to the end customer to the labs is a wise decision. The question of the value of incumbent apps where the product is primarily consumed by agents will only get louder in the future.

Taking liability is a recurring argument being touted in defence of vertical AI lately, a point INTU reiterated.

In our category, accuracy, compliance, security, reliability of financial decisions, and the liability that comes with it are critical to our customers. It’s our advantage and it’s why we win.

Management presented a narrative of AI enhancing Human Intelligence, i.e. AI-enabled services.

Our success rests on our powerful combination of proprietary data, domain-specific AI platform capabilities and AI-powered human intelligence, which we’ll refer to as HI.

Our system of intelligence combines AI and HI to deliver done-for-you experiences with accuracy, compliance, security, reliability and data privacy that create a durable competitive advantage. This foundation delivers what matters most to customers when it comes to financial insights, money management, taxes, bookkeeping and accounting, leading to complete confidence in their high stakes financial decisions.

AI-enabled services firms have raised boatloads of funding across law, insurance, accounting, banking and healthcare. The AI+HI traction INTU is seeing is a strong data point for the services-as-software thesis..

INTU’s agents are being rolled out at scale, with 3 million customers using them and repeat engagement of more than 85%.

INTU sees 3 monetisation levers: agents drive higher willingness to pay for subscriptions, agents drive cross-sell, agents drive higher services adoption (HI) - no credit-based consumption as less likely to see compute-intensive workloads relative to MDNY and HUBS.

Mailchimp continues to struggle, speaking to broader SME weakness that the MNDY no-touch performance already hinted at. INTU’s push into mid-market ERP is going much better, with new IES contracts growing nearly 50% quarter-over-quarter.

KVYO

KVYO’s business model is more naturally aligned with consumption.

Revenue scales with active consumer profiles and message/interaction volume, not with the number of humans operating the platform. When AI agents generate more campaigns, send more messages, handle more customer service conversations, and drive more transactions, Klaviyo’s revenue grows automatically because interaction volume increases.

When Marketing Agent enables a small team to run more campaigns without adding headcount, Klaviyo captures the increased message volume regardless of how many humans were involved in creation.

KVYO’s AI uptake might be one of the most underrated stories in public markets right now: for Marketing Agent adopters, more than half of campaigns are now AI-generated, performing as well as or better than manually created campaigns while taking significantly less time. Customer Agent resolution rates increased 20 points since launch, with monthly resolution volume up 50%+ since Black Friday/Cyber Monday. 85%+ all-time repeat engagement across 3 million+ customers using agents.

In keeping with this growth, the company is reframing its category - marketing automation is out, ‘autonomous B2C CRM’ is in.

Our technology marries the customer database we founded Klaviyo on and our robust marketing messaging infrastructure with agents for marketing and customer service that will autonomously create, deliver and optimize customer experiences on behalf of a business. And this agent layer that is designing and delivering experiences to billions of consumers is trained from our deep expertise and the trillions of experiences we’ve delivered for businesses already.

Enterprise traction accelerated meaningfully in FY2025. Customers with $50K+ ARR grew 37% to 3,912, with 349 net new additions in Q4 alone — beating the previous record by 25%. Customers generating $1M+ ARR doubled year-over-year. It’s no surprise KVYO hired ex-WDAY co-CEO Chano Fernandez as their new co-CEO. Another sign of the move upmarket is an Accenture partnership who are building practices around “marketing reinvention and service reinvention” powered by Klaviyo, targeting the largest consumer brands globally.

FY2026 revenue guidance of $1.501-$1.509B (21.5-22.5% growth) represents a meaningful deceleration from 32% in FY2025, but this assumes minimal contribution from AI.

KVYO’s answers on long-term durability were grounded in technical differentation rather than business context:

Our ability to be the agent and the platform of choice finds its roots in how our platform was built. This is our durable advantage. At the core is the database and data infrastructure, specifically built to handle the scale of consumer data and indexing and enriching through machine learning and serving hundreds of thousands of requests per second with millisecond latencies.

It allows Klaviyo to ingest, aggregate and govern first-party data in real time, so every consumer behavior, transaction, preference and consent is available to our users and now to our agents to deliver the best possible consumer experience. That foundation is coupled with our marketing platform, which not only provides high throughput, scalable systems to render messages, deliver them with excellent deliverability and enforce compliance, but also integrates with our customer data infrastructure to make last-mile personalization decisions on content, incentives, timing and channels at the moment we deliver a message or experience to an end consumer. Speed and scale matter.

Like INTU, KVYO supports Claude via MCP and has an app for ChatGPT, but remains confident in its infrastructure ensuring durability.

Lastly, the company is one of the best example of operating leverage and margin expansion through AI, with operating margins expansion 900bps YoY in Q4.

BOX

Aaron Levie is one of the most AI-pilled enterprise software CEOs and he deftly managed to tie file-based systems as the winning agent tool-calling paradigm to Box’s strength in enterprise content management:

Files are quite simply the native unit of work for agents. Agents use files to keep track of their work…

Thus, to have an effective AI agent strategy, companies fundamentally need a content strategy. They need a secure platform to manage critical content and ensure it can connect to all of their people, agents and applications. This is what we’re building at Box with our Intelligent Content Management Platform.

The narrative being proposed is that agent proliferation = inflection in volume of enterprise content that needs to be managed, governed, and secured. Every Claude Cowork session, every OpenClaw task, every custom agent workflow generates files that need to persist beyond the stateless agent session. The agent’s compute environment may disappear, but the documents it produced, the contracts it reviewed, the research it synthesised — all of that must be retained, governed, and discoverable.

The strategic implication is that BOX’s TAM grows with agent adoption rather than being threatened by it. More agents = more files = more content management demand. This is why Levie described Claude Cowork and OpenClaw as “universally good things” for BOX — every agentic knowledge work session creates content that needs a secure home.

BOX is seeing its AI SKUs drive enterprise penetration and expansion. Enterprise Advanced launched approximately one year ago and already accounts for 10% of Box’s revenue. The pricing uplift from Enterprise Plus to Enterprise Advanced has been 30-40% on a per-seat basis — at the high end of the 20-40% range initially anticipated. Total Suites customers now represent 66% of revenue, up from 60% a year ago.

Even so, top-line growth was still in the single-digits at 9%.

On monetisation, BOX sees themselves as serving agent demand through API consumption in a headless-first world, or more seats if human-in-the-loop use cases grow. The consumption model is still immaterial overall, but might be the future when the agent-first economy takes off.

SAP

SAP has leaned into the ‘Business Data Cloud’ with a framing that foundation models lack sufficient business context to be reliable. Citing the example of recently won customers like H&M, CEO Christian Klein emphasised SAP’s position as embedded across mission-critical workflows.

And so when I think about the future of AI and SAP, I’m super happy that I have our ERP. I’m super happy that I have our apps because without those apps, I wouldn’t have the data. And without the data, I wouldn’t have an AI.

Management also stressed their right to win against a backdrop of geopolitical turbulence:

While geopolitical and trade tensions have taken a certain toll on our top line performance in 2025, the growing need for sovereignty and resilience also offers unique opportunities for those vendors that could offer technologies and tools to reduce dependencies from dominant offering. As largest non-U.S. software, SaaS and PaaS vendor, there is no company better positioned than SAP to satisfy this rapidly growing demand.

Two-thirds of deals closed with AI in some shape or form.

Looking across MNDY, HUBS and SAP, there’s an argument for SAP’s position being strongest because of the criticality of the data they sit on: ERP, supply chain, manufacturing.

Conclusions

• AI impact on revenue still early, particularly the premise of agentic consumption of APIs or credits

• SMB segment showing weakness, indicating both limits of PLG and competitive threat from labs’ prosumer AI assistants

• Companies are either leaning on accumulated business logic as a durable asset in a future agent economy where software is headless, or on sheer infrastructure differentiation

• Enterprises still value governance and security of AI, as well as counterparties owning liabilities; will incumbents win on this vector?

• Partnerships with labs via connectors and MCPs not seen as major threat to enterprise value, even if surface area of product and customer interactions shrinks; no rev-share or data being transferred. How will these lines evolve?

Have any feedback? Reach out on LinkedIn or X.

Roundtables

March 5: The Future of Software Engineering with Anthropic

Last week, we hosted a roundtable on the future of AI compute with Michael, CEO of Spectral Compute, who have built a compiler that compiles CUDA source code directly to native machine instructions for non-NVIDIA GPUs. We were joined by attendees from Graphcore, Cerebras, DeepMind and more.

1. Nvidia’s Monopoly Position

Michael opened by framing the current state of AI compute as a “monopoly moment.” Key points:

• Nvidia controls 80%+ of datacenter GPU market share and briefly touched $5 trillion in market cap.

• Their dominance stems from a head start in accelerated computing that dates back to CUDA’s first release in 2006–2007. Nearly two decades of consistent innovation in parallel compute has created an entrenched ecosystem.

• Although open-source alternatives exist (e.g., Kronos Group standards, which Nvidia itself co-chairs), none approach CUDA’s adoption or defaultness.

• There are now hundreds of millions of lines of CUDA code in the wild. Developers have built deep expertise around CUDA over years, creating massive switching costs. Michael’s analogy: convincing the world to drop CUDA is like convincing everyone to adopt the UK power plug — technically superior, but practically impossible.

2. Why Vendor Optionality Matters

Michael argued that having alternatives to Nvidia is critical for multiple reasons beyond just cost.

Cost Leverage

• Nvidia was selling H100s at roughly 10x manufacturing cost, enabled by their monopolistic position.

• These margins are unsustainable long-term, but the market hasn’t corrected because of ecosystem lock-in.

Supply Chain Resilience

• Single-vendor dependency carries severe supply risk. When Nvidia misses production targets or its suppliers slip, customers are stuck.

• Worst-case wait times for latest Nvidia compute are 18+ months.

• Geographic prioritisation compounds the problem: North American customers get served first, APAC gets deprioritised, and Middle Eastern customers wait until policy deals are brokered.

Geopolitics

• Compute allocation has become a geopolitical instrument. Where you sit geographically determines your access to cutting-edge chips.

3. Current Challengers and the Competitive Landscape

Michael surveyed the emerging alternatives to Nvidia:

AMD Instinct Series

• Architecturally the most similar chips to Nvidia’s GPUs. The Instinct MI300 series is shipping, with the MI4XX series announced.

• Meta signed a deal with AMD to deploy 6 GW of Instinct compute by end of 2027 — a massive commitment, though execution remains to be seen.

• AMD GPUs are generally ~30% cheaper than equivalent Nvidia hardware, but people still aren’t switching en masse due to software incompatibility.

Cerebras

• Cerebras’ wafer-scale engine draws ~27 kilowatts but is more efficient on a per-token basis at scale.

• Cerebras announced a 500–750 MW deployment partnership.

• Crucially, Michael noted that Cerebras’s chip is not an ASIC — it is programmable and general purpose, which he considers a wise design choice nowadays.

Google TPUs

• TPUs are not purchasable outside of Google Cloud Platform; they are fully vertically integrated. You access them via GCP subscriptions.

• Programming model is fundamentally different: systolic arrays with data-flow architecture rather than traditional threads/warps.

• Writing kernels for TPUs works differently — there aren’t traditional kernels.

ASIC-Based Startups

• There are ASICs focused on transformers like Etched, Positron and even Taalas who baked the transformer architecture into chip-level design, claiming speeds like 17,000 tokens per second.

• Michael was critical of this approach: baking in transformer-specific logic creates the same depreciation cliff seen with Bitcoin miners (90%+ depreciation within 16–18 months).

• He questioned the practical utility of extreme token speeds: “Your coding agent is going to ask you dumb questions a lot faster” — the model quality running on these chips matters more than raw throughput.

• Current speeds on 8B models might scale to 250B+ models in the future, making specialisation worthwhile.

• Michael’s response: AI innovation is outpacing chip tape-out timelines. The field hasn’t settled on a final architecture — the transformer itself may be disrupted. Betting silicon on a fixed architecture is premature.

Other Notable Mentions

• Next Silicon (Israeli) — Michael was most enthusiastic about their approach: they’ve etched the selection DAG part of a compiler into the chip, a novel architectural choice.

• Axelera AI (European, embedded/small-scale inference)

• Michael referenced a list of ~50 startups globally attempting to build AI chips.

Microsoft Maia

• Microsoft launched Maia with an SDK access request form. Michael noted he still hadn’t received his SDK access, suggesting the chip may have incurred into delays.

4. The Depreciation Problem and Programmability

A substantial debate emerged around chip depreciation and what it means for investment decisions.

The Financial Reality

• H100 chips launched at $40k per chip; a single node costs $400k+.

• These are not disposable assets — buyers need confidence the hardware retains value beyond a one-year cadence.

• Bitcoin miners depreciate 90%+ within 16–18 months, creating a cliff that makes planning impossible.

Cerebras

If CUDA is truly so powerful and versatile, why does hardware depreciate so fast? In theory, CUDA’s universality should let you keep running older chips (e.g., multiple A100s instead of one H100).

Michael’s response acknowledged the tension:

• Nvidia has a financial incentive to drive new GPU purchases — they make money on new silicon, not on customers reusing old chips.

• There’s deliberate market segmentation: Nvidia explicitly wants H100s in certain deployments and not others. They don’t even use the term “heterogeneous” for their own chip generations.

• Additionally, hardware constraints make simple substitution impossible: precision (FP) differences between A100 and H100, and capped InfiniBand on A100s mean you can’t just swap older interconnects with newer ones for large-scale training.

The Core Argument

Michael’s central thesis: programmability and general-purpose capability are the most important chip design attributes because they preserve long-term value as workloads evolve. Application-specific chips face existential risk from architectural shifts in AI.

With tape-out costs at $20–30M minimum and 2–3 year chip lifecycles, a startup raising $100M gets essentially one generation of silicon. If the AI landscape shifts (e.g., mixture-of-experts changing distributed compute patterns), an architecture-specific chip is stranded.

5. Spectral Compute

The CPU Analogy

Michael explained the problem by contrasting CPUs and GPUs:

• In the CPU world, you write code once and it runs on AMD, Intel, or ARM without modification. You pick hardware on merits (price, performance, cache) without rewriting your software.

• This freedom does not exist for GPUs. Switching from Nvidia to AMD requires porting code, despite the chips being architecturally very similar. AMD calls thread groups “waves” where Nvidia calls them “warps” — substantively the same thing, but different enough to break compatibility.

The Spectral Approach

• Embrace CUDA as the de facto standard rather than trying to replace it.

• Let developers write CUDA code, pipe it through Spectral’s compiler, and have it compile directly to AMD native instructions (and eventually other architectures).

• This is not transpilation — it’s ahead-of-time compilation that goes all the way down to native hardware instructions, enabling deep optimisation.

• Developers can still hyper-optimise for specific hardware via #ifdef blocks for particular devices, but the default path works across all supported GPUs.

Why Direct Compilation Matters

• AMD’s CUDA equivalent (HIP/ROCm) is frustratingly different in subtle ways that break programs and is also slower in many cases.

• Transpiling to HIP would lose optimisation opportunities. By compiling directly to AMD native instructions, Spectral can reason about the full pipeline from high-level CUDA to transistor-level operations.

• A specific example: AMD uses opposite column/row format for tensor operations compared to Nvidia. Spectral’s compiler can detect this, prove that reordering is safe, and perform the transformation at compile time with zero runtime overhead.

Performance Results

• Spectral showed benchmarks comparing their compiler against AMD’s HIP toolkit. In several cases Spectral outperforms AMD’s own software stack; in others there’s room for improvement on both sides.

• They also compared against OpenCL: on Nvidia, OpenCL is behind; on AMD, it’s roughly on par. But there’s limited OpenCL code in the wild to benchmark against.

Coverage Status

• Michael showed their current CUDA API coverage, prioritised by real-world usage. Core CUDA APIs are substantially covered, including GrowMax operations.

• PyTorch support is their key milestone: early builds expected mid-Q2 2025. This is significant because PyTorch exposes nearly all low-level CUDA APIs to Python, so supporting it effectively means comprehensive CUDA coverage.

Business Traction

• Most traction currently in academia — researchers using some of the world’s biggest supercomputers.

• Working with a consortium of premier motorport teams on safety applications.

• Engagements in HFT/algorithmic trading.

• Goal for next 12 months: serious deployments with large AI labs and adoption by neocloud providers as the standard portability layer.

Team and Hiring

• 22 people, remote-first, clustered around Europe (6 in greater London, 4 in Edinburgh, 3 in the Netherlands, 2 in Greece, 1 in Denmark, 1 in Italy, co-founder in California, 1 in Singapore).

• Talent is extremely scarce: people who are both GPU architecture experts AND compiler specialists number roughly ~200 globally. Spectral recruits from universities and cross-trains.

Open Source Strategy

• Not currently open-sourcing the core technology. The “coup de grâce” is their vendor-neutral compiler optimisations, which they consider proprietary IP — “for the same reason Nvidia isn’t open-sourcing theirs.”

6. AI-Assisted Compiler Optimisation

A notable exchange occurred around AI’s role in GPU code optimisation.

• An attendee noted his team went from two months to two weeks of optimisation using AI-assisted coding tools at low-level GPU programming.

• He questioned whether traditional compilers could ever make the leap from naive attention to Flash Attention — something that took human researchers two years to develop.

• Michael’s view: the compiler “just isn’t smart enough yet” — nobody has written compilers to fully exploit GPU architectural possibilities. That’s a core part of Spectral’s R&D.

• He argued AI-assisted code generation will eventually outperform human hand-optimisation because it can brute-force search a much larger space of possible programs. Combined with fast correctness verification, this creates a direct path to highly optimized code.

• Importantly, that optimised code should be written in CUDA (not HIP or another niche language), for the same reason LLMs are better at English than Danish: the training data corpus is vastly larger.

7. The Cloud Hyperscaler Lock-In Dynamic

Discussion turned to how cloud providers are compounding the vendor lock-in problem:

• Google, Microsoft, and Amazon each want to control the full vertical stack — custom chips, custom SDKs, and proprietary APIs.

• Once customers build on a hyperscaler’s fully integrated stack, switching costs become prohibitive (similar to AWS’s original API lock-in playbook).

• The landscape will fracture further, not consolidate. Unlike the CPU world (Intel, AMD, ARM — three similar architectures), the GPU/accelerator world could see 20–30 different architectures.

• Each hyperscaler’s new chip generation introduces architectural changes, deepening incompatibility.

• This trend is likely to intensify as the industry shifts from training toward inference, where specialised inference chips from each hyperscaler will capture more market share.

8. Training vs. Inference Shift

A significant data point emerged in discussion:

• 60%+ of total compute used to train foundational models is now post-training (reinforcement learning), not pre-training. The pre-training dominant era is over.

• This RL-heavy workload can be disaggregated across data centers but still faces throughput constraints between clusters.

• This shift has implications for chip design and data center architecture — different workloads may favour different hardware configurations.

9. Data Center and Infrastructure Constraints

Cooling Capacity

• Cerebras’s wafer-scale engine draws 27 kW. Regardless of chip vendor, liquid cooling and high-power infrastructure is inevitable.

• However, the buildout is slow: companies like Cloudflare with hundreds of thousands of PoPs globally can’t upgrade all sites simultaneously.

• Meanwhile, the most common PCIe inference card (H100 NVLink) fits standard 600W PCIe slots. Nvidia hasn’t even updated this form factor for Blackwell. There’s an underserved niche for inference in existing data center infrastructure.

Multi-Data-Center Training

• It was noted that if you can link H100s across two data centers via high-bandwidth interconnects, you can approximate the performance of newer GPUs without ripping and replacing.

• Michael agreed this works for some workloads that can be disaggregated, but not all.

10. Key Debates and Open Questions

Will Nvidia’s Dominance Erode Through Interoperability?

• Developers won’t switch because alternatives aren’t faster; alternatives can’t get faster without developer adoption.

• Michael’s answer: some large players (Meta with AMD) are making the leap. Spectral’s role is to be the “universal Babel translation layer” that removes the switching cost entirely. Their roadmap includes supporting AMD’s ROCm libraries on Nvidia hardware — enabling developers to use the best of both ecosystems on either vendor’s chips.

Groq and Nvidia’s Acquisition

• Nvidia acquired a non-exclusive IP license to Groq’s core technology and hired their CEO and senior staff in late 2024. The deal was characterised as more than 3x Groq’s previous valuation, effectively an “acquihire” structure.

GPU Utilization Gaps

• There’s research showing traditional kernels only use 40–50% of Nvidia GPU compute capacity, largely due to memory-bound bottlenecks.

• Michael framed this as an algorithmic optimisation problem with multiple attack vectors: data compression, hardware changes, and compiler improvements. No single approach solves it universally.

• Nvidia’s tensor cores are themselves an example of the hybrid approach: an ASIC-like unit embedded within a general-purpose programmable chip.

The Cisco Networking Analogy

• There’s a parallel to Cisco’s networking dominance: Cisco created an entire certification ecosystem that made switching to Arista painful. However, customers with extreme performance needs (e.g., low-latency trading) will always seek alternatives.

• Cerebras signed with OpenAI specifically because OpenAI couldn’t achieve >1,000 tokens/second inference with other hardware and needed breakthrough performance.

Signals

What I’m Reading

A Level Headed Look at State of Software

The path to ubiquitous AI

Strategies for learning

Earnings Commentary

Have any feedback? Reach out on LinkedIn or X.

Gradient Descending Roundtables

February 25: The Future of AI Compute

March 5: The Future of Software Engineering with Anthropic

Last week, we hosted Matt and Thomas from Scale AI, following a previous roundtable on Rubrics as Rewards. We discussed RL environments, one of the bigger themes in AI this year.

Core Challenges in Building RL Environments

1. The Optimisation/Scale Problem. The ideal RL environment should be extensive and cover many domains, but larger environments are computationally heavier. There’s a fundamental tension between realism/breadth and efficiency. GPU constraints are significant across training, inference, and environment simulation simultaneously. Anyone doing large-scale RL environment training would face GPU, DRAM, and potentially CPU bottlenecks across all three vectors.

2. Verifiability Across Domains. Coding is the gold standard for RL because outputs can be programmatically tested. But many domains lack this property. Matt and Thomas described using “rubrics” — sets of 20-30 rules used as an LLM judge to score outputs in harder-to-verify domains — but acknowledged this still carries reward hacking risk, is compute-intensive (running two sets of inference at scale), and depends heavily on rubric quality, which requires expert curation.

3. Real-World Relevance. Environments must actually drive performance on things end customers care about. Building an environment around vending machines might not justify huge investment. The benchmark and the environment need to be jointly validated — you need confidence that improving the benchmark actually reflects improving real-world performance, not just benchmark hacking.

The Vending Machine Case Study

This was presented as a Scale AI internal hackathon project, partly inspired by an actual Anthropic experiment where Claude was given control of a real office vending machine (stocking items, web searching for suppliers, sending emails to reorder). The hackathon version was scoped down to fit a 2-day format.

Environment Structure

The environment was built around a simple loop:

• Observation (input to model): Current inventory (item slots, quantities, prices), available cash balance, current time, and a delta log of changes since the last step (deliveries, price changes, sales)

• Action (output from model): A JSON object specifying restock orders and/or price changes

• Step function: Takes the action JSON, updates the simulator state, returns the new observation

The model used was GPT OSS (with special tokens visible in the transcript examples). Reasoning tokens were artificially constrained because unconstrained reasoning allowed models to crush benchmarks.

Mechanics of the Simulator

• Demand was simulated based on price (lower price = higher demand)

• Operating costs were baked in so the agent couldn’t simply do nothing — inaction leads to consistent losses

• Expiration and other costs added further realism

• Delivery lag was simulated (restock orders appear in future observations)

Training Run

• Used GRPO (Group Relative Policy Optimization) algorithm

• Only 50 training steps — a very short run

• Eval: 10 independent runs of 4 days each (16 steps per run)

• Base model (pre-training): Lost ~50% of starting capital on average

• Post-training: Generated massive returns (example cited: ~£3,500 from £100 starting capital)

Key Finding — Reward Hacking vs. Environment Realism The team initially suspected reward hacking given the extraordinary returns, but upon inspecting actual generation traces, the model had genuinely learned good business operations. The real problem was environment realism: the simulation had no competition. In the real world, high-margin strategies attract competitors who arbitrage profits away. The model found the globally optimal strategy within the (unrealistic) environment, not a strategy that would work in the real world.

Reward Hacking Example Encountered Early in development, the team found the agent exploiting a loophole: one item had very low supplier cost but very high demand. The agent filled every slot with that single item at maximum margin. This was caught by manually reviewing generation logs — the entire output was just repeated restock orders for the same item.

Reward Design: Sparse vs. Dense Rewards

A significant portion of the discussion covered reward function design:

Sparse Rewards (traditional approach)

• Run the entire simulation, assign a binary 1/0 at the end (success/failure)

• Extremely compute-inefficient — you spend enormous compute for 1 bit of signal

• Poor for scaling training

Dense Rewards (what they used) Three improvements over sparse:

1. Continuous reward instead of binary: Use actual cash balance as reward (e.g., £100 profit → reward of 100, not just 1)

2. Intermediate step rewards: Assign small rewards/penalties at each step, not just at episode end. If the agent makes a small profit in step 3, that’s reflected in the step-3 reward. This provides dramatically more training signal per compute dollar

3. Curriculum/shaping: Adding human prior knowledge through reward structure helps models learn faster, though it introduces the risk of over-constraining behaviour

The tension highlighted: more intermediate rewards guide the model better but risk teaching it to optimise for the reward signal rather than the true objective. The answer given was that compute constraints force you toward denser rewards in practice — in an ideal world with unlimited compute you’d just use a single sparse end reward, but that’s not viable at scale.

Verifiability and Domain Expansion

Domains that work well with RL environments

• Coding (programmatic test execution)

• Computer use / tool interaction (did it navigate to the right page? Binary, immediate)

• Specific financial tasks (binary compliance checks)

Rubric-based approach for harder domains

• Construct 20-30 expert-curated rules

• Use an LLM as judge to score outputs against the rubric

• Scale has “PR Bench” (public) specifically for financial services and legal

• Risks: reward hacking via rubric exploitation, LLM judge biases, compute cost of dual inference

• In enterprise settings, they often build narrow domain-specific benchmarks rather than broad ones

Key insight on domain selection: For RL environments to be valuable, the benchmark must be: (a) something you can agree actually represents desired performance, and (b) tight enough that improving it reflects real capability gains rather than benchmark gaming. “Maximise profit in an undefined scenario” is too broad; “improve Rust coding on this specific task suite” is sufficiently constrained.

Reward Hacking Detection and Prevention

How it’s caught: Primarily by sampling generations and reading the raw model outputs manually. No automated detection was described as reliable.

Why it’s hard to prevent by design: Models find exploits humans don’t anticipate because they run thousands of simulations and inevitably discover edge cases in the environment logic.

Real-world example cited: Chinese labs reportedly hacked SWE-Bench (a coding benchmark) by looking at future git commits to copy solutions rather than actually solving problems — even frontier labs struggle with this.

Dual-benchmark approach: Run training against benchmark A, but also monitor performance on a separate benchmark B that targets the same real-world capability. If B collapses while A improves, strong evidence of benchmark hacking.

Discussion: Multi-Objective and Competing Goals

There are challenges with competing goals — e.g., a shopping agent maximising short-term basket size but damaging long-term customer retention:

• The environment itself doesn’t resolve competing objectives; the agent is free to explore any strategy

• The horizon is fixed (e.g., 4 days) and the agent learns whatever maximizes reward within that horizon

• Competing long-term goals (e.g., a B-corp optimizing for both profit and sustainability) are hard to encode — you’d need to design separate reward components, or accept that some goals require other training methods (RLHF rather than RLVR)

• RLHF (human preference feedback) is the approach for subjective domains (does this slide look good?) while RLVR (verifiable rewards) handles objective ones

Scalability and the Real World vs. Simulation Debate

Can you just use the real world as the environment? Why not use a real corner shop? The answer: you don’t have infinite compute to represent the real world, and the real world is too slow and expensive to run millions of times. The optimization is finding a simulation that is realistic enough to drive genuine performance improvement but constrained enough to be computationally tractable. This is described as an open problem for 2026.

Multiplayer/competitive environments as middle ground OpenAI Five (Dota 2) was cited as the canonical example — a rich, competitive, multi-agent environment that can be run millions of times. Video games were popular for this, but labs have moved away because real-world tasks (coding, office work) drive more economic value. However, for robotics and world models, game-like simulation environments are coming back into favor.

CPU as emerging bottleneck As environments become more complex (computer use, video editing, Linux environments), CPU becomes a constraint alongside GPU. Running heavy OS-level simulations at RL training scale could make CPU the binding constraint. One participant suggested rewriting environments in JAX to run on GPU, though this clearly doesn’t apply to high-fidelity real-world simulations.

Training Pipeline Efficiency

Emphasised as critical because RL is inherently sequential (run simulation → training step → repeat, cannot parallelise easily):

• Any inefficiency in any layer (kernels, model, environment simulator) compounds across the entire training run

• GPU utilisation must be maximised since RL idle time is extremely costly

• Environment step functions must be highly optimised to not block training

Multi-Agent Training Clarification

A point of confusion clarified in Q&A: the 10 evaluation runs shown in graphs were not separate agents learning from each other — they were 10 independent runs of the same model (to account for non-determinism). The training consisted of 50 gradient update steps where all runs contributed to shared model weight updates via GRPO. Post-training, another 10 independent eval runs were conducted.

Signals

What I’m Reading

Founders Should Chase Secrets

Time’s Up for SaaS (Grow Faster or Vanish)

End Game Play

In defense of vertical software

You’ve been kicked out of the arena, you just don’t know it yet

Earnings Commentary

Have any feedback? Reach out on LinkedIn or X.

Gradient Descending Roundtables

February 18: RL Environments with Scale AI

February 25: The Future of AI Compute

March 4: The Future of Software Engineering with Anthropic

Before Moltbook took over the internet this past weekend, the sell-off of public software companies reignited debates around the future of software.

Andrej Karpathy comments on recent advances in AI coding read ominously:

LLM agent capabilities (Claude & Codex especially) have crossed some kind of threshold of coherence around December 2025 and caused a phase shift in software engineering and closely related. The intelligence part suddenly feels quite a bit ahead of all the rest of it - integrations (tools, knowledge), the necessity for new organizational workflows, processes, diffusion more generally. 2026 is going to be a high energy year as the industry metabolizes the new capability.

The markets are spooked even as incumbents are crushing earnings estimates, as fundamental assumptions behind the quality of the software business model have come into question, as Jamin Ball wrote:

Mainly, confidence in the SaaS business model has shattered. SaaS businesses were long thought of as “cash flow annuities.” Loose money early on, flip profitable, and then every year print cash predictably. You could then calculate the “intrinsic value” of a SaaS business by summing the present value of every annual cash flow, with a terminal value assumption. More specifically, calculate the present value of the next 10 years of cash flows (discounted back to today), and make an assumption of the terminal value (ie year 11 onward).

AI is creating huge questions about what the future retention rates of these “stable” software companies will be. Software bears will say this platform shift will lead to deteriorating retention rates as companies leave behind legacy SaaS vendors for modern AI native alternatives. At the same time (and related), this is increasing the probability that the terminal value is in fact 0 for some companies.

Retention has stabilised, having compressed from 2021 levels.

Software companies have become cash flow profitable, but this hasn’t translated to true economic profitability on a GAAP basis given the impact of stock-based compensation.

In light of this, there are three scenarios being debated, all against a backdrop of a potential historic upgrade cycle across the enterprise:

1. Incumbents infuse just enough AI across their product surface area, combined with their distribution advantages, to drive renewals

2. AI-native disruptors win the upgrade cycle and displace incumbents

3. Enterprises build their own applications/agents

The most compelling argument against the latter is simple: focus.

Which leads me to the greatest irony of all: the companies that spend the most time and effort replacing SaaS with internal tools are the most likely to be disrupted. In business, nothing matters more than focus. The second you start expending engineering, product teams, and designers to reinvent the wheel, you’re distracting from your core competency, slowing everything and everyone down. Why do you think Netflix won the streaming wars? Why does Spotify lead in streaming music? Why does TSMC make all the chips in the world and not Intel? Focus. The best thing any business can do is invest time and effort in amplifying its own product offerings with AI. Replacing existing products to save negligible costs is a waste of time.

Followed closely by the delta in capabilities/total cost of ownership delivered by third-party vendors who are themselves singularly focused on their core product, versus software developed in-house.

SaaS is used because you get an exceptional product at extremely compelling financials; building it internally makes ZERO sense. The corporate version of Slack costs $18 per head per month. For a company of 1,000 people, you’re paying approximately $220,000 a year for Slack. $220K/year, which yields a proxy for roughly $75M in R&D effort annually. In other words, you’re earning a 340x amplifier for your money. SaaS has historically been an exceptional business because of how compelling the financials are for everyone. Slack is able to build an exceptional product because it focuses on one thing, their messaging app. Then they offer it at very compelling financial terms because they can leverage scale. It’s cheap for their customers while still highly profitable for them due to margins and scale.

JP Morgan’s internal tech team made similar remarks: AI is a TAM-expanding technology and the best use of incremental dollars is on the core business rather than rebuilding third-party apps.

Recent surveys suggest this sentiment still holds true in the enterprise.

For now, at least, the upgrade cycle is going to be contested by incumbents and disruptors.

Klarna’s narrative in the run up to its IPO is instructive.

Staking its credentials as an AI-native company, the company decided to churn from Salesforce, among other SaaS apps, and consolidate around a graph database built on neo4j.

Sebastian Siematkowski’s explanation nearly one year ago was prescient:

Key to our explorations became the conclusion that the utilization of SaaS to store all forms of knowledge of what Klarna is, why it exists (docs), what it tries to accomplish (slides, tickets, kanban boards), how it is doing (sheets, analytics), who is it dealing with (CRM, supplier management), who works here (ERP, HR) and what it has learnt was fragmented over these SaaS—most of them having their own ideas and concepts and creating an unnavigable web of knowledge that required a tremendous amount of Klarna specific expertise to operate and utilize.

Since then, incumbents have continued to erect walls around their data, and the concept of context graphs to collect company-specific decision traces has gone viral.

Directionally, it’s becoming clearer that enterprises want to remove silos and instrument the telemetry to capture decision traces that will enable agents.

That future, one where stateful agents have freedom to operate across a company’s data estate, will reward vendors who are able to build the best harnesses around foundation models for their domain.

The best harness will translate to the strongest capabilities, and by extension the highest ROI. Support agents with the best harness will deliver the highest resolution rate. Coding agents with the best harness will have the highest throughput of PRs that get merged. Legal agents with the best harness will be faster, cheaper and more accurate at reviewing contracts.

Incumbents have the distribution, ecosystem, long-standing CIO relationships and platforms. The startups taking share away from incumbents are leaning into the future enterprises want and shipping 10x better products. The gulf has to be big enough to win this generational upgrade cycle.

A number of debates will be settled in coming years.

What’s the right UI for knowledge work in an agent-first world?

Will UIs optimise for the experience of agent delegation rather than humans doing the work?

We’re moving at warp speed. Buckle up.

Signals

What I’m Reading

How to Bet on AGI

Lessons from Building AI Agents for Financial Services

eating lobster souls Part III (the finale): Escape the Moltrix

Intel Earnings, The Agentic Opportunity, Intel’s Mistaken Pessimism

A Straussian reading of The Adolescence of Technology

Earnings Commentary

Have any feedback? Email me at akash@earlybird.com.

Gradient Descending Roundtables

February 18: RL Environments with Scale AI

February 25: The Future of AI Compute

Today we’re speaking with Alex Shevchenko and Alex Stauffer, the Leads behind Ramp Labs. They released Ramp Sheets in November to a strong reception, with an exciting roadmap ahead.

Alex Shevchenko is the Engineering Lead at Ramp Labs. He’s been at Ramp for two years, working on the Applied AI team. He started on AI platform-level infrastructure (NVIDIA CUDA drivers, memory partitioning for model services) before moving into experimental projects like Ramp Tour Guide (an early computer use prototype). He set up Ramp Labs with Alex Stauffer.

Alex Stauffer is the Product Lead at Ramp Labs. He’s also been at Ramp for two years and previously was on the founding team at Actively AI. Based in New York, he leads the team responsible for pushing the boundaries of AI on the product and application layer.

The Mission of Ramp Labs

Alex Shevchenko: One of the big things when we were starting out is that there wasn’t a clear mandate given to us. We were free to roam and figure out things that we think are interesting in the applied AI space and to make products out of them. We already had some experiments in flight just from trying to make our finance team more efficient, which is actually the backstory for Sheets.

We had no proper guideline of “you need to do growth, you need to get this many followers, or you need to save this many hours for our team.” It was really just exploring things, and getting to something that’s useful in the long run. We don’t need to hit a specific metric this quarter. We can just build out some incredible design, some incredible product. It might not necessarily be useful right away, but it can then get disassembled into parts that will be used in Ramp to deliver more saved time, more saved money for our customers later on without us being constrained to really think every minute about our KPIs.

Alex Stauffer: To build off of that, one of our first projects was an AI form filler. This is a huge problem in finance teams. A lot of their time is just spent filling in forms whenever they need to wire money. It’s incredible how much wasted time there is. They’re just inputting the same company information, the same beneficial owner information all the time.

We shipped that product, we got a lot of impressions on Twitter and a lot of users. But then the broader story is that technology is being phased into all of the Ramp product today. There are 55,000 businesses on Ramp, millions of users. And now whenever it pains a finance admin to go fill in forms in any part of the product, this is being turned into an automated process so they don’t have to do that.

The Launch and Early Momentum

Alex Stauffer: We launched a little bit before Thanksgiving last year, during the holiday season, which was interesting timing. We got over 2 million impressions when we launched. We have over 10,000 users today, and a lot of the top VC funds, PE funds, top founders, small business owners; those are really the core users that we’ve seen on the product so far.

It’s really exciting. Users keep coming back. There’s a lot of organic growth just through word of mouth. We might throw some paid marketing at this soon, but right now it’s all been organic. And a lot of users are asking for more product features, and that’s what I’m really excited for. They’re giving us a lot of great feedback about how to improve the product.

For example, having multiple options for formatting, or you could have custom formatting based on how your company does things. So these are really interesting insights. The product is constantly being improved right now.

Alex Stauffer: Most of it is the standard use cases: small business owners, CFOs, small finance teams is definitely one cohort. And then there’s another that’s VC, investment banking, consultants.

In the first cohort, there’s 13-week cash flow and, for example, modeling out, “Okay, what if I hire three more people to the sales team this quarter? How does that affect my burn?” Another interesting point is we’ve seen founders use it to create valuations of their companies. That’s pretty novel to us. We did not expect that. These are things that have been self-reported from customers and through our user interviews; we can’t see your data at all.

On the investment banking side, a lot of friends in the IB space are signing up for this, even though they’re using their personal Gmails to not do it through their work. But it has been helping them quite a lot. There’s a lot of leveraged buyout models. These things take hours and days to create. And with Ramp Sheets, you can get a really good version of this in 10 to 20 minutes.

Why Spreadsheets?

Alex Shevchenko: This is a project that we started a long time ago, where we were tasked with improving the speed and efficiency of our own internal finance team. That’s a non-trivial thing to do because Ramp’s financial structure is very complex and very mature. So there’s not a lot of super easy wins to be had there.

When you think about it, you can find an accountant or a bookkeeper, sit down with them, spend a day explaining a task that normally takes them 30 minutes, and then take two days to automate it away. So you spend three days of engineering time to save 30 minutes of a finance professional’s time that happens every month. The math doesn’t work out there.

So one of the first things I tried to do was streamline the process of getting the context from the finance person’s brain into the engineer’s brain. We started out with a tool where they could record all of the actions that they take for a certain process. And with an LLM, we would process it and create basically a piece of documentation that would map out the entire process with an explanation and the tools they’re using. So you could take that and start writing a Python script or an automation for it much quicker.

Then the logical next step was, well, you already have all of this information. Maybe you can just use an LLM to vibe-code something directly on your behalf. So we started generating Python or N8N workflow automations from the videos.

We made this prototype and we came to Finance and presented it, and they said, “Well, this is really neat and all, but I don’t really trust it. This is very black box. The output is this Python thing that is very hard to verify.” And it also uses Python blocks or JavaScript blocks for automation. So you’re generating this artifact that is not verifiable by the professional. They don’t want to really use it because finance is very high-stakes. You can’t make mistakes, and the artifact that you automatically generate is not verifiable by them.

So we took a step back, and I was going through the Loom recordings of the processes that they sent to us. And I realized that you drop your cursor anywhere in that video and 95% chance that you’re in Excel. Finance professionals live in Excel. That’s all they do because it’s such a powerful tool and it’s the thing that they understand the best. As an engineer, I can read through Python code and my brain is set up for that, but their brain is set up to quickly parse an Excel file and get the information out of it.

So we decided to take this video automation and instead of generating Python, we’re going to generate step-by-step instructions for the Excel modifications so that they have a clear view of what is happening. They can audit it very quickly because they’re used to Excel, and then they can have it as a repeatable process.

Then for the public launch, we decided that the video modality doesn’t necessarily make sense for a lot of people. The finance team at Ramp is very strong and has their processes in place. But that’s not necessarily the case when you’re a startup founder and you don’t know what an example of a good model is. So we opened it up to be a text box. You just come into it and you explain what you’re looking for, and then it takes care of the specifics for you.

Building the Infrastructure

Alex Shevchenko: For something that’s running on Excel, the harness is OpenAI Agent SDK for the most part. We have modified it quite a bit, but that’s the core of it. The hardest piece on something that interfaces with Excel is just how complex Excel is. There are quadrillions of different configurations and settings—incremental cell calculation for self-referential formula chains. In general, just a random formula is also very complex.

Making those Excel calculations is really the hardest piece. I know that, for example, Anthropic Claude for Excel uses LibreOffice Excel to do those calculations. They’ll spin up a headless version of it and make the modifications in that. That’s not the approach that we took. We have a more complicated approach that I don’t want to divulge fully, but for the formula calculation and formula understanding—getting the calculated value versus the cached value (which is the number that you see compared to the formula)—those are the hardest pieces to get really right. Otherwise, you’d get a huge sheet with #VALUE!, #REF!, and just other errors all over the place. That was really the hardest piece to get correct from the technical point of view.

Alex Stauffer: There’s also a lot of design delights that went into this. We tried to make it as simple as possible. That’s very clearly shown in the designs. There are a couple of other similar products that are a lot more cumbersome and complex, and we tried to avoid that. A lot of those tools are made for IB and all these crazy workflows. We really wanted to make this just general for the consumer and then also those specific users.

We care a lot about design and product at Ramp. It’s what we’re known for. So we wanted to make it very sleek, very simple. We actually have a lot of new features coming out soon. We’re going to have templates, and this is actually going to supercharge Ramp Sheets. It’ll be quite similar to Granola recipes where we’re going to have a lot of pre-baked templates that are for those core use cases, and it’ll be super easy to fire off prompts.

One thing that we’ve realized with a lot of these AI products is that prompting isn’t super easy at times. The whole concept of the blank text box is quite jarring to a lot of people. People don’t know what to do, people don’t know what to say. We’ve tried to make this easy in the UX. We have some basic templates already that are more general-coded, but this is going to be a lot more specific and a lot better.

Views on Agent Infrastructure

Alex Shevchenko: That’s the hardest question to answer because it’s very case-by-case. I think whatever you’re building, a lot of the general chat interfaces are obviously going to have a ton of frameworks built around them. If you want to build out a customer support chatbot, there’s a billion different ways of doing it because there’s a bunch of providers that all have really good solutions.

But then there’s stuff that needs to be customized, like an Excel editor. If you want to create something for text-to-3D modeling or something along those lines. Obviously, in those cases, you will probably end up having to write a lot of custom code that interfaces between them.

In general, it’s really a gradient of how much hand-holding you get. The more hand-holding you get, the less customizable or the more finished-up solution you get. All of these are engineering trade-offs. Obviously you can move very fast on a finished solution from a vendor, but then you’re going to lose out on some of the customisability, and it’s case-by-case whether that helps you out or not.

I think one of the biggest things is sandbox providers. We use Modal for Inspect, for example. I think that’s a great product that is going to become very popular as we start letting agents roam much more freely. Letting them live in a bunch of sandboxes, you spin up one for each time you run something on the user’s behalf. Now it’s a sandbox within a Linux VM, but maybe they start giving them computer interfaces or something more complex. I feel like that’s one of the things I’m pretty bullish on.

The Roadmap for Ramp Sheets

Alex Stauffer: First of all, it’s going to be integrated heavily with the Ramp product. There are a lot of workflows today that finance teams are doing within Ramp, and they ultimately end up in a spreadsheet afterwards. We’re going to do a lot more to automate some of this and have them end up in Ramp Sheets with that AI agent that they can just go talk to and manipulate the data, which is a lot easier than having to do it themselves.

There are also a lot of really unique and creative product experiences that Ramp Sheets will support which Ramp is creating and will be shipping soon. You’ll see it coming soon, but it’s heavily impacting the product in a really positive and fast way.

And then for Ramp Sheets itself, it’s proving to be very successful and growing on its own. So we’re going to keep growing it and investing in the product and platform. We’re having more resourcing on our side to do that. The goal here is if we can provide a lot of value to a lot of new people and then have them learn about Ramp in the process, then it’s really a huge success.

Ramp’s Product Culture

Alex Stauffer: I think the biggest thing is that everyone here is very AGI-pilled. That’s probably the biggest difference between a lot of the top startups today and the ones that aren’t. AI is in everything. We’re using all the AI products available. Everyone is using AI to speed up their workflows. Claude Code and Cursor are upgrading everyone’s velocity for how fast we can ship.

We also have an internal agent called Inspect. Everyone is shipping. It’s really 10x-ing the velocity within the company.

I think another main cultural difference is that Ramp is still founder-led, and Eric and Karim are some of the best to do it. They want to grow Ramp into a massive business.

As a result, I think the culture is very non-bureaucratic here. It’s pretty flat. Everyone is an IC. Pretty much everyone’s shipping. And a lot of the top talent is here as well.

Shipping Velocity vs. Polish

Alex Stauffer: It’s the former. Our CPO, Geoff Charles, talks about this. We have a whole system of Alpha, Beta, and GA releases. The goal is to just get something in the hands of these alpha customers who have opted in, which are probably five customers/users of the product. Get their feedback really early. Tell us what they like and what they don’t like, and from there just go to the next version and then send that to the beta, which has more users, and then eventually GA the whole thing.

We think of velocity as actually enabling high quality product versus the other way around. The opposite is a bad habit to get into—I would actually describe it as an AI that just keeps thinking in loops. That’s the worst thing to happen where the customer isn’t involved. You just have a lot of iteration without actually understanding if you’re solving the problem. So that’s what we try to avoid.

Signals

What I’m Reading

The Death of Software 2.0 (A Better Analogy!)

Reflections on 2025

Meta Compute, The Meta-OpenAI Battle, The Reality Labs Sacrifice

Who Wins the AI Race?

The Cyborg Era: What AI means for jobs

Earnings Commentary

Have any feedback? Email me at akash@earlybird.com.

Hey everyone,

Hope you had a good break over the holiday period!

I took time off from writing to recharge. Time away from the constant stream of AI news was a priceless way to mentally reset, step back, and think about the ‘bigger picture’, as they say.

Step back to 2015, when OpenAI was just founded. Today, much of the global economy’s growth depends on AI and its attendant infrastructure buildout. There are no benchmarks that can’t be beat.

What progress will the next 10 years hold? It’s perhaps terrifying and captivating in equal measure.

I have two primary vehicles for thinking about the future: writing and community. I’m excited to resume both writing and the community building that we started last year with the ‘Gradient Descending’ series of events.

As always, I’m grateful to each and every reader for joining.

1. Mid-Training: Enterprises Shop Model Checkpoints

Model training will become more of a continuum.

Enterprise customers will increasingly be able to choose their preferred model checkpoints to inject their internal data.

As Ben Thompson wrote of AWS’ Nova Forge offering:

Right now you have two ways to incorporate your company’s data into an AI model: first, you can use RAG to basically have a model search your company’s data in the context of providing an answer. Second, you can post-train a model on your company’s data. The shortcoming in both approaches is that your company’s data isn’t actually in the model, which can lead to unsatisfying results.

Nova Forge is an offering built on AWS’s internally produced AI models; because they own the Nova models, they own the training checkpoints. What you can do with Nova Forge is choose a checkpoint — say, when the model is 80% trained — and infuse your company’s data at that point, so that the data is integrated into the model itself, and not simply searched or trained-in after-the-fact.

As more practitioners seek the bare ‘cognitive core’ capabilities of language models without the drawbacks induced by low-quality internet data, it’ll be interesting to see how this market evolves to cater to this demand.

Will customers choose models at the 80% training stage, or will they desire even earlier checkpoints for the above reason? It depends on how much transparency will be provided into the training corpus. Then again, that level of disclosure hasn’t been met by most of the ‘open source’ model providers who’ve chosen to only go as far as making their weights open.

2. Advertising Finally Arrives

OpenEvidence, rumoured to be raising at a $12bn valuation, grew from $8m ARR in 2024 to c. $150m (!) as of the end of November 2025.

This astonishing growth was purely advertising revenue. This model enabled viral adoption by clinicians, as the platform now serves 15 million clinicians per month (40% of the US market). Not bad for a four-year-old company.

OpenAI’s recent Gemini-induced Code Red mode delayed their roll-out of advertising, but as Ben Thompson has argued many times, advertising is the inevitable internet business model.

xAI’s position in the ‘Muskonomy’ (SpaceX/Starlink, Tesla, X) may yet see it become more competitive in the consumer market, but for now I see it as a two-horse race between Google and OpenAI. Both are still aggressively competing for market share, to be sure, but engagement metrics will continue improving as agentic commerce takes off (and more capabilities like memory are unlocked/improved).

Advertisers will finally be able to occupy the valuable ChatGPT/Gemini real estate.

3. European Industrial Policy Gains Momentum

Kanishka Narayan’s (UK Minister for AI and Online Safety) vibeshift narrative is much needed.

It’s easy to be cynical about Europe’s prospects, but I’m taking a glass half-full view and believe that policymakers are going to invest heavily across the AI supply chain.

From faster planning permissions, to new sources of energy and to data centre construction in AI Growth Zones, the physical, atom-based constraints on AI need to be lifted in order to realise accelerated GDP growth.

The EU’s AI Continent Action Plan has a number of promising proposals, including the The Cloud and AI Development Act which aims to triple the EU’s data centre capacity within the next five to seven years. The Plan also weighs up the importance of data foundries, proposing a Data Union strategy that ‘will, among other measures, explore the development of Data Labs as integral components of the AI Factories, to enable the provision, pooling, and secure sharing of high-quality data.’

It’s reasonable to be sceptical of sufficient follow-through on these recommendations, as was the case recently with the Draghi report. I’m naive enough to believe this period is different.

It has to be.

4. Breakout Consumer Agents In China

As I’ve written before, some of the world’s biggest consumer AI apps after ChatGPT are in China. Meta buying Manus (originally HQ’d in Beijing) is an indication of the cutting-edge agent harnesses being built in China.

The antecedents of China’s hyper-competitive market are too lengthy to labour on here, but the resulting environment is one that is: heavily oriented towards consumer apps, big tech are constantly entering new business lines and competing with each other, business models are heavily reliant on advertising rather than subscriptions.

Upcoming IPOs of labs like Z.ai and Minimax and competitive model releases from companies like Xiaomi are consistent with China’s focus on efficient models that can run on devices (and, importantly, in factories).

Freda Duan wrote about the impressive Doubao Mobile Assistant from ByteDance in December:

The key idea: “operating-system-level collaboration between Doubao and mobile phone manufacturers.”
Meaning: this is not a standalone app. It is a system service baked into the ROM, able to drive the UI like a human across apps, mostly via real-time voice.

“Find which KFC spicy chicken burger is cheaper on JD vs Meituan vs Taobao, pick the lowest price, deliver to X address, add note ‘leave at front desk,’ then screenshot the order and send to A on WeChat.”

The agent literally opens and navigates multiple shopping apps, compares prices, and completes the entire workflow.

There’s been lots of speculation around whether Google would lean into Pixel more heavily as they would also be well placed to deliver this kind of experience. As Apple’s struggles continue, it might be a Chinese company that sets a new high watermark of consumer agents.

5. Memory Cracked And Portable Context Wars

‘Sign in with OpenAI’ has been a long time coming, given the rich semantic and behavioural data collected in the last three years.

These ‘behavioural signatures’, as my friend Antoine Moyroud described them, could unlock radically better personalisation in software:

Logging into a new tool will no longer feel like introducing yourself from scratch but more like picking a memory provider that brings your context with you. Your preferences, tone, goals, personality, all imported with a simple click.

The exact architecture for how memory will be stored, updated, and accessed is still being refined, with different approaches representing different trade-offs.

Google introduced ‘test-time memorisation’ in December, a way for new ‘surprising’ inputs to be stored in long-term memory. I expect to see more developments like this that go beyond improved summarisation of past conversations.

The portability of context/memory is contentious as in theory it’ll reduce switching costs for consumers. The labs will continue erecting walled gardens around the metadata they’ve collected on you, allowing you to use it to onboard onto new software but not export the raw data. New UI paradigms will emerge for how consumers manage their memory repos.

Open Questions for 2026

1. Will AI app markets start consolidating? Legal, healthcare, finance and several other end markets are heavily capitalised with efforts to kingmake doing little to restrict more capital flooding into these markets. The same can be said for markets like customer support and go-to-market software. We’re 1-3 years in for several consensus categories - will 2026 be the year of culling as the winners pull away (defined as companies with durable revenue, talent vortexes, and early upmarket penetration, among other things)?

2. Will credit funds continue financing the AI infrastructure buildout? The scale of capex today has already far surpassed the free cash flow thrown out by big tech. There have already been signs that the reliance on external capital may be untenable.

3. Will operating leverage hold as the 2022/2023 class of AI companies continue scaling? To date, the best AI companies have crushed the leading software companies of the past decades when it comes to metrics like ARR per FTE, OpEx per FTE, and so on. How will these numbers hold up as they scale?

4. Will new distribution channels gain traction? Founder-led content has helped companies like Lovable break out of a noisy market by building a brand directly with users and potential hires. Products with a prosumer motion might yet iterate on ways they can leverage AI more effectively, e.g. for demos. In the enterprise, if the role of an FDE will continue, how will it evolve and become more specialised?

What questions are you thinking about? Shoot me a note.

Signals

What I’m Reading

Portable Memory & Behavioral Signatures: the missing layer for AI personalisation

2025 Letter by Dan Wang

2025 Letter by Zhengdong Wang

TPU Mania

Earnings Commentary

Have any feedback? Email me at akash@earlybird.com.

Last week we hosted David and Eric from Cursor to discuss Cursor’s approach to benchmarking models, product development, the future of IDEs and CLIs, and more.

Cursor 2.2 Launch

1. Debug Mode

• Collects runtime context by automatically adding logging statements to code

• Agent reproduces issues and analyses actual data flow

• Launches a Node.js server locally to capture logs across multiple repos

• Works across different languages (Python, JavaScript) via HTTP requests

2. Plan Mode Improvements

• Emerged from developers using markdown files to steer models

• Allows upfront alignment between developer and AI before code execution

• Treats plans as markdown files that agents can search and reference

• Plans may evolve into long-term memory/context for projects

• Moving toward storing all artifacts (plans, chat histories) as files rather than in databases

3. Multi-Agent Judging

• Runs same task across multiple models simultaneously

• Uses LLM-as-judge (currently Opus 4.5) to recommend best solution

• The judge model sees original prompt and outputs, not tools calls or thinking process, not implementation details

• Future: Tournament-style bracket judging for 8+ models

• Already shown to outrank any individual model in Cursor’s benchmarks

Technical Philosophy & Infrastructure

Context as Core Focus

The team emphasised gathering context at two critical points:

• Pre-generation: Plan mode for upfront alignment

• Post-generation: Debug mode for validation

“Everything as Files” Strategy

• Models excel at reading/writing files

• Converting all artifacts to files enables agents to grep and semantic search

• Chat histories moving from SQLite to markdown files

• Leverages existing primitives agents already understand

CursorBench (Private Benchmark)

• Internal benchmark using real software engineering tasks

• Must remain private to prevent training data contamination

• Currently tracks best single model vs ensemble approach:

• Better predictor than public benchmarks (SWE-bench now potentially in training data)

Model Evaluation Insights

Quality Metrics

Two key signals tracked:

1. Code persistence: How long generated code remains in codebase after commits

2. Sentiment analysis: Next message indicates success (topic change) vs. failure (error logs, continued same issue)

Model Usage Patterns

• Internal data shows many engineers are using Tab less, and more agent-heavy workflows

• Sonnet family most popular, with spikes when new versions release

• Brief Gemini adoption, then return to Claude/OpenAI models

Development Culture & Process

Internal PMF First

• Ship features internally before external release

• Debug mode initially built on weekends, faced skepticism

Rapid Iteration

• Minimal RFC process for most features

• Each person is empowered to ship internally when they have ideas

• RFCs mainly for context/knowledge sharing and alignment

• RFC writing now valuable because agents can implement from specs

Sub-Agents

• Had sub-agents in April but no internal PMF

• Bringing back with Composer model (fast, nearly Sonnet-quality)

• Models improved enough to make the experience acceptable

Product Strategy & Roadmap

Interfaces (IDE, CLI)

• P0 (primary product) is to automate coding - the interface will evolve over time

Computer Use for Validation

• Using computer use to verify code changes work correctly

• Example: Bug reported on Slack → Cursor fixes → Returns video proof → PR merged

• Async workflow advantage - not waiting for human to test

Other Features

• Deep Links for sharing: Commands and rules shareable via links

• Voice Input: Already working well, working on voice output back from Cursor

Future Features Discussed

• Extension Marketplace: Reuse VS Code marketplace for commands, rules, MCP servers

• Video Input: Waiting for model providers to add capability

Technical Deep Dives

Tool Management for MCP

Solutions being explored:

1. Lazy loading: Agent sees tool list, loads definitions only when needed

2. File-based discovery: Tools stored as files, agents grep/search

3. Code Mode: Wrap MCPs as APIs (models better at APIs than MCP definitions), enables programmatic chaining

Context Window Challenges

• Opus 4.5 only 180K context window (surprisingly small)

• Team stops using chat at ~60% capacity (quality degrades)

• Preference for one-shot prompts vs. multi-turn conversations

• Summarisation degrades quality significantly

Other Insights

BugBot Feature

• Automatic PR review bot in Cursor

• Intentionally catches fewer bugs than competitors

• Reason: Filtering false positives - too many warnings = users ignore them

• Custom model trained for false positive detection

Internal Tools Usage

• Can see internal data on feature usage, model preferences

• Agent quality team (8 people) solely focused on harness/evals/tools

Competitive Positioning

• Cursor Bench used as de facto model ranking

• Model selector in IDE trusted more than public benchmarks

Signals

What I’m Reading

Titans + MIRAS: Helping AI have long-term memory

Portable Memory & Behavioral Signatures: the missing layer for AI personalisation

TPU Mania

OS Agent | Byte’s Mobile Assistant & Implications

B2B “AI products” that are really services in a trenchcoat

Earnings Commentary

Have any feedback? Email me at akash@earlybird.com.

Subscribe now

Share

Leave a comment

Upcoming events in London

Last week we hosted Vlad from Alibaba Cloud to discuss the Qwen model family, which are some of the most heavily used open source models.

Alibaba Cloud

Alibaba Cloud positions itself as one of the few public hyperscale cloud providers with a complete, vertically integrated AI stack. The company operates 92 data centres globally across 29 regions.

Current European presence includes five data centres (three in Frankfurt, two in London), with upcoming regions in France and Netherlands. The company was founded in 2009 as part of Alibaba Group’s IT team before becoming a standalone cloud business focused on “user first and AI driven” strategy.

Alibaba Cloud maintains its own homegrown technology stack from the operating system level up. This includes custom kernels, compute infrastructure, storage services, networking, and security layers.

The Qwen Model Family

Core Philosophy

Qwen represents Alibaba’s commitment to open source AI development under Apache 2.0 licensing, providing significant freedom for commercial use. The models are designed to balance performance with efficiency (reflecting the broader Chinese AI ecosystem’s adaptation to compute access).

Model Variants & Specifications

Text Models:

• Dense models: 0.6B to 32B parameters

• Mixture of Experts (MoE): Qwen3-30B-A3B 32B and Qwen3-235B-A22B (with 22B active at inference)

• Qwen3-Max: flagship commercial model exceeding 1 trillion parameters

• Qwen3-Next-80B-A3B: 8B parameter model with 3B active parameters

The variety in sizing allows developers to choose optimal performance-to-resource ratios for their specific use cases.

Multimodal Capabilities

Qwen3-VL (Vision-Language):

• Supports 256,000 token context window for large video processing

• Near real-time performance (5-second delay) for video analysis

• Applications include video labeling, tagging, content moderation, quality management

• Particularly strong with complex images like tables and diagrams

Qwen-Audio:

• Speech recognition and audio processing

• Pairs with VL model for comprehensive multimodal applications

Qwen-Omni:

• Experimental single model for all modalities

• Goal is eventual AGI-like architecture

• Demonstrated in edge computing scenarios (mobile devices)

Qwen Coder:

• Specialised for code generation

• Trained on massive code base during second phase of model training

• Engineers had to drop down to PTX (low-level instruction set) for optimization

• Competitive with leading code models

• Available across multiple IDEs

Reasoning & Thinking Models

Qwen initially released a hybrid reasoning/non-reasoning model but later separated them based on community feedback. Key innovation is the “thinking budget” feature that allows users to define token limits for reasoning. Once the budget expires, the model automatically switches to instruct mode, optimising GPU resource utilisation.

The flagship Qwen3-Max commercial model ranks competitively with OpenAI, Claude, and Google Gemini on major benchmarks, though the competitive landscape shifts rapidly with frequent releases.

Multilingual Support

Qwen3 supports 119 languages and dialects, making it effectively universal for global deployment. This represents a major breakthrough from Qwen2.5’s ~30 language support.

Generative AI: The Wan Family

The Wan family handles video and image generation with multilingual capabilities. Wan 2.2 is open source under Apache 2.0, while Wan 2.5 is commercial.

Capabilities:

• Text-to-video, text-to-image

• Image-to-video, video-to-video editing

• Currently supports 15-second video generation

• Performance comparable to Google Veo and Sora

• Brand customisation for organisational alignment (colours, fonts, shapes)

Use Cases:

• Social media content generation

• Broadcasting and sports content

• Corporate marketing materials

Commercial vs Open Source Distinctions

Development & Training

Commercial Models:

• 36 trillion tokens for general training

• Continuous post-release tuning and updates

• Enhanced security and content generation support

• Extended context windows up to 1 million tokens

• Access to caching and optimisation features

Open Source Models:

• ~20 trillion tokens (roughly 1.5x less than commercial)

• Fixed at release date, no continuous updates

• Context windows: 32K to 128K depending on configuration

• Community-driven development and derivative models

• Self-managed training and fine-tuning

Support & Pricing

Commercial:

• Token-based pricing through Model Studio

• Full technical support from Alibaba Cloud

• Fine-tuning tools and evaluation models (in pilot, coming soon internationally)

• Available in Singapore, Hong Kong;; Europe in 2026

Open Source:

• Free to use and modify

• Community support

• Self-hosted with user-controlled infrastructure

• Derivative models emerging (Japanese, Arabic language variants)

Technical Infrastructure & Deployment

Model Studio & Platform-as-a-Service (PaaS)

Model Studio provides fully managed AI services combining:

• Qwen family models (text, vision, audio)

• Partner models (DeepSeek, Glm, Kimi, Llama, others)

• Token-based consumption across multiple models

• Agent orchestration capabilities

• MCP native support

PAI (Platform for AI)

Elastic Algorithm Service enabling:

• Few-click deployment of latest Qwen models on GPU infrastructure

• GPU pool creation and load distribution

• Fully managed infrastructure

Vector Database Options

Multiple managed services:

• AnalyticDB for Postgres (with vector engine, built on Greenplum)

• Hologres (MPP database with vector engine, homegrown product)

• Elasticsearch and OpenSearch (managed services)

• Milvus (fully managed)

Choice depends on use case: AnalyticDB works with larger data platforms (MaxCompute), Hologres optimised for specific workflows.

Agent Framework & Tools

Qwen Agent (Open Source):

• Platform for agent orchestration

• Supports planning, decision making, information management

• Integration with various tools and functions

• MCP-based tool selection and execution

Qwen Guard:

• Enterprise guard-railing solution

• Uses LLM to control input/output

• Customisable policies with default integration

• Protection against prompt injection and other attacks

Additional Tools:

• Qwen3 Embedding and Reranker

• Open source RAG framework

• OCR capabilities (Qwen OCR for text-heavy documents)

Key Discussion Points & Insights

Adoption Patterns

The community shows fast adoption of Qwen open source models. Major platforms like HuggingFace integrated Qwen models into HuggingChat based on technical merit rather than advocacy. Companies like Airbnb have said that they use OpenAI for pilots but switch to open source models like Qwen for production due to cost efficiency at scale.

Derivative models demonstrate strong community engagement, particularly language-specific implementations (Japanese model topped charts for extended periods, Arabic implementations showing strong performance).

Performance & Optimisation

Performance varies significantly across hosting solutions. Alibaba Cloud uses its custom acceleration technology (PAI-Lingjun) but doesn’t test Qwen performance on competitors like Google Cloud or AWS. Each cloud provider applies different optimisations and accelerations, making direct comparisons difficult.

Cost Economics

The token-based pricing for commercial models can become expensive at scale, driving production workloads toward open source alternatives. The calculation of when self-hosting becomes more economical than API calls depends heavily on specific use cases and scale. Alibaba Cloud offers both Model Studio (token-based) and PAI (AI infrastructure-based) to provide flexibility.

Fine-Tuning Strategy

Fine-tuning capabilities are being piloted within Alibaba Cloud Model studio in some regions. The recommended approach follows standard best practices:

1. Start with system prompts

2. Try LoRA (Low-Rank Adaptation)

3. Full fine-tuning only if necessary (requires significant engineering expertise)

Fine-tuning isn’t always necessary for solving specific problems, and over-reliance on it can indicate other issues with the implementation.

Future Roadmap & Technical Forecasts

Context Length Evolution

• Current: Up to 1 million tokens (commercial)

• Forecast: 10 million to 100 million tokens

• Challenge: Processing massive documents while maintaining attention across entire context

Model Architecture Questions

The engineering team forecasts several architectural considerations:

1. All-modality convergence toward single universal models

2. Potential limitations of transformer architecture at current scaling

3. Focus on test-time compute scaling

4. Data scaling and quality improvements

5. Enhanced reinforcement learning implementation

Some experts suggest the industry may have reached transformer architecture limitations, requiring new approaches beyond simply increasing model size.

Open Source Strategy

Alibaba remains committed to releasing both commercial and open source versions of new models. This dual approach serves multiple purposes:

• Developer community engagement and ecosystem building

• Direct user feedback for rapid iteration

• Path to commercial model adoption

• Competitive differentiation in a crowded market

The Qwen2.5 Math and Qwen2.5 Coder models extensively generated synthetic training data that fed back into pretraining for subsequent versions, creating a virtuous cycle of improvement.

Competitive Context & Market Position

Chinese AI Ecosystem

The broader Chinese AI landscape features intense competition. Following DeepSeek’s breakthrough, major tech companies (Alibaba, Baidu, Moonshot, others) significantly accelerated foundation model investments. DeepSeek pioneered many efficiency innovations but hasn’t maintained momentum recently. Moonshot’s Kimi briefly overtook Qwen on some leaderboards.

Qwen models remain among the most popular open source options globally, with MetaLM showing strong consumption across various API reselling platforms. The competitive intensity means new model versions release constantly, making benchmarks quickly outdated.

Vertical Integration Advantage

Alibaba’s control of models, and cloud infrastructure represents a rare combination. Most competitors lack global cloud presence. This vertical integration enables:

• Full optimisation across the stack

• Better cost control for model training

• Custom acceleration technologies

• Faster iteration and deployment

Strategic Implications

Open Source as Go-to-Market

Alibaba’s open source strategy serves multiple strategic purposes:

1. Developer mindshare and ecosystem development

2. Rapid feedback loops for model improvement

3. Lower barrier to experimentation leading to commercial adoption

4. Competitive differentiation against closed-source-only competitors

The community’s willingness to create derivative models (Japanese, Arabic variants) validates technical quality while expanding Qwen’s reach into specialized domains and languages.

Build vs Buy Dynamics

For enterprises, the decision calculus involves:

• API services: Fast to start, expensive at scale, less control

• Self-hosted open source: Higher upfront engineering cost, better long-term economics at scale, full control

• Managed services: Middle ground with varying levels of optimization

The emergence of companies like Fireworks and Together.ai suggests a sustainable market for managed open source model hosting, capturing value from optimization and operations.

Data & Synthetic Training

The use of Qwen2.5 models to generate synthetic training data for Qwen3 hints at an important trend: models improving through self-generated data. This approach requires careful quality control but can overcome data scarcity constraints, particularly relevant given export restrictions and data access challenges.

Q&A

Downstream RL Training

One attendee noted Qwen3 serves as an excellent seed model for downstream reinforcement learning training and domain specialization. However, reward hacking occurs during RL training. Alibaba expressed interest in collecting specific feedback to bring back to engineering teams for future model development.

Computer Use Agents

When asked about computer use capabilities (similar to Lovable or v0), Vlad clarified that Qwen Agent framework provides orchestration tools, but interactive computer control features might be product-level additions to Qwen Chat rather than core model capabilities. This mirrors how ChatGPT’s Operator exists as a product feature rather than model capability.

Voice Capabilities

Alibaba offers cascaded voice models (ASR and TTS) but attendees noted the industry trend toward native voice-to-voice models for better latency and naturalness. While cascaded models currently offer better reliability for enterprise use cases requiring tool calling and reasoning, native voice-to-voice models will likely improve significantly in the coming months for consumer applications.

Prompting Challenges

A significant practical concern raised: switching from one model to another (e.g., Gemini to Qwen) requires substantial prompt engineering adjustments. While competitors like Anthropic provide detailed migration guides explaining how to adjust prompts between model versions, Qwen’s documentation lacks comprehensive prompting guides, especially for cross-model migration.

Signals

What I’m Reading

Moats Before (Gross) Margins: Revisited

LLM Memory Systems Explained

The Agent Labs Thesis

Earnings Commentary

Have any feedback? Email me at akash@earlybird.com.

Subscribe now

Share

Leave a comment

Gradient Descending Roundtables in London

Last week we hosted Jonathan and Amelie, cofounders of design venture studio SPACING, to unpack how to approach design for AI-Native startups.

Moving Beyond Chat

Most startups SPACING worked with requested a chat interface, treating it as the default pattern for AI interaction. This led to:

• Homogeneous user experiences across products

• Home screens with generic prompts like “What do you want to achieve today?”

• Users struggling to understand capabilities and how to articulate requests

• Suboptimal workflows for specific use cases

The Chat Interface Paradox

Historical Context:

• Command-line interfaces → Graphical User Interfaces (GUIs) → back to text-based chat

• The key difference: Natural language processing enables conversational interaction

• But this doesn’t mean chat is always optimal

When Chat Works:

• Cursor - Engineers working with code and text (natural medium)

• Text-heavy workflows

• Open-ended exploration

• Specific questions requiring detailed responses

When Chat Fails:

• Tasks requiring visual precision (e.g., changing button colors)

• Workflows needing immediate visual feedback

• Actions better suited to direct manipulation

• Users without clear prompting knowledge

Direct Manipulation vs Chat

Changing a button colour through chat:

• Requires writing: “Change the colour to red”

• May not get the exact shade desired

• Requires specifying hex codes for precision

• Lacks immediate visual feedback

• Multiple iterations needed

vs. Direct manipulation:

• Click colour picker

• See all colour options instantly

• Immediate visual feedback

• Faster and more accurate

Four Ideas Beyond Chat

1. AI as a Collaborative Partner

Philosophy: AI as a companion rather than a replacement, working alongside users to enhance capabilities.

Implementation Strategies:

a) Contextual Commenting System

• Integrate AI into existing UI patterns (commenting features)

• Tag AI in comments on canvas elements instead of switching to separate chat

• AI responds in context, maintaining workflow continuity

• Example: Comment on a design element → tag AI → AI suggests variations in place

b) Project-Aware AI Companion

• AI “lives” within design files with persistent context

• Receives master instructions about project goals, brand guidelines, tone of voice

• Eliminates repeated context-sharing (unlike ChatGPT workflow)

• Press button to request copywriting assistance

• AI works in background while designer continues other tasks

c) Exploratory AI Features

• “Surprise button” next to colour picker

• Generates random colour suggestions

• Press multiple times for variety

• Sparks inspiration

d) Icon Generation

• Quick icon generation on demand

• No need for external icon libraries

• Integrated into workflow

e) AI-Generated First Drafts

• Addresses “blank canvas” problem

• AI creates initial structure (3 screens, basic flow, components)

• Goal: No accuracy, but starting point for iteration

• Leverages human strength: Critiquing existing work vs. creating from nothing

• AI generates initial automation flow so users can modify rather than build from scratch

Key Insight: People excel at criticising and improving existing work but struggle starting from nothing. AI provides that initial draft.

2. Proactive AI

Philosophy: AI anticipates next actions and surfaces suggestions contextually.

Implementation Examples:

a) Contextual Suggestions

• User selects table cells

• AI surfaces most obvious next action

• Keyboard shortcut to execute

• Example: Select cells → AI suggests rewriting content → Press shortcut to execute

b) Smart Filtering

• 100+ job applicants to review

• AI pre-scores applicants

• Surfaces top 3 candidates

• Provides strength/weakness summaries

• Generates interview questions

• User focuses on high-value activities (interviewing) rather than screening

c) Autonomous Content Creation

• AI learns brand identity and competitor landscape

• Generates new content pieces daily without prompting

• User wakes up to 5 new AI-generated video outputs

• Proactive value delivery while user sleeps

Key Challenge: Onboarding & Data Collection

• Initial approach: Comprehensive onboarding collecting information manually

• Problem: Onboarding too long, but insufficient data = poor results

• Solution: Balanced approach

  • Collect minimum data for first value

  • Continuously gather data during usage

  • Feedback mechanisms: Like/unlike, quality ratings, script preferences

  • Long-term: Close the loop with analytics showing ad performance

Critical Balance: Minimise user friction while maximising AI intelligence

3. Task-Driven Workflows

Phase 1: Chat-First Approach

• Assumption: Give users conversational interface for maximum flexibility

• Reality: Users didn’t understand what to type

• Problem: Too much freedom without guidance

• Especially challenging for users without marketing expertise

• Result: Poor user experience, confusion, abandonment

Phase 2: Guided Workflow

• Broke down required information into structured steps

• Used AI to enhance each input field

• Clear progression: Step 1 → Step 2 → Output

• Result:

  • Shorter time to value

  • Better outcomes

  • Higher user satisfaction

  • More consistent quality

Key Lesson

“We started with uncontrolled freedom...and the result was no one could figure out what to do. Then we moved to a guided approach...and the overall experience was just better.”

Supporting Features:

a) Prompt Enhancement

• User types short, simple prompt

• “Enhance prompt” button

• AI expands to sophisticated prompt automatically

• Better results without prompt engineering knowledge

b) Example Templates

• Pre-filled examples to start from

• Users click, see result, then iterate

• Educates users on capabilities through interaction

c) Smart Defaults

• Pre-populated based on user type (designer vs. founder)

• User only edits exceptions rather than filling blank forms

• Reduces cognitive load

4. Personalised Software

Philosophy: Software adapts to individual users and teams rather than one-size-fits-all.

The Home Analogy

“Your home should be arranged to fit your needs. You shouldn’t move into a house and have to live with someone else’s furniture arrangement.”

Personalisation Dimensions:

a) Custom Feature Development

• Users build features on top of base software

• Example: Calendly + Stripe integration

  • Many users want to charge for meeting slots

  • Calendly doesn’t offer this

  • Vision: User prompts to build Stripe integration themselves

  • Enables non-technical users to extend software

b) User-Type Adaptation

• Onboarding asks: Designer or Founder?

• Designer view: Shows all customisation tools, auto-layout options, advanced features

• Founder view: Hides overwhelming technical details, surfaces agentic AI features

• Software adapts to expertise level

c) Usage Pattern Learning

• Keyboard shortcut user: Hide tooltips, maximise screen space

• Visual navigation user: Show labels, icons, clear UI elements

• AI observes behaviour and adjusts interface accordingly

d) Team-Level Customisation

• Company-specific workflows

• Example: Custom performance review process in Linear/Asana

• Not in standard software

• Team prompts tool to build their unique workflow

Critical Challenges Discussed:

Challenge 1: Consistency Across Users

• Problem: If everyone’s UI looks different, how do colleagues help each other?

• Zoom calls: “Click top left” but it’s in different position for colleague

• Potential solution: Team-level personalisation rather than individual

• Open question: Balance between personalisation and shared understanding

Challenge 2: Guardrails

• What can AI adjust vs. what must remain stable?

• Need clear boundaries for personalisation scope

• Risk: AI moving critical elements unexpectedly

Challenge 3: Progressive Disclosure

• Different users see different features

• How to maintain product coherence?

• When does personalisation become fragmentation?

Q&A Discussion Highlights

On Data Collection & AI Quality

Question: How much onboarding data is needed for proactive AI?

Super Scale Experience:

• Started with website scraping

• Added competitor analysis for niche understanding

• Iteratively removed unnecessary fields

• Balance: Long onboarding vs. quick value

• Solution: Minimum viable onboarding + continuous learning

• Feedback loops: Scoring systems, like/dislike, style preferences

• Future: Analytics closing the loop (ad performance → agent improvement)

On User Friction & Context

Challenge: Getting users to provide context without disrupting work

Strategies:

• Automatic data collection where possible

• Example: Connect accounts and scrape data directly instead of asking for information manually

• Reduce manual input through intelligent defaults

Key Principle: Users won’t write perfect prompts or provide extensive context. Design must accommodate this reality.

On Getting Users Started with Chat

The “Empty Chat Box” Problem: Users don’t know what to type when facing a blank chat interface.

Solutions Implemented:

1. Prompt Templates Below Chat

   • Pre-written use cases users can click

   • Shows capabilities through examples

   • Especially important for users unfamiliar with AI tools (e.g., lawyers)

2. Structured Input Forms

   • Guide users through required information

   • AI enhances each field

   • Clear progression to output

3. Visual Examples/Galleries

   • Show what others have created

   • User-generated content inspiration

   • Similar to Notion templates, V0 examples

   • Calibrates expectations and demonstrates value

4. Starting Point Generation

   • User clicks example

   • Gets initial result (may not be perfect)

   • Iterates from there

   • Educates on capabilities through interaction

On Feedback Mechanisms

Question: How do you collect meaningful AI feedback beyond thumbs up/down?

Challenges:

• Users rarely provide feedback

• Thumbs up/down insufficient

• Need both quantitative and qualitative insights

Approaches:

Quantitative Tracking:

• Time to export/completion

• Number of regenerations needed

• Whether users edit prompts after submission (indicates dissatisfaction)

• Usage of manual editing tools (signals AI output inadequacy)

Qualitative Feedback:

• Optional detailed feedback forms

• “Why don’t you like this?” prompts

• Script/style preferences

• Problem: Low user engagement with feedback

Indirect Signals:

• If user goes to manual editor after AI generation = AI missed the mark

• Multiple regenerations = AI not understanding requirements

• Quick export = successful generation

Challenge: People are “annoyed” by feedback requests (like ChatGPT’s constant questions about response quality)

On Voice Interfaces

Context: Discussion of voice as alternative to chat

When Voice Works:

• Thinking out loud / brainstorming

• Therapeutic “dumping” of thoughts to ChatGPT

• Quick input capture

• Natural expression of ideas

When Voice Fails:

• Open office environments (privacy, noise)

• Latency issues

• Can’t easily review or edit what was said

• Reading is faster than listening to responses

• Forces real-time engagement (can’t pause and think)

Key Insight: “You talk quicker than you type, but you read quicker than you speak” - voice works for input but text is better for output consumption.

On Model Comparison & Iteration

Cursor Feature Highlight:

• Run multiple models in parallel

• Compare outputs side-by-side

• Some models more conservative, others more creative

• Users choose best result

Inline Commenting Approach:

• Highlight specific parts of AI output

• Comment on what needs changing

• Regenerate specific sections rather than entire output

• Reduces back-and-forth iterations

On Progressive Disclosure & User Types

Question: Who decides user type - the person or AI?

Answer: AI decides based on:

• Onboarding questionnaire responses

• Observed usage patterns

• Keyboard shortcut usage vs. visual navigation

• Feature utilisation over time

• Evolving understanding of user sophistication

Guardrails Discussion:

• Need rules for what AI can/cannot adjust

• Some elements must remain stable

• Balance adaptability with predictability

Case Study: From Chat to Guided Workflow

Initial Vision:

• Chat interface for video ad creation

• Assumption: Natural language = better UX

• “Create this ad” → AI generates video

Problems Encountered:

1. Users didn’t understand capabilities

2. No marketing expertise to craft effective prompts

3. Too much freedom = paralysis

4. Prompts too vague for quality output

5. Long wait times followed by disappointment

The Pivot: Moved to structured, guided flow:

• Step 1: Brand information (URL, identity)

• Step 2: Voice and tone

• Step 3: Niche analysis (automatic)

• Step 4: Output preferences

• AI enhances each input field

• Clear progression to output

Results:

• Dramatically shorter time to value

• Higher quality outputs

• Better user satisfaction

• More predictable results

• Users understand capabilities through structure

Signals

What I’m Reading

Some thoughts on AI

From Context Engineering to Context Platforms

Is Space Investable?

Earnings Commentary

Have any feedback? Email me at akash@earlybird.com.

Subscribe now

Share

Leave a comment

Gradient Descending Roundtables

This week, we hosted Matt from Cloudflare to discuss Agent Frameworks and Memory. Thanks to everyone who came and made the discussion so insightful!

I’m sharing the summary of our discussion below.

1. Framework Fatigue & The “Library vs Framework” Debate

Core tension: Most participants expressed frustration with bloated frameworks

• General consensus: “Everyone’s trying to build frameworks and products, no one’s trying to build libraries”

• Many have abandoned complex frameworks in favor of simpler approaches (OpenAI SDK, Anthropic SDK directly)

• Matt noted this is his 4th agent framework - each iteration has reduced complexity

Current preferences:

• Direct SDK usage (OpenAI/Anthropic) increasingly popular

• Cloudflare’s Agents SDK seen as lighter-weight abstraction

• LangGraph still used but developers “find myself going back to deterministic workflows”

2. Code Mode

What is Code Mode?

• Generating an SDK from tools, then having LLM write code against that SDK instead of direct tool calls

• Cloudflare implementation uses dynamic loaders to run generated code in isolated workers

• ~1ms cold start times on V8 isolates (not full sandboxes)

Key advantages identified:

• Massive token efficiency - compress 30 tools into one code-generation tool

• Enables data flow without seeing data (like bash pipes)

• Deterministic execution with compile-time validation

• Can use minified code since it’s machine-executed

Open question Matt posed: Should this be framework-level abstraction or let developers implement themselves?

3. Memory & Context Management Strategies

Minimal RAG adoption: Only 1-2 participants using embeddings-based retrieval

• One team: “glob and grab is unreasonably strong baseline” - hard to beat for code agents

• When RAG is used: Hybrid approach with knowledge graphs + embeddings

Graph-based approaches:

• Neo4j implementation by Cisco team for platform engineering

• Using LLMs to build knowledge graphs from documents

• Challenges: injection attack concerns, access control complexity

• Pattern: Find relevant node via embeddings → K-nearest neighbors traversal

“Predictive context loading” - Most novel approach discussed:

• Track agent behaviour patterns across evals

• Pre-load context based on statistical patterns (e.g., “if agent touches file X, 90% chance it needs files Y, Z next”)

• Comparison to web prefetching/autocorrect

• “Old school ML” middle layer between agent and tools

4. Session Management & Sandboxing

Pain points with Claude Code SDK:

• “Very insistent on file system” - hard to extract/resume sessions

• Teams need to fork agents thousands of times over months

• Built custom session managers to work around limitations

Current approaches:

• Micro-VMs (e.g., Firecracker via E2B) for code execution

• Cloudflare’s durable objects - “distributed little objects with SQLite store”

• First iteration: “Durable object as agent was a one-liner”

5. Tracing

Unanimous pain point: Existing tools (LangSmith, LangFuse, etc.) inadequate

Key limitations identified:

• Built for simple LLM calls, not complex agent traces

• Can’t visualize tree searches, test-time scaling

• Fail for sessions spanning months with thousands of forks

• “Not the same as distributed tracing for microservices”

Solutions:

• Teams building custom tracing UIs

• Atla fine-tuned a model specifically for analyzing agent traces against rubrics

• Cloudflare just released tracing for Workers

6. Optimisation Strategies & Architectural Patterns

Tool design:

• Debate: Many simple tools vs. few complex tools with parameters

• Calling LLMs inside tools “becomes painfully slow”

• Context reduction: Some teams using separate models to filter tool relevance before main agent

Multi-agent vs. Single-agent evolution:

• Pattern: Teams started with “orchestrator + investigators + verifiers”

• Newer models collapsing this: “Just pass everything to coding SDK - it’s comparable”

• “Handoffs are much faster” than complex tool returns

Prompt engineering shifts:

• “A year ago building complicated prompt workflows... now just ‘repo tool and guide’”

• Less manual XML construction, more reliance on model intelligence

7. Production Patterns

Pre-determined flows for common patterns:

• Recruiting agent example: Binary tree of prompts based on user requirements

• Generate “master prompt” from selected sub-prompts

• Reduces hallucination for structured interactions

Latency optimisation:

• Parallel tool calls causing provider rate limits/timeouts

• Hard to control with external APIs

• Moving toward edge execution for speed

8. MCP Discussion

When MCP makes sense:

• Remote tool servers where contract flexibility matters

• Tools that need to adapt per-user/session

• “Much better transport layer than API” for dynamic use cases

When NOT to use MCP:

• Local execution: “Never suggest MCP server + client on same machine - pointless”

• Well-documented TypeScript APIs: “Just use the API directly”

• Code mode might reduce MCP need, though Matt sees them as complementary

Critical distinction: MCP = discovery + transport, not execution. Code mode = execution optimisation.

Security Considerations

• Injection attacks for graph queries (Neo4j)

• Static analysis on generated tool code

• Cloudflare’s isolated execution model mitigates many concerns

• Access control in multi-tenant knowledge graphs

Further Reading

• The Model Is The Product

• Beyond Chunks: Why Context Engineering is the Future of RAG

• Code Mode: the better way to use MCP

Have any feedback? Email me at akash@earlybird.com.

Subscribe now

Share

Leave a comment

Gradient Descending Roundtables in London

Application Software Is Dead, Again

Three years in, the debate of value accrual in AI has only intensified as the model labs have transformed into product companies capable of shipping excellent consumer and developer-facing products at a clip that’s incomparable to incumbents of the past.

The debate was reignited on X this week.

There was a thoughtful follow-up from the author, which roughly approximates to: past technology cycles afforded more time for startups to create and capture value, whereas in AI the model layer is changing every 9-12 months, creating an impossibly short timeline for startups to outrun obsolescence by building enterprise relationships and an enduring brand.

The discourse on application software is of course one part of a broader discussion on how software is fundamentally changing, and, importantly, over what time horizon.

Agents Built On The Lakehouse

One view of the future, which I’ve written about before, is that we’re marching towards enterprises preparing their data estates for economies of agents to perform work end-to-end.

The unbundling and bundling of the modern stack is instructive.

As wrote on the fivetran and dbt merger:

every $1 to dbt means $10 to snowflake. i’ve heard numerous snowflake account execs confirm that while dbt only recently crossed $100m in revenue, they’re driving $500m in snowflake costs. which makes snowflake reps love to sell dbt in all of their transactions. this caused the dbt community to grow massively. which means snowflake account execs were basically dbt’s commission-only sales team. except instead of splitting the money, dbt got 2% and snowflake kept 98%. very generous partnership structure.

The majority of the dollars spent on data and value always accrued to Databricks/Snowflake, where the computing/querying was happening. For a while, they were happy to support an ecosystem of adjacent tooling so long as they were driving consumption upstream/downstream.

Eventually, the Lakehouses decided to consolidate the sprawling landscape of data infrastructure vendors on the premise of a lower TCO for their customers (fewer FTEs needed to manage different vendors).

This view of the world would see enterprises continue to embrace open table formats, consolidate their data and store it cheaply, and, most importantly, decide what query engine to use.

The same reasoning explains why Salesforce acquired Informatica, ServiceNow is focusing on their Data Fabric, and SAP announced a partnership with Databricks: help customers consolidate their data and perform work in your environment.

The only difference is that instead of humans performing the queries, eventually it’ll be agents.

Tooling for observability, governance, analytics, and security will still be necessary for enterprises to comfortably deploy agents fairly autonomously across their data estate.

For this view of the world to prevail, the customer’s business logic or domain-specific reasoning will have been codified (whether that’s packaged by a vertical AI vendor focused on domain-specific reasoning, captured by a Palantir FDE sitting with your subject matter experts, or learned through processing of your data corpora).

That’s of course across hundreds or thousands of processes, which is why the concept of an equally large number of small language models tuned to each specific process (running locally at little to no inference cost/COGS) makes so much sense.

AI’s Diffusion Rate

As plausible as it sounds, this scenario is still many, many years away, for two key reasons.

First, despite the unprecedented diffusion rate of AI for such a consequential technology, the change management required for enterprises to transform how they operate in a post-AI world is greater than anything that’s come before.

Satya Nadella opined on the future of software in an interview by Dwarkesh Patel and Dylan Patel:

Our business, which today is an end user tools business, will become essentially an infrastructure business in support of agents doing work.

However,

What took 70 years, maybe 100, 150 years for the industrial revolution may happen in 20 years, 25 years. I would love to compress what happened in 200 years of the industrial revolution into a 20-year period if you’re lucky.

Even if the tech is diffusing fast this time around, for true economic growth to appear, it has to sort of diffuse to a point where the work, the work artifact and the workflow has to change. And so that’s kind of one place where I think the change management required for a corporation to truly change, I think is something we shouldn’t discount.

The second key constraint is the development of agents, which we should measure over a time horizon of decades rather than years, as Andrej Karpathy has argued:

When you’re talking about an agent, or what the labs have in mind and what maybe I have in mind as well, you should think of it almost like an employee or like an intern that you would hire to work with you. For example, you work with some employees here. When would you prefer to have an agent like Claude or Codex do that work? Currently, of course, they can’t. What would it take for them to be able to do that? Why don’t you do it today?

The reason you don’t do it today is because they just don’t work. They don’t have enough intelligence, they’re not multimodal enough, they can’t do computer use and all this kind of stuff. And they don’t do a lot of the things that you’ve alluded to earlier. You know, they don’t have continued learning. You can’t just tell them something and they’ll remember it. And they’re just cognitively lacking and it’s just not working. And I just think that it will take about a decade to work through all of those issues.

This is by no means understating the diffusion rate of AI among consumers, where we’re still in the early innings of building AI-native experiences, let alone on new computing interfaces like wearables.

Computer using agents will no doubt soon be reliable at common consumer use cases like travel and shopping, with the enabling infrastructure for agentic commerce quickly coming together, further accelerating adoption. The consumerisation of enterprise software will continue to be a tailwind for AI penetration, and AI will catalyse adoption among SMBs in verticals that have been resistant to software eating the world.

The ground truth in the enterprise is that over half of enterprises have yet to put their first AI projects into production.

With Microsoft expected to be the biggest share gainer/beneficiary of AI workloads.

There are a lot of reasons behind this, but the structural reasons of change management and model capabilities are the two biggest ones.

It goes without saying that there are already use cases where the capabilities meet the minimum viable quality threshold and outsourcing was the default, such as customer support, where large companies are already being built.

The breakout companies in those verticals, though, are not cleanly defined as application or infrastructure companies - they’re both.

Agent Labs

I’ve written before about how applied AI companies closest to the end user earn the right to collect valuable training/reward data that can then be used to tackle gnarly infrastructure/research problems that help AI products cover the last mile in the enterprise, which has always been the longest.

The customer support market is an early indicator of this, where research engineers/scientists are becoming a more common hire in order to train custom models (e.g. reranker models) that drive higher resolution rates, one of the key metrics AI startups are being benchmarked on.

I’d predict that the title ‘Member of Technical Staff’ will become much more common at companies that have historically been deemed ‘application software’ companies.

This view of the world will see the lines between layers of the AI stack continuously blurring, perhaps even earlier than they already are in the current cohort of AI companies. Model labs will continue moving up the stack, whilst agent labs will relentlessly move down the stack to capture more margins.

An analysis I’ve been wanting to do for a while is to look at the attach-rate for model lab products to get a better sense of whether the right-to-win that is typically ascribed to the labs is justified. Of course, IBM, Microsoft, and Google were monoliths in different technology paradigms that had ambitions to win in as many markets as possible.

The notion that this time is different is appealing.

Reasoning from first principles, models continue to advance as a function of the data we can provide them, with the focus shifting to reasoning data. The labs will inevitably go after select verticals where marketplaces like Mercor make the acquisition of expert reasoning data easier than ever.

In vertical software, however, the heterogeneity in workflows/reasoning steps/data sets/integrations/GTM channels would make it very difficult for the model labs to execute against each opportunity with the level of focus that a dedicated agent lab can channel.

In horizontal software, being multi-model is a strength when measuring a portfolio of models against different criteria for specific use cases.

As agents become capable at handling existing workflows end-to-end, the best companies of the future will leverage AI to define new ways of creating value that we can only imagine.

We won’t have to wait long for that future, as the revenue ramp of Cursor, Sierra, Cognition, and various other companies is already proving. After all, some of the biggest companies in recent memory have built large businesses selling to other tech companies.

These customers embrace the future and build their technology stacks from the ground up for performance.

Time will tell which version of the future this cohort of AI companies builds on.

Signals

What I’m Reading

RL Environments and the Hierarchy of Agentic Capabilities

How Big was IBM?

Earned Intuition #003: Vertical AI: What Would You Never Ask a Human to Do?

2025 Vertical & SMB SaaS Benchmark Report

The Future of AI History

Earnings Commentary

Have any feedback? Email me at akash@earlybird.com.

Subscribe now

Share

Leave a comment

Gradient Descending Roundtables in London

This week, we hosted Marlene and Chris from Microsoft AI to discuss Small Language Models, Context Engineering and MCP. Thanks to everyone who came and made the discussion so insightful!

I’m sharing the summary of our discussion below.

Edge AI & Microsoft Phi Models

Microsoft’s SLM Strategy

Marlene presented Microsoft’s edge AI push, centered around the Phi model family (currently at v4):

Key components:

• Phi 4 flagship: General-purpose SLM optimized for edge deployment

• Phi 4 reasoning: Reasoning version with strength in mathematical reasoning

• Foundry Local: Microsoft’s Ollama-equivalent platform for downloading and running local models, with a path to Azure cloud services

Competitive positioning:

• Compared favorably to Qwen models (described as “model of choice” by many practitioners)

• Also competing with Mistral 7B, Gemma, and other distilled models

• Cost advantage: Phi 4 is approximately 150x cheaper than GPT-4.5 for serverless compute

The Email Agent Case Study: A Context Engineering Example

Marlene’s email agent project serves as an excellent case study in the practical challenges of deploying SLMs:

Architecture:

User Query → Supervisor Agent → {
    - Manage Email Agent
    - Scheduling Event Agent  
    - Search Email History Agent (with Postgres + semantic search)
} → MCP Server (M365 tools) → Results

Critical Design Decision: Sub-agent Architecture

The most significant insight was the necessity of dividing context across specialised sub-agents rather than loading all MCP tools into a single agent. This addresses a fundamental problem: when you connect an MCP server with dozens of tools, the tool descriptions alone overwhelm the SLM’s context window, degrading performance catastrophically.

Solution strategies employed:

1. Tool segmentation: Each sub-agent receives only relevant tools for its domain

2. Middleware layer: Custom JSON generation to work around Phi’s lack of native function calling (though this is being addressed soon)

3. Result summarisation: Critical challenge of managing tool output that floods context windows

4. Semantic search: Using Postgres for email history rather than raw context stuffing

Performance metrics:

• Response latency: ~2 seconds (surprisingly fast for local inference)

• Key bottleneck: Not compute, but memory/storage on the device

The Gaming Paradigm

LLM Paradigm vs. SLM Paradigm

LLM world:

• Time is abundant (10+ seconds acceptable)

• Quality is paramount

• Single-threaded workflows

• Cloud-centric

• Cost scales with usage

SLM world (especially gaming):

• Sub-second latency requirements (ideally <1 second)

• Minimum viable quality threshold

• Massively parallel workflows

• Device-constrained

• Fixed cost model

The GPU Budget War

Game developers traditionally allocate GPU budgets across departments (sound: 10%, graphics: 40%, etc.). AI is now demanding its own substantial budget allocation, forcing painful trade-offs. This explains why gaming hasn’t yet widely adopted generative AI - it’s not just a technical challenge but a fundamental resource reallocation problem.

Client-Side AI in Gaming: Current Reality vs. Future Vision

Current constraints:

• Can typically run 1-2 SLMs on-device simultaneously

• Most prioritise single model for maximum quality after distillation

• Hybrid approaches: Pre-generate assets server-side, deliver real-time locally

• Examples: “Operators” game doing text-to-speech with smart caching

Future paradigm shift:

• Multiple specialised SLMs running in parallel

• One for dialogue generation

• One for text-to-speech

• Others for translation, behaviour trees, etc.

The killer use case: NPC conversations

Walking into a room with 10 NPCs having simultaneous conversations. Cloud models can’t scale this (10 parallel LLM calls = cost explosion), but device-based SLMs could handle multiple concurrent agents efficiently.

The Streaming Gaming Convergence

A strategic insight emerged about cloud gaming services (AWS Luna, etc.): They create an opportunity to co-locate GPU streaming and GPU inference, enabling hybrid architectures where:

• Latency-critical elements run locally

• High-fidelity generation happens cloud-side but close to streaming source

• New compression paradigms could enable prompt-based video streaming

Context Engineering: The Central Challenge

Context management is the defining challenge for SLM deployment:

Problem 1: Tool Description Overload

• Loading all MCP tools overwhelms context window

• Model performance degrades even before executing any tools

• Anthropic’s recent blog post confirmed this industry-wide issue

Problem 2: Tool Result Flooding

• Example: “Find emails from past 3 months” returns massive data

• Keeping results in context causes model failure

• Need to balance information preservation with context limits

Mitigation Strategies Discussed

1. Architectural solutions:

   • Sub-agent decomposition (Marlene’s approach)

   • Virtual file systems (Cloudflare’s code mode approach)

   • Durable execution patterns (Temporal workflows)

2. Data management:

   • Result trimming (lossy but pragmatic)

   • Summarisation layers (risk of losing critical context)

   • Semantic indexing (Postgres example)

3. Emerging solutions:

   • Samba model: New Microsoft Research model claiming “unlimited context windows” through novel compression/persistence mechanisms

   • Middleware patterns for context transformation

The Agent vs. Workflow Debate:

Agent-heavy approaches:

• Tool-calling paradigm with autonomous decision-making

• Fills context windows quickly

• Unpredictable latency

• Popular but problematic for constrained environments

Workflow/chaining approaches:

• Airflow-style deterministic pipelines

• More predictable, cheaper, faster

• Better for real-time requirements

Hardware Evolution: The Coming NPU Revolution

The CUDA Parallel

When Nvidia first released CUDA, AMD had faster chips, and people questioned dedicating silicon to such a niche use case. Today, CUDA lock-in dominates AI infrastructure.

The NPU trajectory will follow a similar path:

• Current state: Hardware lagging software (unusual reversal)

• Microsoft shipping next-gen PCs with NPUs optimized for inference

• Gaming consoles (PlayStation, Xbox) will include AI-specific silicon

• Mobile devices (especially iPhones) already have powerful NPUs

AI is already embedded in every layer of computing (OS, browser, applications). Without optimized hardware, most features become unusable. This creates an inevitable forcing function for NPU adoption.

Infrastructure Layer: Inference Providers & Trade-offs

Groq/Cerebras Discussion

The conversation revealed nuanced understanding of specialized inference providers:

Groq advantages:

• Extreme speed (2-second responses vs. 20 seconds for Anthropic)

• Quality now on par with frontier models

• Critical for customer support, voice agents

• Cost-effective for fire-and-forget tasks

Groq/Cerebras limitations:

• No prompt caching (due to architectural choices around on-chip memory)

• Makes agentic workflows economically unviable

• Can’t support coding workflows that rely on caching

Optimal use cases:

• Summarisation (no caching benefit anyway)

• Single-pass tasks

• Speed-critical applications without iterative loops

The Caching Dependency

The entire agentic paradigm is built on prompt caching. Providers without caching are fundamentally non-viable for agent workflows, regardless of speed advantages. This suggests prompt caching has become infrastructure-critical, not just an optimisation.

Cost-Performance Frontier

The 150x Cost Advantage

Implications:

• B2C AI becomes economically viable (currently mostly B2B due to costs)

• Multiple SLM agents cheaper than single LLM call

• Enables new business models previously impossible

Quality Threshold Debate

Current consensus:

• Adequate for experimentation

• Not yet production-ready for most applications

• Quality gap expected to close within 1-2 years

• Gaming sector: Still struggling to ship SLM-powered features

The deployment readiness spectrum:

• Ready now: Summarisation, classification, simple extraction

• Close: Customer support, basic agents

• Not ready: Complex reasoning, multimodal generation, parallel agent orchestration

Model Context Protocol

Why MCP matters:

• Abstracts away complex APIs (Microsoft Graph example)

• Pre-built authentication flows

• Rapid prototyping without API expertise

• Standardised tool interface

The Cybersecurity Shadow

Chris’s book and warnings introduce crucial caution: MCP creates supply chain attack vectors similar to NPM/pip packages:

• Malicious MCP servers could exfiltrate credentials

• Users connecting servers via Claude Desktop or GitHub Copilot may not understand risks

• Analogous to the JavaScript package ecosystem’s security challenges

The tension: Ease of use vs. security. MCP democratizes AI tool integration but potentially at the cost of introducing new attack surfaces.

Is MCP the Right Abstraction?

Marlene posed whether MCP is even the correct model for providing context to SLMs. The underlying question: Should we be using RPC-style protocols for context?

Quantisation Quality Cliff

Model quantisation has a quality cliff:

• 16-bit quantisation: Acceptable quality

• 12-bit and below: “Absolute garbage”

• Industry marketing problem: Everyone reports performance on 16-bit variants

Implications for edge deployment: The vaunted small size of SLMs may be illusory if quality degradation from aggressive quantisation makes them unusable. This suggests edge devices need more capable NPUs than currently assumed.

Further Reading

• The Phi Cook Book

• Foundry Local (Microsoft’s offering for local models)

• Samba (a new local model moving towards unlimited context)

• Edge AI for Beginners (a course we have for working with local models)

Have any feedback? Email me at akash@earlybird.com.

Subscribe now

Share

Leave a comment

Gradient Descending Roundtables in London

There’s a new Pareto frontier defined by latency and determinism.

Elon Musk’s claim that diffusion-based LLMs will surpass autoregressive transformers comes on the heels of Inception Labs announcing a $50m to continue training their Mercury series of models that we first covered in March.

Time to first token is slower for diffusion-based language models but total generation speed is much faster.

Why does time to first token matter so much, though?

Fal cofounder Gorkem has a front-row seat powering image and video workloads for large enterprises:

Latency is really important. One of our customers actually did a very extensive AB test of like they on purposely slow down latency on file to see how it impacts their metrics. And it had a huge part in it. And it’s, it’s almost like page load time when the page load slower, you know, you make less money. I think Amazon famously did a very big AB test on this. It’s, it’s very similar. Like when you, when the user asks for an image and, you know, iterating on it, if it’s slower to create, they’re less engaged, they create fewer number of images and, and things like that.

Though slower to first token, thereafter diffusion models quickly eclipse autoregressive models:

New research suggests diffusion language models achieve roughly 3x the data efficiency of autoregressive models - pretty handy as we hit data walls!

There will be plenty of workloads willing to trading off time to first token for total generation time and data efficiency.

Determinism is now attainable with the same trade-off.

Significant capital and talent has been focusing on how to provide the harnesses for language models to serve verticals and workloads that have low tolerance for hallucinations and inaccuracies - RAG and guardrails are just two of the resulting mechanisms.

In September, Thinking Machines Lab released their first product tackling non-determinism. At last, I got around to reading the accompanying paper.

Like many others, I had long assumed that the probabilistic nature of LLMs is intrinsic to the architecture of transformer-based language models.

On the contrary, this non-determinism is a property of GPU operations.

The wonderful property of parallel processing that makes GPUs so powerful for accelerated computing compared to CPUs is actually the culprit behind a lack of determinism.

A primer on how this works:

1. AI server load varies throughout the day

2. Batch size varies (for simplicity, imagine a batch size of 1 during quiet times, 32 during peak)

3. GPU kernel chooses algorithm based on batch size

4. Different algorithms sum numbers in different orders

5. Floating-point arithmetic makes addition order matter

6. Same input produces different outputs (nondeterminism)

Let’s unpack some of those concept.

Floating point non-associativity

Why do we use floating point numbers? Floating point maintains “significant figures” across wildly different scales.

Because it gives us “dynamic precision” - we can represent both:

• Tiny numbers: 0.0000000486

• Huge numbers: 3,450,000,000

The price is that addition order matters, because floating point operations necessarily lose precision and are order dependent.

GPU Kernels

At runtime, different orders are used for different batch sizes.

To illustrate, let’s imagine a smaller batch size at a quiet time of the day.

GPUs are optimised for parallelism, so for small batches they will use split-reduction to use as many cores of the GPU as possible. This is different to how larger batches would be handled, where sequential addition would be used.

Thinking Machine Labs’ library proposes to resolve this by using the same reduction order independent of batch size, achieving batch invariance.

In doing so, parallelism and latency are traded off for determinism.

The results: Asked Qwen-3-235B “Tell me about Richard Feynman” 1000 times at temperature = 0

• Without their kernels: 80 unique completions

  • Most common appeared 78 times

  • Divergence at token 103: “Queens, New York” (992x) vs “New York City” (8x)

• With their kernels: 1 completion, all 1000 identical

The trade-off in latency is below.

For enterprises in regulated industries, stuck in pilots waiting to move to production with their AI workloads, these trade-offs will be worth making..

Signals

What I’m Reading

The Evolution from RAG to Agentic RAG to Agent Memory

Inside Cursor

What does OSWorld tell us about AI’s ability to use computers?

Why behind AI: Data centres in space

Post Training Ensemble vs. Singular Model Approaches with Tinker

Earnings Commentary

Have any feedback? Email me at akash@earlybird.com.

Subscribe now

Share

Leave a comment

Gradient Descending Roundtables in London

Last week, Cursor finally released their first frontier coding model, Composer 1 Alpha.

Many are speculating whether the model is a fine-tuned Chinese MoE model, which seems plausible given Cursor’s stance on where their edge lies:

Our primary focus is on RL post-training. We think that is the best way to get the model to be a strong interactive agent.

Earlier that morning, we hosted Aidan and Matt in our office to discuss Scale AI’s recently published research on ‘Rubrics as Rewards’ and its implications for post-training base models.

I’ve written at length about how application-layer companies (or ‘agent labs’) capture reward signals that are becoming increasingly valuable as the cost and complexity of post-training collapse (fine-tuning APIs, managed infra).

‘Reward Engineering’ has proven to be one of the defining AI themes of the year as companies like Thinking Machines Lab, Applied Compute, Osmosis and others emerged with a value proposition of post-training custom models underpinned by high quality reward data. This is all against a backdrop of rapid advances in large model capabilities across objectively verifiable domains like coding and mathematics (where the reward is binary; a proposed code change either runs or it doesn’t).

As soon as this vector of scaling RL became clear, the immediate next question was how to apply it to non-verifiable domains where subjectivity plays a bigger role.

That’s what the Scale paper is focusing on, proposing a richer way of capturing tribal knowledge and reasoning than simple RLHF or preference-ranking. In effect, this is closer to having a process reward model that not only rewards the final outcome but also the steps taken to get to it - and the results are striking.

Below are the notes from our discussion. The relevant papers are here and here - you can also see examples of Rubrics here on the Scale website.

Evolution of Model Training: Post-Training

The progression of post-training approaches:

1. Supervised Fine-Tuning (SFT) - Early approach

   • Provides prompt-response pairs with “correct” answers

   • Useful for smaller, specialised models

   • Falls away for larger, more generalised models

   • Can lead to overfitting in complex reasoning scenarios

2. RLHF (Reinforcement Learning from Human Feedback) - Current approach

   • Humans select preferred responses between model outputs

   • Effective for simple queries but struggles with multi-step reasoning

   • Volume-intensive and requires extensive human evaluation

   • Risk of reward hacking

3. Rubrics-Based Approach - New approach

   • Combines expert knowledge with scalable automation

   • Particularly valuable for unverifiable domains

The Continuous Evaluation Loop

Training follows a cyclical process:

• Evaluation → Data Production → Performance Improvement → Re-evaluation

• Typically runs in 6-month cycles with 2-3 iterations per model development phase

• Requires constant creation of new data based on previous evaluation results

• Includes adversarial components (red teaming) to identify security and safety weaknesses

Understanding Rubrics as Rewards

What Are Rubrics?

Rubrics are structured evaluation frameworks consisting of:

• Binary or weighted criteria that can be objectively assessed

• Multiple independent factors that together evaluate quality

• Specific rules stating what makes an ideal response

• Examples to help models understand expectations

Simple Example Structure

For a basic prompt, a rubric might include:

• Must address the core question

• Should provide specific details

• Must avoid certain pitfalls

• Should follow appropriate tone/style

Complex Professional Example

In an investment context (e.g., market entry analysis for EVs in South America):

• 20-30 different evaluation components

• Checks for specific data points (market size, timelines, partnerships)

• Validates reasoning steps

• Assesses structural clarity and relevance

• Can be binary (present/not present) or weighted

Three Key Advantages of Rubrics

1. Mitigating Reward Hacking

The Problem: When using simple preference selection, models may learn unintended patterns

• Example: Accidentally training preference for “red cars over blue cars” when color wasn’t the intended differentiator

• Human evaluators have intrinsic biases that can skew preference data

How Rubrics Help:

• Break down evaluation into specific, objective criteria

• Exclude components that might drive unintended behavior

• Allow larger volumes of diverse data to be generated

• A recent Scale AI paper empirically demonstrated reduced reward hacking with rubrics

2. Adaptability

The Challenge: Models need constant refinement based on real-world performance

• Errors emerge during deployment

• Requirements change based on user feedback

• New security issues are discovered

Rubrics Solution:

• Much easier to tweak a rubric than retrain thousands of human evaluators

• Can quickly adjust weighting or add new criteria

• Enables faster iteration cycles

• More agile response to identified problems

3. Domain-Specific Control

Different domains require different priorities:

• Creative writing: Style, tone, human-like quality matter most

• Mathematics: Correct answer and logical reasoning are paramount

• Professional contexts: Structural clarity, verifiable data, reasoning transparency

Rubrics enable precise weighting of different evaluation components based on what matters in each specific domain.

The Role of LLMs in Rubric Evaluation

Automation Through AI

• Humans design the rubrics (expert knowledge)

• LLMs automate the evaluation against those rubrics

• This creates scalable volume while maintaining expert-defined standards

Why This Works

LLMs are particularly good at:

• Checking if specific elements are present

• Verifying binary conditions

• Following clear, objective rules

Model Selection for Evaluation

• For research purposes (as in the paper): Used GPT-4 Mini as LLM judge to ensure standardisation

• For production: Smaller models can work if rubrics are well-designed

• Frontier labs building cutting-edge models still prefer human-written rubrics

• Smaller budget projects can use LLM-generated rubrics with acceptable results

Practical Applications

Investment Due Diligence

Multi-step reasoning example:

• Initial analysis of market conditions

• Financial projections

• Partnership identification

• Risk assessment

• Each step can have its own rubric

• Important to maintain observable reasoning pathways

Healthcare/Medicine

Unverifiable domain characteristics:

• No single “correct” diagnosis for complex presentations

• Expert intuition difficult to codify

• Rubrics help capture decision-making criteria

• Can evaluate diagnostic reasoning steps

Legal Services

Contract analysis requirements:

• Specific clause identification

• Risk assessment

• Compliance checking

• Precedent application

Insurance

Knowledge extraction challenge:

• Much expertise exists only “in people’s heads”

• Retiring workforce creates knowledge drain

• Rubrics help codify expert decision-making

• Quality assurance already uses similar scoring systems

Technical Details from the Research

Experimental Setup

• Base model: Qwen 2

• Comparison: Rubrics approach vs. direct preference ranking

• Evaluation: GPT-4 Mini as judge for standardisation

• Training: Offline RL (not online due to cost)

Performance Results

The rubrics approach showed improvements in:

• Context awareness

• Communication quality

• Accuracy in reasoning-heavy tasks

Key Finding

Rubrics are particularly effective for:

• Complex, multi-step reasoning

• Domains without clear “correct” answers

• Professional/expert knowledge domains

• Tasks requiring observable reasoning chains

Critical Considerations and Debates

Evaluating the Evaluators

The recursion problem:

• How do you evaluate whether rubrics are good?

• Ultimately requires testing impact on model performance

• Need comprehensive evaluation datasets for each domain

• Iterative refinement based on real-world results

Human vs. Automated Rubric Creation

For frontier performance: Human-written rubrics still superior

For smaller models/budgets: LLM-generated rubrics can work

Key insight: The intersection of domain expertise and understanding of model behaviour is rare and valuable

Market Implications

The Changing Nature of AI Data Work

From volume to expertise:

• Less emphasis on mass preference labelling

• More focus on expert rubric design

• Smaller teams of highly skilled data creators

• Higher per-person cost but better outcomes

Professional Knowledge Extraction

The hidden expertise problem:

• Most professional knowledge isn’t documented

• Exists in practitioners’ intuition and experience

• Examples: Insurance brokers, doctors, lawyers, investment analysts

• Rubrics provide a framework to codify this tacit knowledge

Future Opportunities

• Industries with retiring workforces (insurance, legal)

• Domains with complex, multi-step reasoning

• Applications requiring explainable AI decisions

• Situations where “average” data isn’t sufficient

Autonomous Improvement Discussion

The Holy Grail Question

Can frontier models achieve fully autonomous self-improvement using only AI-generated rubrics?

Current state:

• Deepseek recently claimed full self-improvement in specific domains

• Depends heavily on model size and domain complexity

• Still an open research question

• Likely varies significantly by application

Hybrid Approaches

Most realistic near-term:

• Human-designed rubrics for frontier domains

• LLM-automated evaluation and data generation

• Human oversight for refinement and validation

• Iterative improvement cycles

Key Takeaways

1. Rubrics represent a middle ground between fully manual RLHF and purely automated approaches

2. Most effective for unverifiable domains where there’s no single correct answer but expert judgment exists

3. Enables scaling expert knowledge by codifying it into structured, scalable evaluation criteria

4. Reduces but doesn’t eliminate human involvement - shifts humans to higher-level rubric design rather than individual preference selection

5. Particularly valuable for professional applications in law, medicine, finance, and other expert domains

6. The data market is shifting from volume-based preference ranking to expert-driven, rubric-based approaches

7. Quality of rubric design is critical - requires intersection of domain expertise and understanding of AI model behavior

Future Directions

• Application to visual language models and robotics

• Integration with existing quality assurance frameworks

• Automated rubric generation for non-frontier applications

• Expansion into multimodal domains

• Development of standardised rubric libraries for common domains

Signals

What I’m Reading

How to Keep Winning

Top Down and Bottom Up Investors

The Case for the Return of Fine-Tuning

Is Diffusion the Future of LLMs?

Marc Andreessen and Charlie Songhurst on the past, present, and future of Silicon Valley

Europe’s Hidden Sovereign AI Gem: Nebius

Earnings Commentary

Have any feedback? Email me at akash@earlybird.com.

Subscribe now

Share

Leave a comment

Gradient Descending Roundtables in London:

• October 29th: Rubrics as Reward: RL Beyond Verifiable Domains with Scale AI

---

Last week, we hosted Maria Zervou, Chief AI Officer EMEA at Databricks for a Gradient Descending roundtable.

Databricks’ phenomenal growth at scale (50% YoY, $4bn run-rate, 140% NDR) underlines its position as the leading platform for enterprises to deploy AI.

The roundtable covered a lot of ground, with the below key takeaways:

Data Intelligence vs. General AI

Databricks’ approach is distinct from generic LLM applications:

• General AI: Using OpenAI/Anthropic models with prompt engineering alone

• Data Intelligence: Grounding AI in proprietary data with robust governance

• Competitive advantage comes from data access, not just model selection

Three Pillars of Databricks’ AI Platform

Governance (Unity Catalog)

• Centralised permissions for data, models, and tools

• End-to-end lineage tracking from data → application → end user

• Critical for EU compliance and financial services

• Open source governance layer

Grounded AI

• Opening data/tools to agents in controlled ways

• Python tools, MCP servers, SQL access, vector stores

• Permission propagation from end user to data layer

Flexible AI

• Partnership with all major cloud providers, OpenAI, Anthropic

• Support for all open-source models (including Chinese models via self-hosting)

• Model routing capabilities for cost/performance optimisation

The Evaluation Challenge

Current State:

• Customers don’t know which models work best for specific tasks

Databricks’ Solution:

• Custom “judges” (fine-tuned models for specific evaluation tasks)

  • Out-of-box judges for: grounding, safety, bias

• MLflow-based evaluation framework (open source)

• Version tracking and A/B testing capabilities

• SME labelling sessions for continuous improvement

What’s Happening In The Field

Potential Reasons For The MIT “95% Failure Rate” Report:

• Primary reason: AI products cannot log, trace, understand, or prove value

• Lack of end-user involvement in development process

• Insufficient hands-on support and change management

Real-World Agent Limitations:

• Multi-step execution: Maximum ~5 steps in production

• Steps are templatised and controlled, not free-form

• Financial services require predefined workflows with reasoning at each step

Human-in-the-Loop is Reality:

• Automation dream vs. reality disconnect

• Even with reasoning capabilities, manual approval required

• Example: Credit limit increases still need human sign-off

Fine-tuning?

• Not much adoption despite initial hype

• Reasons:

  • Maintenance burden (retraining, drift monitoring)

  • Unclear ROI vs. larger base models

  • Only justified for specific industry vocabulary or behaviour constraints

    • Example: Medical app - never give advice, only facilitate conversation

Use Case Evolution

Three Tiers of AI Adoption:

1. Productivity Improvements (2023-2024)

   • Copilots for individual users

   • Incremental efficiency gains

2. Process Automation (Current focus)

   • Templatising business processes

   • Agent-driven workflow automation

   • Scaling human tasks

3. Business Model Innovation (Emerging)

   • Complete rethinking of value propositions

   • Example: Behavioural assessment company → AI-powered team formation and task assignment platform

   • Represents genuine innovation beyond productivity

Agent Development: Different Approaches With Databricks

1. Full Hands-On Development

• Complete control over agent behaviour

  • For complex, custom requirements

  • Requires engineering expertise

2. Agent Bricks (Semi-Hands-On)

• Pre-built components:

  • Information extraction

  • Knowledge assistant (RAG)

  • AI BI Genie (text-to-SQL)

  • Customer LLM (fine-tuning)

  • Multi-agent supervisor

• Still requires understanding of extraction processes and evaluation

• Not for non technical business users

GTM Dynamics

Historical Approach (Pre-2024):

• Engineering-focused, bottom-up sales

• Build POCs/MVPs with engineering teams

• Blocked at large organisations without C-suite buy-in

Current Approach:

• Top-down + bottom-up simultaneously

• FDE (Forward Deployment Engineers) for hands-on work

• C-suite positioning for change management

• Industry roundtables for peer learning

What Works:

• Bringing customers together to share implementations

  • “Customers don’t want to be left behind”

• Success tracking: New use cases 1-2 months post-event

• Digital natives adopt much faster than traditional enterprises

Data Marketplace

Delta Sharing:

• Structured data exchange between companies

• Use case examples:

  • Retail: Suppliers ↔ Supermarkets

  • Travel: Airports ↔ Airlines

• Enables better collaborative models without competitive compromise

Solution Marketplace:

• Customers can sell bespoke agents to other companies

  • Example: Travel industry agent

• Revenue stream for customers

MCP Integration

• Databricks has made a large investment in MCP support

• Auto-release of tools as MCP servers (text-to-SQL, similarity search)

• Bringing external MCPs into Unity Catalog with permission management

• Planning MCP/agent marketplace

• External MCP usage: Expect usage-based charging similar to APIs

Data

Coatue’s updated Fantastic 40

OpenAI’s ramp is unprecedented by a mile, dominated by ChatGPT

Reflection AI might change things, but clear that a large % of AI ecosystem now reliant on truly open models from China

Reading

The Data Source #28: Mapping out the Future of Compute 🌎

Building in the AI Era: The HeyGen Way

The Hard Way Pays Off: Inside Sierra’s Design Partner Strategy

The New World

Seeing Science Like a Language Model

Have any feedback? Email me at akash@earlybird.com.

Subscribe now

Share

Leave a comment