CPUs are the Bottleneck - and Branching is the Unlock

Why VERS (HDR Research) sits at a durable leverage point of the agentic compute stack

HEADS UP: this article is technical. We’ve worked to translate Tynan’s brilliant everyday explanations into something digestible for a general audience, but he says it best himself. To hear it straight from him, reach out to Tynan, Founder of VERS, at tynan.daly@hdr.is.

---

For the better part of 2 years, CPUs have been a neglected topic when it comes to discourse on AI infrastructure, while GPUs have dominated every conversation: first for training, then for inference. That assumption has been falling apart over the last six months. Agentic AI has emerged as the killer app inference was waiting for, and the shape of the workload is different enough that the old “CPU-as-housekeeper” assumption no longer holds.

💡Just as a housekeeper enables residents to focus on their main tasks, the CPU was seen as a component that only performed tasks to support the GPU, which was the main workhorse for AI computations. As AI models become more complex and require more diverse tasks (e.g., data retrieval, task planning, API calls), the CPU is taking on a more central role in coordinating and executing these tasks. The GPU is no longer the sole protagonist.

Every piece of analysis on this shift converges on the same finding: in representative agentic workloads, CPU side tool processing accounts for 50% to 90% of total latency. The GPU sits allocated but idle, waiting on the CPU to finish orchestration, tool execution, and state management before it gets the next batch of tokens. Given the cost differential between owning and operating the GPU and CPU, the natural conclusion is we should optimize the CPU ratio and efficacy to ensure the GPU is always utilized.

The common framing is that CPUs are in short supply and more of them are needed. That framing is correct but incomplete. What runs on top of them matters more.

This piece argues 3 things.

1. the CPU/GPU ratio is moving and the silicon vendors have already adjusted.

2. the binding constraint in agentic compute is not raw CPU cycles, it is how fast you can isolate, branch, snapshot, and resume running compute state.

3. the margin in this next phase accrues to the layer that turns GPU hours into useful work, not the layer that produces GPU hours.

VERS (HDR Research) has been building at that layer for 18 months, and the technical primitive they’ve built (a live VM you can fork in microseconds) is turning out to be the right shape for the workloads the industry is now scaling into.

The CPU to GPU Ratio is Moving

The CPU-to-GPU ratio in AI clusters has sat at roughly 1:4 to 1:8 for the training era. That ratio was reasonable when the dominant workload was a single large matrix-multiplication job pinned to an accelerator. It stops being reasonable once the workload becomes hundreds of concurrent sub-agents per user.

The silicon vendors have absorbed this faster than most public commentary:

• Nvidia’s Vera CPU is being deployed as a standalone platform for agentic processing. CoreWeave is the first public customer; Jensen has said there will be many more. A standalone CPU platform from the company that owns the GPU narrative is a large signal.

• Arm launched the AGI CPU in March, its first production silicon in 35 years (!!) A company that has been a pure IP licensor since the 1990s does not turn into a direct seller of chips unless the data-center CPU TAM is about to re-rate!

• Intel’s most recent earnings call caught management visibly off-guard on CPU demand. A company whose entire franchise is data-center CPUs being surprised by CPU demand tells you how early the shift is (but also lol, Intel)

💡 Market research now expects the CPU-to-GPU ratio to compress from 1:4–1:8 toward 1:1–1:2. At 1:1, the CPU market associated with agentic AI becomes several multiples larger than it is today.

We haven’t even gotten into the issues in the physical supply chain yet. Component costs for Blackwell systems have moved roughly 50% in the last six months. Our BOM for our data center priced at $36M in late 2025, the same BOM is now ~$55M and tracking toward $60M, driven in large part by CPU and memory inflation.

The industry is paying more for Blackwells that are producing fewer tokens per dollar than the same silicon produced 6 months ago. That creates an unusually strong incentive to extract more work from existing compute.

What Agent Orchestration Does to a Host Machine

Pure inference vs. agent orchestration

In the pure inference workflow that dominated 2025, the CPU’s job was narrow: tokenize the request, hand it to the GPU, de-tokenize the output, return it to the user.

Agent orchestration inverts this. The CPU becomes the command layer.

The loop is:

• A user request arrives

• The CPU tokenizes it

• The first inference call returns a plan

• The CPU then spawns N sub-agents, each of which may run its own inference loop, call tools, query databases, scrape web pages, or execute code

• The CPU holds the state, integrates partial results, decides when to reflect, when to branch, when to retry

The GPU is just one of the things the CPU calls.

Three things need to work well on the host machine:

1. Each individual core¹ has to be fast, fast enough to watch what every sub-agent is doing and respond immediately when one of them finishes. If a core is slow, agents wait on it instead of the other way around.

2. There have to be many cores, and each core should be able to juggle multiple agents at once. This is called simultaneous multi threading, or SMT (a concept that lets 1 physical core look like 2 to the software). With SMT, a 256-core chip can handle 512 agents in parallel. Intel’s newest server chip dropped SMT, which is why people who care about action-heavy agent workloads are not happy!

3. Memory has to be close. Modern server CPUs pack so many cores onto one chip that the chip is internally divided into zones. Each cluster of cores has “its” memory, and talking across zones is slower than talking within one.²

These three requirements pull in different directions depending on what your agents are actually doing.

The Reasoning to Action Spectrum

Agent workloads sit on a spectrum.

• At one end are reasoning heavy tasks: a small number of agents, each thinking hard for a long time, generating thousands of words of reasoning. The GPU is doing 80%+ of the work; the CPU is ensuring the GPU never runs out of data. A deep research agent writing a 40-page report is a reasoning heavy task.

• At the other end are action heavy tasks: hundreds of agents, each doing lots of small things: reading databases, calling APIs, scraping webpages, etc. The GPU barely participates. The CPU is doing ~80% of the work. A customer operations agent processing thousands of support tickets in parallel is action heavy.

These two profiles map cleanly to the two polestar CPUs of the current generation: Nvidia's Vera for reasoning, AMD's Venice Dense for action. The days of "a CPU is a CPU" are ending. The interesting question is which one is right for which workload and what that says about where the industry is going. The gap between them on each other's turf is wide enough that the two roadmaps can be read as evidence that CPU designs are openly diverging along the reasoning vs action axis.

Left: AMD’s Venice Dense Architecture, Right: NVIDIA Vera - Arm-based data center CPU designed specifically for agentic AI and RL

The Bottleneck is BELOW the CPU Metric

The CPU bottleneck analysis stops one level short of where the engineering work is actually happening: virtual machines (VMs). The CPU bottleneck story is, in fact, actually a VM bottleneck story.

This matters because VMs are expensive to spin up, expensive to snapshot, and designed for a different shape of workload than agents need. The primitives the industry has been using for a decade (Firecracker, Docker checkpoints) assume you boot a machine, run a workload to completion, and tear it down. Agent workloads have a different shape: many short bursts of work, with frequent need to branch state and explore alternatives in parallel. You want cheap forks, cheap commits, cheap resumes. The existing stack makes all three expensive.

The current stack is hyper optimized for a specific shape of workload. The rise of async agent workloads changes the shape. — Ty Daly, Founder/CEO, VERS

What VERS Actually Is

VERS has built a system for managing virtual machines (VMs) the way programmers manage code. The core idea is that every running VM is a branch.

Just as a programmer can branch a codebase to experiment with changes and merge them back in if they work, VERS lets you branch a running VM to test a new configuration or run an update. If something goes wrong, you can kill the branch without affecting the main VM. If the update looks good, you can snapshot the branch and use that snapshot to update the main VM.

Concretely, VERS exposes four operations on VMs:

1. Fork: Create an isolated copy of a running VM. The copy starts in the exact same state as the original, but any changes to the copy don’t affect the original. This lets you test risky changes in a sandbox.

2. Commit: Take a point-in-time snapshot of a VM. Snapshots are read-only, so you can use them as known-good restore points or for reproducible builds.

3. Connect: Share state between branches. If two VMs need to communicate, you can connect them directly without going through a network layer.

4. Resume: Start a new VM from a snapshot. This is how you restore a failed machine or roll back to a known-good state.

If you squint, this is a bit like version control for VMs. Your main VM is the stable base that you branch from and merge changes into. Each branch is an experiment. Each commit is a tag. And just as version control lets you rewind a codebase to any point in its history, Vers lets you rewind a running VM to any snapshot.

The Number That is Not Yet Public

VERS internal benchmarking now shows sub-second VM branching on CPU — P99 latency in the low hundreds of microseconds for a new VM, against a copy-on-write memory system that lets one physical host hold branched state for many concurrent agents by storing diffs rather than full VM images. The team is preparing to publish formal benchmarks against leading cloud agent environment providers in the coming weeks.

If that holds up in 3rd party testing, the efficiency gain is on the order of 100x in the number of concurrent agent workloads a given CPU can serve, relative to approaches that keep a full VM in memory per agent. When an agent is waiting on a tool call, a web fetch, or a model response, its host machine is not actually doing useful work most of the time. Holding the full machine in memory during those waits is wasteful.

This is the problem the team is now pushing to solve next. CPUs and GPUs go brrrr.

Where the Wedge is — RL Environments

The most immediate commercial opportunity is not enterprise, it is reinforcement learning environments for the labs and RL-focused startups now raising substantial capital to train the next wave of agentic models.

AI task horizons are doubling roughly every 7-8 months. The labs are already running workloads that last hours to days. When an environment crashes mid-run at hour 46 of a two-day job, the consequences are bad news bears. At current rates of task-length growth, this problem gets significantly worse every quarter. We have been watching this closely ourselves here at Crucible …

Branchable, snapshotable, resumable VMs are the natural solution. A long-running workload can checkpoint at regular intervals. If the environment fails, the job resumes from the last snapshot instead of restarting from zero. If the lab wants to explore multiple paths from the same starting point, they can fork the VM at any snapshot and run the branches in parallel. This turns a multi-day task into a multi-hour one, not because any individual step is faster, but because you never lose progress.

The other attractive feature of the RL environments wedge: the buyers have money, and they are actively shopping for a solution. The existing RL training infrastructure is not good. The labs that have tried to build environments in-house are realizing that doing so is a serious distraction from their core work, and the ones that have outsourced to hosted environment providers are running into the same state management and checkpoint efficiency limits that VERS is now solving at the primitive level. VERS is in conversations with one of the largest hosted environment providers and has the opportunity to become the branching substrate inside its stack.

The Structural Bet — Where the Margin is Moving

Zoom out to the level of the cluster economics. The various layers of the stack today have very different margin profile:

• Upstream producers turn power and GPUs into GPU hours. VERS’ neocloud peers, for example, target a roughly 2x margin multiple here. This layer is capital-intensive, debt-heavy, and largely commoditized.

• Downstream refiners, primarily inference labs, turn GPU hours into tokens by wrapping orchestration and optimization around raw bare metal compute. Anthropic’s gross margin has skyrocketed from 30% to approximately 90%, showcasing a new phenomenon in the industry.

This margin expansion is the most rapid in the stack right now on a relative basis, driven by a combination of factors: declining cost of token production due to model compression and increasingly powerful chips, coupled with the continued exponential growth in token demand. The interesting question is what happens as the downstream refiner layer consolidates. If the winners in that layer are the ones that can extract the most tokens per GPU hour, the primitive that lets them do that is extraordinarily valuable.

VERS’ pitch to a cluster owner or neocloud is a version of:

There’s a deeper technical shift happening that’s easy to miss if you only focus on the CPU-GPU ratio: tokens might not be the right unit of communication between agents in the first place.

When agents talk to each other today, they use natural language: one agent writes a few sentences, the other agent reads those sentences and writes a few back. This is very natural for humans, but it’s strange for software… like two programs passing notes to each other instead of just sharing memory.

The natural way for two agents to communicate is to pass state directly: not a verbal description of the state, but the actual data structures. If agent A has already fetched a piece of data into its its working memory, it should be able to send that cache entry directly to agent B, without going through the token compression step. If this happens, it will collapse the bandwidth needed for agents to share context, because they’ll be sharing pointers to state objects instead of passing copies of the objects’ text representations back and forth.

The VERS team is thinking about this too. The infrastructure required to pass cache entries between agents is almost exactly the infrastructure required to do VM forking and state commits. A memory architecture that can branch a running process can also pass memory between processes. So whether or not KV cache based communication becomes the norm, VERS is well positioned for it — because the abstractions they’ve built for other reasons map cleanly to the next generation of agent-to-agent communication.

What This All Means

The CPU bottleneck story is being told well by the silicon analysts. The silicon vendors have adjusted their roadmaps.

The layer above the silicon (how agents actually get isolated, branched, coordinated, and recovered) is still under analyzed. It is the layer where the refinery’s margin is made possible, and it is the layer where VERS is focused. Branching is the right primitive for agent compute, the existing VM stack is the wrong shape for agent compute, and the gap between the two becomes more valuable every month that CPU-to-GPU ratios compress and agent task horizons grow.

The commercial thesis has three pieces:

1. RL environments are the near term wedge, where the problem is most acute today and the buyers have budget approved line items ready to go.

2. Self serve prosumer and developer facing use cases are the next natural expansion, with the onboarding problem as the tip of the spear.

3. Embedding into the orchestration layer of neoclouds and inference providers is the structural endgame — where VERS goes from being a point solution for RL to being the substrate that holds up the next generation of agent platforms.

If you made it this far … congrats!!! This is dense content. For real though, if you believe the CPU-bottleneck analysis (and at this point the data makes it hard not to) then the interesting question is not which CPU wins, it is what runs on top of those CPUs when the ratio finishes moving. A large part of the answer is branchable VMs. If you’d like to talk to the VERS Team, ping us or Tynan Daly ASAP! (tynan.daly@hdr.is)

---

Sources and Further Reading:

• Raj, Wang, Krishna — “A CPU-Centric Perspective on Agentic AI” (arXiv:2511.00739, Nov 2025).

• Sekar — “The CPU Bottleneck in Agentic AI” (Vik’s Newsletter, Feb 2026).

• Burke — “Arm’s $15 Billion CPU Opportunity” (Futurum Research).

• TrendForce — “The Great Rebalance: How Agentic AI Is Reshaping the CPU:GPU Ratio.”

• Vers — vers.sh, github.com/hdresearch.

Footnotes:

1. Cores are the brains of a CPU, the parts that actually run software instructions. For agent workloads, each agent maps to one or more threads, so the number of cores determines how many agents can run at the same time without contention. If you have more agents than cores, the CPU has to interleave them (context switch), which adds overhead. So in a very literal sense, more cores means more agents per CPU. It’s why the CPU-to-GPU ratio question matters so much. If you have the wrong ratio, you either starve your GPUs (not enough agent diversity to keep them busy) or you under utilize them (too many agents contending for too few GPU cycles).

2. This is called NUMA, Non-Uniform Memory Access. If you’re running thousands of agents and each one happens to land on a core whose memory lives across the chip, you pay a tax on every operation. Handling this well is a software problem as much as a hardware one, and it’s the reason high-core-count chips (which everyone wants for agents) have a hidden cost that isn’t obvious from the spec sheet.