What Happens on the CPU When an AI Agent Works?

The answer arrives in rounds.

Why does one question take sixteen steps? Because the chat window shows you two things, your request and the answer, and hides everything in between. Let’s open that gap up.

Here is what actually happens. A coding agent does not answer your question. It asks a model a much smaller one: what should I do next? The model replies with an action, such as search the repository for the formatter. Software around the model, the agent harness, carries that action out on a real computer, collects what came back, and asks the model again: now what? That is one round. One user turn can contain many model calls. The task is a stack of rounds, and it ends only when the model says it is finished, a limit is reached, or something breaks.

Follow the trace and you can count the rounds. The first useful action is not an answer. The agent has to locate the formatter and its tests. Search results identify the files. The source reveals the faulty condition. A compiler checks the program. A failing test confirms the behavior. That failure is not the end of the task. It is new information for the next round.

How many rounds is normal? There is no typical number, but there are measured ones. In the Autellix systems study, the evaluated ShareGPT and BFCL traces averaged 6.66 and 10.75 model calls, with maxima of 80 and 70; a tree-search LATS workload averaged 159.7. Those are research workloads, not a typical count for Codex. They make one point: a single chat turn is not a meaningful unit of hardware work.

Each round covers real distance. The request leaves your machine, waits in a queue, is prepared by CPU-side serving code, runs on an accelerator, and travels back, all before the harness can start the tool. Then the tool creates its own processes and its own reads and writes. Notice what this does not mean: there are not hundreds of identical CPU-to-GPU hops, and not every step uses both kinds of chip. The number and shape of the handoffs depend on the program, the serving system, batching, where the tools live, and when the loop is told to stop.

One more thing hides in the word context. The session record and what the model actually sees are not the same. A long-running agent keeps a durable history but retrieves, summarizes, or selects only what the next request needs. Choosing that is work too: it costs CPU time, memory, storage, and sometimes a network call before the accelerator sees a single token.

A tool call becomes work on a real computer.

What does “run the tests” actually make a computer do? Let’s take one tool call apart. “Search the repository,” “read this file,” and “run the tests” are structured requests, not the work itself. The harness must translate each one into operations a computer can perform.

Start with the search in Figure 1, step six. The harness asks the operating system to start a program, gives it permissions and a place in the filesystem, and hands it the command. That program may start others. A shell can launch a search tool; a package script can launch a compiler; a test runner can create workers. The operating system schedules their instructions, maps memory for them, serves their file requests, captures their output, and reports whether they exited successfully. Every one of those things is CPU work.

You will hear the word sandbox for the fence around this activity. It describes permissions, not one universal kind of machine. Depending on the product, an agent may be constrained by operating-system rules, a container, a virtual machine, or several layers together. The fence controls which files, networks, credentials, and system calls a process can reach. It does not make the work abstract.

The measured fixture ran a small version of this path, and you watched it: an isolated working directory, a bounded search, one file read and one file replaced, two compiles, and the same tests run before and after the edit. The first run failed in the expected way. The second passed all three assertions. For each subprocess the capture kept elapsed time, coarse user and system CPU time, peak resident memory, and the exit code.

Those numbers need careful reading; steps eight and twelve in Figure 1 are the place to look. Elapsed time is how long a person waits for a command to return. CPU time counts up while cores execute that process’s instructions, so a compiler spread across several cores can rack up more CPU time than wall time; that is exactly what the two compile steps show. A process can also spend its wall time waiting: for a file, a child process, a lock, or a network reply. One duration cannot tell you which part was busy at every instant.

Files carry the same ambiguity. The byte length of a source file is not physical device traffic. The file may arrive from a cache in memory; metadata and dependencies may cause more reads than the named file; writes may be buffered. The fixture records the logical bytes it handled, not total filesystem traffic, cache behavior, or storage queueing.

Network is plural as well. The laptop talks to a harness service; the harness calls a hosted model; a tool may use local input and output or remote HTTP; a build may reach a package registry; a browser may contact many origins. A single model-latency number can include transport and provider-side waiting that the client cannot separate.

One concrete architecture makes the roles visible.

Where does each of those steps physically live? Agent systems can place these layers in different locations. Figure 2 chooses one tangible arrangement so the path does not dissolve into boxes labeled “compute.” Here is how to read it.

At one end is a MacBook Pro running Codex: the only part you can see. Between it and everything else sits the control layer, the harness that keeps the session and routes tools; it runs on a server somewhere, so the figure shows it as the log it writes rather than as a box. The work itself lands on a test machine. In the figure that is a Linux container, Node.js, TypeScript, and an NVMe workspace, drawn as one logic board. The observed fixture was narrower than that picture: an Intel Core i7-9750H host running Darwin/x64, an isolated temporary directory, and a bounded set of search, compile, edit, and test operations. It is a laptop-class chip, and every measured step in this task ran on it.

At the far end is the model server, and here the figure borrows NVIDIA’s published Vera Rubin platform because it makes the roles legible. Think of that board as two workers at a shared desk. Vera, the CPU, runs the serving program: it takes the request off the network, prepares it, and feeds the accelerator. Rubin, the GPU, does the model’s arithmetic. HBM4 is the desk Rubin works from, the memory that holds the model’s weights close enough to read at full speed. NVLink-C2C is the door between the two workers, wide enough that they can read each other’s memory instead of copying it. Around them, ConnectX-9 is where the request arrives, BlueField-4 moves data so the CPU does not have to, and Spectrum-6 connects the board to the rest of the building. Model serving is already a whole system, not a GPU in isolation.

Now the honest part. That board tells you where work could live. It does not say how Codex is deployed, which device handled this task, how much time or energy each part used, or whether a queue at the provider dominated the wait. The trace did not run on Vera Rubin and did not observe its coherent link. The event record says what the fixture actually saw; the board is there so the words CPU, GPU, and memory point at something real.

Memory needs the same care, because the word covers several different things. The laptop’s working memory, the operating system’s file cache, a sandbox’s saved state, the GPU’s weights, its activations, and its KV cache are not interchangeable. Capacity is not bandwidth. Peak use is not average use. Allocated memory is not consumed memory. A useful trace names the memory it measured and leaves the rest unknown.

The slowest component is not always the bottleneck.

The compiler took 1.4 seconds and the model took an unknown time, so which one made you wait? A task finishes when its chain of dependencies finishes. The component holding that chain open can change from one step to the next.

Look at the two compile steps in Figure 1, then at the three model steps that carry no number. During a compile, several cores are busy and nothing else can proceed; during a model call, the same cores sit idle while a remote request is answered. A warm file read is almost invisible; a cold dependency install can dominate a task. A model endpoint can be slow because of queueing, batching, transport, or the inference itself. So faster model serving helps only the model spans, and faster execution helps only the environment spans, and only when they are the ones the chain is waiting on.

Put many agents on the same hardware and the machine changes without the task changing. Agents wait for a model slot, a tool worker, a sandbox, a process slot, a CPU core, a storage operation, a connection, a rate limit, or a human approval. Batching can raise accelerator throughput while making the first token slower. Shared CPU caches, memory channels, storage, and network links become contended even when a single-agent run looked comfortable.

That is why a trustworthy systems trace needs parent-child relationships, monotonic timestamps, queue boundaries, and a distinction between active work, transport, blocking, and waiting on a dependency. Add every component’s duration together and you double-count overlap. Look only at elapsed time and you hide who worked and who waited.

Sources: OpenAI documentation on the Codex loop, harness, tool execution, and sandbox safety; Anthropic engineering on managed agents and containment; Autellix on multi-call agent programs; NVIDIA Vera CPU and Rubin platform documentation; and the disclosed MTS fixture. Direct links and qualifications appear in Methodology.