Ch 31. Splitting the work

In Ch 29, one agent did every step of the job in one conversation. Every tool result stayed in its prompt, so each call read more than the last. On a big job, like comparing ten products, the prompt fills up with search results the agent no longer needs.

The agent in Ch 29 did every step in one conversation. Every tool result stayed in its prompt, so each call read more than the one before. On a big job, such as comparing ten products, the prompt fills with search results the agent has already used, and the harness has to drop or summarize them, as in Ch 24.

Another way is to split the job. A lead agent breaks it into parts and writes a short task for each. Each part goes to a subagent, a helper that starts with a fresh prompt holding only its task, makes its own tool calls, and sends back a short summary. The lead only ever reads the summaries. Anthropic’s research feature works like this.^[1]

Another way is to split the job. A lead agent breaks it into parts and writes a short task for each. Each part goes to a subagent, a helper that is an agent loop of its own: it starts with a fresh prompt holding only its task, makes its own tool calls, and sends back a short summary. The lead never sees the helpers’ tool results, only their summaries. Anthropic’s research feature works this way: a lead agent plans the search and starts helpers that each explore one part of the question with their own context, then pass back what matters.^[1][1]

With three parts of three tool calls each, one agent’s last prompt is 1,110 tokens long. Split up, no prompt is over 540, and because the helpers can work at the same time, fewer calls wait for the one before.

With three subtasks of three tool calls each, one agent makes 10 calls and its last prompt is 1,110 tokens long. Split up, there are 14 calls, but no prompt is over 540 tokens, and because the helpers can run at the same time, only 6 calls wait for the one before. Here the split version even reads fewer tokens in all, 4,260 against 7,050, because no prompt carries every result.

The toy does the same work either way, but real helpers search more widely, so splitting usually costs more. In Anthropic’s data, a system of several agents used about 15 times as many tokens as a chat. It paid off on broad questions that split into separate parts.^[1] It works less well when every part depends on the others, as in most coding, and the lead has to describe each part clearly, or helpers repeat each other’s work.^[1]

The toy keeps the work the same either way. Real systems don’t: their helpers search more widely, so they spend far more tokens. In Anthropic’s data, an agent used about 4 times as many tokens as a chat, and a system of several agents about 15 times. In return, their lead with helpers did 90.2% better than a single agent on their own research test, especially on broad questions that split into independent parts.^[1][1]

It works less well when every part needs the same shared context or depends on the others, which Anthropic says is true of most coding tasks. The lead also has to describe each task well: given vague tasks, their helpers repeated each other’s searches or left gaps.^[1][1] And each helper can have its own instructions and its own tools,^[1][1] so a helper that only needs to read can be given tools that can’t change anything, an idea that comes back in Ch 34.

The arithmetic

One agent doing $N$ subtasks of $s$ tool calls makes $K=Ns+1$ calls and, as in the agent loop, reads $K{p}_0+aK(K-1)/2$ tokens in all, which grows with the square of the work. Split up, the lead’s two calls read $p_0$ and $p_0+Nc$, where $c$ is what each handoff and summary add, and each helper’s $s+1$ calls read $(s+1)h+as(s+1)/2$, where $h$ is a helper’s first prompt. The total,

grows only in proportion to $N$. The widget uses $p_0=300$, $a=90$, $h=150$, and $c=80$.

Anthropic found that how many tokens a system spends explains most of how well it does. On BrowseComp, a test of finding hard-to-find facts on the web, token use alone explained 80% of the differences in score, and the number of tool calls and the choice of model most of the rest.^[1] Splitting the work across helpers is one way to spend more. Their lead also waits for each round of helpers to finish before going on: simpler, but one slow helper holds up the rest.^[1]