LLM Kitchen

Part VIII · Agents

Ch 28. Tools

Arithmetic, one token at a time

Ask a model to multiply two big numbers and it writes the answer the only way it can: one likely-looking token after another (Ch 16). Often that goes wrong. Now give it a calculator:

Ask a model to multiply two four-digit numbers and it writes the answer the only way it can: one likely-looking token after another (Ch 16). Sometimes that works. Often it doesn’t. Now give it a calculator:

Step through

Tools off, tools on

Recorded · Qwen2.5-1.5B-Instruct

“What is 1234 × 5678?”

The model’s reply, written token by token

To calculate \( 1234 \times 5678 \), you can use the standard multiplication method:

\[
\begin{array}{r}
     1234 \\
   \times 5678 \\
   \hline
    98072 \\  (1234 \times 8) \\
   +61700 \\  (1234 \times 7, shifted one place to the left) \\
+617000 \\  (1234 \times 6, shifted two places to the left) \\
+6170000 \\  (1234 \times 5, shifted three places to the left) \\
\hline
   7050952 \\
\end{array}
\]

So, \( 1234 \times 5678 = 7,050,952 \).

Its answer: 7,050,952. The right answer is 7,006,652. Every digit was predicted like any other token.

Ask Qwen2.5-0.5B-Instruct your own sum, with and without a calculator, live. Choose a model tier to run a real model in your browser.

Without tools, the model wrote out a long multiplication and got 7,050,952. The right answer is 7,006,652. With a calculator, it got it right, because it never did the arithmetic at all.

Where the long multiplication went wrong

The model laid out the method correctly: multiply 1234 by each digit of 5678, shift, and add. Only the last of its four rows was right. It wrote 98,072 for 1234 × 8 (really 9,872), 61,700 for 1234 × 70 (really 86,380), and 617,000 for 1234 × 600 (really 740,400); only 6,170,000 for 1234 × 5000 was right. Its rows don’t even add up to its own answer: they make 6,946,772, not 7,050,952. Each number is a run of likely-looking tokens, and nothing checks that the steps agree.

A tool call is just text

The model can’t press buttons or run programs. A is just text the model writes in a fixed format. The harness spots that text, runs the tool itself, adds the result to the conversation, and calls the model again.

The model can’t press buttons or run programs. A is text the model writes in an agreed format, like the answer line in Ch 21, but for the harness to act on. Models can be trained to write calls like this at the right moments.[1][1] Then:

  1. The harness watches the model’s output for that format and stops it there.
  2. It reads out the tool’s name and arguments, and runs the tool itself.
  3. It adds the result to the conversation and calls the model again.

Every model family has its own format. Qwen writes a small block of JSON between <tool_call> tags. On this site, calls are shown in a simpler made-up form, like <call calculator>1234 * 5678</call>, so they’re easy to read.

How a model learns to call tools

Toolformer showed one way to teach this without people labeling examples. A model was shown a few demonstrations of each tool, then wrote candidate calls into ordinary text; the calls were run, and a call was kept only if its result made the following tokens easier to predict. Fine-tuning on the kept examples taught the model when a call helps.[1] Qwen2.5 was tuned for tool use, and its vocabulary reserves two control tokens for it: <tool_call> and </tool_call> are each a single token.[2] Between them, the model here wrote {"name": "calculator", "arguments": {"expression": "1234 * 5678"}}.

The tools in these chapters are toys that run in your browser, like a calculator and a pretend file list. Real assistants use the same pattern for web search, running code, and sending email, which is why the harness has to be careful about which tools it runs.

The tools in these chapters are toys that run in your browser: a calculator, a search over the six nursery rhymes from Ch 1, a word counter, a fake web page, and a pretend file list. Real assistants use the same pattern for web search, code execution, sending email, and more, which is exactly why the harness has to be careful about which tools it runs.

One plug for many tools

At first, every app connected every tool its own way. The , released by Anthropic in November 2024, gives tools a shared plug.[1] A program called an MCP server offers some tools, and any harness that speaks MCP can use them.[2]

At first, every app wired up every tool its own way. The , which Anthropic released as an open standard in November 2024, gives tools a shared plug.[2][3] A program called an MCP server offers some tools, each with a name, a description written for the model, and the inputs it takes, such as a get_weather tool that needs a location. Any harness that speaks MCP can ask a server for its list of tools, describe them in the prompt, and pass the model’s calls on to the server to run.[3][4]

The model never sees MCP. It still reads tool descriptions and writes tool calls in its own format; the harness translates in both directions. Connecting a server means its descriptions and results end up in the prompt, so it has to be trusted like any other tool.

Those descriptions take up tokens on every call, and so does more you never see: when tools are on, Anthropic’s API adds its own system prompt for tool use, a few hundred tokens depending on the model, before counting the tools’ own descriptions.[3][4][5]

MCP on the wire

MCP messages are JSON-RPC 2.0: small JSON objects with a method name and parameters. A harness sends tools/list and gets back each tool’s name, description, and inputSchema, a JSON Schema for its inputs. To run one, it sends tools/call with the tool’s name and arguments, such as a location of New York, and gets back a list of content, usually text, which it puts into the prompt as the tool result. A server can also tell the harness when its list of tools changes.[4]

Recipe card

Recap

  • Models are unreliable at exact work like arithmetic, because every digit is a predicted token.
  • A tool call is text in an agreed format. The model asks; the harness runs the tool.
  • The harness adds the result to the prompt and calls the model again.
  • MCP is a shared way for a harness to find tools offered by other programs and call them; the model still just writes text.
  • Qwen marks a call with reserved tokens; MCP carries tools/list and tools/call between the harness and a server.

Next question: one tool call helps. What if a task needs several, one after another?

Sources

  1. Anthropic (2024). Introducing the Model Context Protocol. Anthropic.
  2. Model Context Protocol (2026). Tools (Model Context Protocol specification). modelcontextprotocol.io. Accessed October 23, 2026.
  3. Anthropic (2026). Pricing. Claude Developer Platform documentation. Accessed October 25, 2026.

Sources

  1. Timo Schick et al. (2023). Toolformer: Language Models Can Teach Themselves to Use Tools. NeurIPS 2023.
  2. Anthropic (2024). Introducing the Model Context Protocol. Anthropic.
  3. Model Context Protocol (2026). Tools (Model Context Protocol specification). modelcontextprotocol.io. Accessed October 23, 2026.
  4. Anthropic (2026). Pricing. Claude Developer Platform documentation. Accessed October 25, 2026.

Sources

  1. Timo Schick et al. (2023). Toolformer: Language Models Can Teach Themselves to Use Tools. NeurIPS 2023.
  2. Qwen Team et al. (2024). Qwen2.5 Technical Report. arXiv.
  3. Anthropic (2024). Introducing the Model Context Protocol. Anthropic.
  4. Model Context Protocol (2026). Tools (Model Context Protocol specification). modelcontextprotocol.io. Accessed October 23, 2026.
  5. Anthropic (2026). Pricing. Claude Developer Platform documentation. Accessed October 25, 2026.