LEARN, BUILD, APPLY

This is the central question, and its answer is disarmingly elegant: since the LLM can only produce text, let's make tool use… text.

The mechanism, in principle, takes four steps:

1. Describe the tools to the model, in text. In the prompt (typically the system prompt, the hidden instruction that precedes the conversation), you write something like: "You have a tool weather(city) that returns current weather, and a tool calculator(expression) that evaluates a math expression. To use a tool, write name(arguments)."

2. The model generates a call instead of an answer. Faced with "What's the weather in Chiang Mai?", the model — if it has learned to — generates weather("Chiang Mai") and stops.

3. Ordinary code intercepts that text and actually runs the tool. This is the crucial point: the LLM executes nothing itself. A plain piece of code (roughly, a Python loop) detects the call, queries the real weather API, and gets the result: .

4. The result is injected back into the context, as if it were the next part of the conversation, and the model resumes generating: "It's currently 31°C in Chiang Mai, with thunderstorms…"

Reread those four steps: at no point did we modify the Transformer's architecture. Everything happens in the text — tool descriptions, call, result. This is what we now call function calling, and the loop that executes things is called the runtime or orchestrator. Three remarks worth keeping, because they come back constantly:

• The model decides; the runtime executes. This separation is fundamental, including for safety: you can refuse to execute a call, ask a human for confirmation, or restrict the available tools (sandboxing).
• Everything lives in the context. Tool descriptions, calls, results — all of it must fit in the model's context window. Now you know why the size of that window (4k tokens yesterday, hundreds of thousands today) is a major industrial battleground.
• Format matters enormously. In practice, calls are emitted in JSON (a standard text format for structured data: ) and tools are described by a schema (name, description, expected parameters and their types). The clearer the description, the better the model uses the tool — it is prompt engineering applied to tools.

Here is how it actually took hold.

2020–2021 — The first hacks. As early as GPT-3, researchers noticed that answers improve when
you inject relevant documents, found by a classic search engine, into the prompt. This is the birth of
RAG (Retrieval-Augmented Generation, Lewis et al., 2020) — technically a cousin of function
calling: the context is augmented with external information, but it is the system, not the model, that
decides what to fetch. In late 2021, OpenAI unveils WebGPT: a GPT-3 fine-tuned to navigate a text
based browser (search, click, cite sources) to answer questions. The first demonstration that an
LLM can drive a complex tool.

February 2023 — Toolformer, the paper you’re about to read. Meta AI (Schick et al.) frames
the question differently: can a model learn to use tools without massive human demonstrations?
Their answer is a model that teaches itself where to insert API calls into text, keeping only the calls
that genuinely improve its predictions. It is a pivotal paper — simple in its idea and deeply revealing
of the connectionist method: even tool use is not programmed, it is learned.

March–June 2023 — Industrialization. OpenAI launches ChatGPT plugins (March), then exposes
function calling directly in its API (June): developers provide the list of tools as JSON Schema, and
the model answers with structured calls. Anthropic, Google, and open models (Llama, Mistral,
Qwen) follow with the same de facto standard. Tool calling stops being a research hack and
becomes a core feature, explicitly trained into models — the chat templates of models on Hugging
Face now include special tokens dedicated to tool calls.

2024–2025 — Standardization. With every vendor describing tools its own way, the ecosystem
fragments. In late 2024, Anthropic publishes MCP (Model Context Protocol), an open protocol for
connecting any data source or service to any model — a kind of “universal USB port” for tools,
widely adopted in 2025. At the same time, tools become spectacular: code execution in sandboxes,
full computer control (mouse, keyboard, screenshots — computer use), web browsing… The line
between “answering” and “acting” blurs. That is exactly the subject of the next post in this series,
on agents.

Take a minute to place this evolution in the world / calculator / horizon grid from the first post.

• The world of the pure LLM was its training corpus — a frozen world, absorbed once and for all into the weights. With tools, the world comes alive again: the model queries reality at the moment of the question. Three quarters of a century later, we are back to Wiener’s cybernetic loop: perceive, act, measure the gap, correct.
• The calculator becomes hybrid. The neural network (intuitive, statistical, fallible) delegates to classical programs (exact, deterministic, verifiable) what they do better: calculating, searching, executing. Some see it as a quiet revenge of symbolic AI: the calculator, the Python interpreter, the database are rule machines — and the network learns when to invoke them. The two paradigms of Cardon’s paper, reconciled in practice.
• The horizon widens: from “produce the most plausible text” to “accomplish the requested task,” relying on verifiable tool results. Hallucination does not disappear, but it recedes wherever a tool can ground the answer in a source.

The paper “Toolformer: Language Models Can Teach Themselves to Use Tools” (Schick et al., 2023, about 20 pages with appendices) is very readable. Roadmap:

1. Abstract + Introduction: the general idea and the five tools used (calculator, question answering, search, translation, calendar). Note their deliberate simplicity.
2. Section 2 (the method — the heart of the paper): read it twice. The self-teaching mechanism takes three steps: (a) the model itself, prompted with a few examples, annotates texts with candidate API calls; (b) those calls are executed; (c) only the calls whose result reduces perplexity — that is, genuinely helps the model predict what comes next — are kept. The model is then fine-tuned on these filtered, annotated texts.
3. Figure 1 and the examples: concrete, and worth a thousand equations.
4. Sections 3–4 (experiments): skim. Remember the headline result: a 6.7-billion-parameter model equipped with tools beats GPT-3 (175 billion) on several tasks. Tools can substitute for scale — a point worth pondering.
5. Limitations (end of the paper + your own critical eye): Toolformer cannot chain tools, nor use the result of one call to decide on another. Hold on to that frustration: it is precisely what leads to agents.

Three questions to carry through the paper: Why is the criterion “does the call reduce perplexity?” such a good idea? What does it
guarantee — and what does it not? How is Toolformer’s method faithful to the “inductive move” described by Cardon (empty the calculator, let the data decide)? What’s the difference between modern function calling (section 1 of this post) and Toolformer’s approach? (Hint: training vs. prompting.)

API (Application Programming Interface) — A service's standardized entry point: you send a formatted request ("weather in Chiang Mai?"), you receive a structured response. LLM tools are almost always APIs.
Context window — Everything the model "sees" when generating: system prompt, conversation, tool descriptions, call results. A limited and precious resource.
Few-shot / zero-shot — Using the model with a few examples in the prompt (few-shot) or none (zero-shot). Toolformer uses few-shot prompting to bootstrap its self-annotation.
Function calling — An LLM's ability to emit, as structured text, a request to execute a tool described in its context. The model asks; the runtime executes.
Grounding — Anchoring the model's answers in verifiable sources (tool results, documents, calculations) rather than in its statistical memory alone. A partial antidote to hallucination.
Hallucination — Producing false information with the confidence of truth. A structural consequence of the "predict the plausible token" objective.
JSON — The universal text format for structured data (``). The lingua franca of tool calls.
JSON Schema — A formal description of a tool: its name, its parameters, their types. What the model reads to know how to call it.
Knowledge cutoff — The end date of the training corpus. Beyond it, the model knows nothing — unless it has a search tool.
MCP (Model Context Protocol) — An open protocol (2024) standardizing connections between models and tools/data sources. The "USB port for tools."
Orchestrator / runtime — The ordinary program that runs the loop: send the context to the model, detect calls, execute tools, inject results back.
Perplexity — A measure of the model's "surprise" at a text: low = the model predicts well. The central filtering criterion in Toolformer.
RAG (Retrieval-Augmented Generation) — An architecture that searches for relevant documents (in a database, on the web) and injects them into the context before generating an answer.
Sandbox — An isolated, restricted execution environment where the model's actions (code, for instance) run without risk to the host system.
Special token — A reserved vocabulary token (invisible in normal text) used to mark structure: start of a tool call, end of a message, and so on. Recent models have them for function calling.
System prompt — An instruction invisible to the user, placed before the conversation, that defines the model's role and, often, its tools.