Ch 44. Is a model the right tool?

You’ve now seen what a language model does: it writes the likely next token, again and again, using only what’s in its prompt and what training left in its weights. That tells you when it will help.

It helps most with writing. In one experiment, 453 professionals did writing tasks from their own jobs, and half of them could use ChatGPT. Those who could finished 40% faster, and their work was rated 18% better.^[1] But it can also make work worse. Consultants who used GPT-4 did better on 18 tasks it was good at, but on one task just beyond it they got the right answer less often than those working alone.^[2] For other jobs, a calculator, a search, or an expert does better. Four questions tell them apart.

Everything so far comes down to one fact: a language model writes the likely next token, again and again, from what’s in its prompt and what training left in its weights. That fact tells you when to reach for one, and when not to.

Writing is where it shines. In a preregistered experiment, 453 college-educated professionals did writing tasks from their own occupations, and half of them, chosen at random, could use ChatGPT. They took 40% less time, and graders rated their work 18% better.^[1][1]

Other jobs go the other way, even ones that look just as hard. In another preregistered experiment, 758 consultants at Boston Consulting Group were given realistic consulting tasks, some with GPT-4 and some without. On 18 tasks within its abilities, those using it finished 12.2% more tasks, 25.1% faster, and with better quality. On one task chosen to be just beyond its abilities, they were 19% less likely to reach a correct answer than those working alone. The researchers called the uneven line between the two a “jagged frontier”: you can’t tell from how hard a task looks which side it’s on.^[2][2] Four questions, each from an earlier chapter, help tell them apart.

Would a likely-sounding answer be a good one? For rewording, summing up, drafting, and brainstorming, yes: you want text that reads well. For a sum or a date, you want one exact answer, and the likely-looking one can be wrong. Without a calculator, the model in Ch 28 worked out 1234 × 5678 step by step and got 7,050,952. The answer is 7,006,652.

Can you give it everything it needs to know? It knows only its prompt and its training: nothing about your files, nothing after its cutoff, and little about rare things. Asked ten times who wrote a novel made up for this site, it named nine different authors (Ch 32). Paste the facts in, or let the app search for them (Ch 25).

Can you check the result quickly, and live with a mistake? The model sounds just as sure when it’s wrong. Asked ten times who won the 1987 Tour de France, it said Eddy Merckx every time; it was Stephen Roche. So a model saves time only when checking its work is quicker than doing the job yourself. For anything that matters, have someone who knows check it.

Is there a simpler tool that gives the exact answer? A calculator, a spreadsheet, or a search engine is exact, fast, and free, and gives the same answer every time. For the same sum over 10,000 rows, write the formula once. A model can help you write it.

Would a likely-sounding answer be a good one? Rewording, summarizing, drafting, translating, and brainstorming ask for text that reads well, which is exactly what the model predicts. A product, a date, or a name asks for one exact answer, and the likely-looking one may not be it (Ch 16). Without a calculator, the model in Ch 28 worked out 1234 × 5678 step by step and got 7,050,952. The answer is 7,006,652. The fix is a tool, or doing the exact part in code.

Can you give it everything it needs to know? A model knows only its prompt and what training left in its weights: nothing after its cutoff, nothing about your files or last week’s meeting, and little about rare topics. Asked ten times who wrote The Lanterns of Velmora, a novel made up for this site, it named nine different authors (Ch 32). Put the facts in the prompt, or have the harness search for them (Ch 25). In Ch 24, once the first exchange was dropped, the model couldn’t say which child loved the cat rhyme; with the exchange retrieved, it said Lily.

Can you check the result quickly, and live with a mistake? The model sounds just as sure when it’s wrong. Asked ten times who won the 1987 Tour de France, it said Eddy Merckx every time; it was Stephen Roche. In Ch 27, the small model was 79% sure that Sydney is Australia’s capital. So a model saves time only when checking its work is quicker than doing the job yourself. A reworded email takes seconds to read; a summary of a report you haven’t read is hard to check without reading the report; a contract clause needs a lawyer. Where mistakes are costly, keep a person in the loop (Ch 34), and test before you rely on it (Ch 35).

Is there a simpler tool that gives the exact answer? A calculator, a spreadsheet formula, a database query, or a search engine is exact, fast, and free, and gives the same answer every time. A model costs tokens on every call, and its answers can change from one call to the next (Ch 2). Even the short call in Ch 43 had the model read a 400-token prompt, so ten thousand rows would be four million tokens. Write the rule once instead; a model can help you write it.

The best setups mix the two. The model does the wording, a tool does the exact part, the app fetches the facts, and checks catch the mistakes. That is what the apps in the Harnesses chapters do (Ch 23).

The questions rarely say “never”. More often they say “not on its own”: the model does the wording, a tool does the exact work (Ch 28), the harness fetches the facts (Ch 25), and checks catch the mistakes (Ch 35). That division of labor is what a harness is for (Ch 23).

The checking rule, written down

Say doing a job yourself takes $T$. With a model, you spend $a$ asking, $c$ checking, and, with probability $p$ that the result is wrong, $f$ fixing it. The model saves time when

Drafting does well: $c$ is a quick read and $f$ a quick edit. An unread report does badly: $c$ is close to $T$, since checking means reading it. High stakes make $f$ large: a wrong clause in a contract costs far more than the time it took to write.

Agreement isn’t a check of truth

The consistency check in Ch 32 asked each of 11 questions ten times and trusted the most common answer only when 8 or more of the ten agreed. That check flagged 4 of the 5 wrong answers and none of the 6 right ones. It missed the Tour de France: all ten answers agreed, and all were wrong. Agreement measures how sure the model is, not whether it is right, so a check you can automate still needs a source of truth for what matters.

Who gains most

In the writing experiment, the gap between the stronger and weaker writers shrank: those who scored lower on their own gained the most from the help.^[1]