Wrap-up

Ch 46. Check yourself

Sixteen questions

Twenty questions

One or more questions from every Part, in reading order. Pick an answer for each, then check them all at once. Each answer links to the chapter that explains it, and any you miss are gathered into a short list of chapters worth another look.

Try it

Quick check

Computed in your browser

What does raising the temperature do when a model picks the next word? Unlikely words get picked more often The model thinks for longer More words are added to its vocabulary The model runs faster
Why might a model miscount the letters in “strawberry”? It reads tokens, not letters It was never shown the word Counting is banned in training Its temperature is too high
Where do a model’s weights come from? Programmers type them in Training nudges every weight to make the right next token a little more likely They are copied from a dictionary They start random and stay that way
How does a model get good at guessing the next word? It practices on lots of text and is nudged a little after each mistake Programmers write a rule for every word It looks each word up in a dictionary It is born knowing the language
What does a language model’s training loss, the cross-entropy, measure? The average of −ln p over the real next tokens: how surprised the model was How many weights changed in the last step The distance between two embeddings How many tokens the model has read
How does attention help with “The trophy didn’t fit in the suitcase because it was too big”? It looks up the answer online It lets each token pull in information from any earlier token It makes the vocabulary bigger It removes the word “it”
What does the model itself produce each time it runs? A whole reply Probabilities for the next token A search of the web A list of facts it checked
You round a model’s weights from 16 bits to 4 bits each. What happens? It needs about a quarter of the memory, and coarse rounding can hurt its answers It learns new facts Its vocabulary shrinks to 16 tokens Nothing at all
A small model drafts tokens and a big model checks them. How does the text compare with the big model writing alone? It is exactly the same, written in fewer passes of the big model It is shorter It mixes the two models’ styles It is the small model’s text
When the big model samples, how does guess-and-check keep its text exactly as likely as before? It keeps a guess with chance min(1, p/q) and redraws a rejected one from what’s left over It only accepts guesses the small model was sure of It averages the two models’ probabilities It lowers the temperature of both models
What does a reasoning model do differently from an ordinary chat model? It looks answers up in a database It runs a separate logic engine It writes scratch work as tokens before answering It never makes mistakes
You add four worked examples to a prompt. What changes inside the model? Its weights are updated to learn the task Nothing but its context: attention can copy the pattern from the examples It switches to a different, specialized model Its vocabulary grows
You put four examples in a prompt. Does the model learn anything that lasts? No: it uses the examples while it reads this prompt, and the model itself doesn’t change Yes: it saves the examples for every future chat Yes: it retrains itself on them No: models ignore examples
A chat app re-sends its whole, growing conversation on every call. Why is that often cheaper than it sounds? The model remembers the earlier calls The provider reuses its work on the unchanged start of the prompt The harness compresses the text Long prompts are free
A chat app sends its whole conversation again on every call. What keeps that from being as slow and costly as it sounds? The provider saves its work on the start of the prompt and reuses it The model remembers what it read last time The conversation is sent as a picture Nothing: every call costs the same as the first
Without a KV cache, how does the work for n new tokens grow? With the square of n, because every step reruns all the text so far In step with n It stays the same however many tokens It shrinks as the text gets longer
When an assistant uses a calculator, who actually runs it? The model The harness The tokenizer The user
Why can a hidden line in a web page change what a summarizer writes? The page breaks into the computer running the model The model reads the page and your request as one stream of text Web pages were part of its training It can’t; models ignore instructions in documents
A web page tells the agent to delete your files. What stops it? The model always ignores web pages Guardrails in the harness, like asking you first The context window Nothing can help
Why can BERT fill in a blank in the middle of a sentence better than GPT-2? It has a bigger vocabulary Its attention can look at the words after the blank too It was trained on more text It uses a higher temperature
Mixtral sends each token to 2 of its 8 experts. Why does a token use 13 of its 47 billion weights, more than a quarter? Attention and the embeddings are shared by every token; only the MLPs are split The router itself is half the model Every expert runs a little for every token The quarter is rounded up to be safe
A Decision Transformer is asked to aim for a higher score than any run it was trained on. What does it do? It invents a better route than any it has seen It plays like the best runs it has seen It refuses to move It tries random moves until one scores higher
Which of these jobs suits a language model on its own best? Adding up 10,000 sales figures Saying from memory who won a race in 1987 Rewording an email you’ll read before you send it Multiplying two four-digit numbers

Thanks for reading. If something here was unclear or wrong, the sources for every claim are listed at the bottom of each chapter.