LLM Basics
The robot doesn’t think. It predicts. Very confidently.
▶ the lesson · narrated sketch video · tap for sound
the intro
Large language models generate text one piece at a time, predicting what comes next from everything before it. Understanding tokens, the context window, and why temperature changes the output explains almost every weird behavior you will meet.
fun fact ✦
LLMs predict the next token from probabilities — which is why they can be both brilliant and confidently wrong.
the docs
the deep dive
the ideas behind the video — concepts, diagrams, a walkthrough and the traps to avoid
An LLM is an extremely well-read autocomplete. It has read a huge slice of the internet, and its only job is: given this text, what token probably comes next? Reasoning, code, poems — all of it is that one trick repeated thousands of times at speed. Once you see the model as prediction instead of lookup, or thinking, the weird behaviors start making sense.
prediction, not lookup
key terms
token
The chunk of text the model actually reads — a word, a piece of a word, or punctuation. Billing and limits count tokens, not letters.
"strawberry" ≈ 3 tokenscontext window
How much the model can see at once, measured in tokens. Instructions, history, and the answer all share one budget.
200k tokens ≈ a 500-page booktemperature
The randomness dial. Low values pick the most likely token every time; high values take risks and vary the output.
0 for code, 0.7 for brainstorminghallucination
A fluent, confident answer that is simply wrong. It is a side effect of prediction, not a bug you can patch away.
invented citations, real-lookinghow it works
- 1Split into tokens
Your text becomes tokens, the units the model reads and bills you for.
- 2Read everything at once
The model weighs every token in the context window simultaneously. That mechanism is attention, from the 2017 transformer paper.
- 3Score the next token
It assigns a probability to every possible next token and samples one. Temperature shapes the sampling.
- 4Append and repeat
The chosen token joins the input and the loop runs again, until a stop condition or the token limit ends it.
the code, explained
one reply, token by token
side by side
low temperature
- same prompt, near-same output
- best for code, math, facts
- the safe default for products
high temperature
- varied wording, fresh ideas
- good for brainstorming
- more chances to go off-rails
- creative writing, ad copy
Ask a model to count the r's in "strawberry" and it can fail — not because it is dumb, but because it never saw letters, only tokens. Counting, spelling, and arithmetic live at the wrong layer of the stack. The fix is a tool that runs real code, not scolding the model.
quick answers
Is the model storing my data?
It stores nothing between calls. Each request is stateless — you resend the whole history every time, which is also why long chats get expensive.
Why does it get math wrong?
It predicts plausible-looking text, and "roughly right" reads as plausible. A calculator or code tool does the actual arithmetic.
What is a context window, in practice?
The token budget for one request: system prompt plus conversation plus answer. Overflow it and old messages get dropped or the call fails.
quick hits about LLM Basics
short answers · interviews, tips, trivia & hidden gems