LLM Basics: Tokens, Context and Temperature

Large language models (LLMs) predict the next token word by word. Knowing the basic terms helps you use them better.

Tokens

A token is a piece of text — about 3-4 characters in English. 1,000 tokens ≈ 750 words. Prices and limits are calculated in tokens.

Context window

The window (e.g. 8k, 32k, 128k, 1M tokens) limits how much text the model can “see”. Anything beyond must be summarized or omitted. Longer context costs more compute and can reduce quality (“lost in the middle”).

Temperature

0.0-0.3: deterministic — for code, facts, extraction.
0.7-1.0: creative — for text, ideas.
>1.0: very random — rarely useful.

Top-p and top-k

Top-p (nucleus sampling) selects from the most likely tokens that together make up p. Top-k limits to the k most likely. Together with temperature they control randomness.

Costs

Input (prompt) is usually 3-10x cheaper than output. Caching, smaller models for simple tasks and batch processing reduce costs.