Temperature parameter

Temperature rescales the probability distribution over the next token before sampling. Low values sharpen it towards the most likely token, high values flatten it. It is applied at inference and changes nothing in the model.

Randomness is the variability in which token gets chosen when several are plausible. Without it, the same input always yields the same output. Temperature controls how much of it there is.

The probability distribution is the model's assessment of how likely each possible next token is. Temperature reshapes it before anything is drawn. The model's judgment is unchanged; the sampling from it is not.

Logits are the raw scores the model assigns to candidate tokens before they become probabilities. Temperature divides them. Everything else follows from that single division.

The softmax layer converts adjusted logits into probabilities that sum to one. It is the step where scores become a distribution. Temperature enters immediately before it.

Tokens are the units generated one at a time, each selected from the distribution. Every token is a fresh draw. Small differences compound across a long output.

Inference is the phase where a trained model produces output. Temperature exists only here. It is a runtime setting rather than a property of the model.

Creativity is the usual explanation and the empirical findings are weaker than the claim. Higher temperature correlates only weakly with judged novelty and more clearly with reduced coherence. It controls variability, and variability is not the same thing as invention.

Predictability rises as temperature falls, because high-probability tokens are chosen more consistently. At zero the output is deterministic. That is valuable wherever repeatability matters more than variety.

Novelty is an evaluated property of output that correlates weakly and positively with temperature. The relationship exists and is small. Treating temperature as a novelty control overstates what the evidence supports.

Coherence is textual consistency, and it correlates moderately and negatively with temperature. Raising temperature costs coherence more reliably than it gains novelty. That trade-off is the practical finding.

Top-p sampling selects from the smallest set of tokens whose cumulative probability passes a threshold. It works alongside temperature rather than instead of it. The two interact and are usually tuned as though independent.

Chunking splits text during preprocessing so that it can be handled in pieces. It belongs to the pipeline around generation. It appears in the consensus because it shares the vocabulary of preparation.

It explains the dial and its felt effect. It leaves out the mathematics it comes from, every other sampling method, every logit-level control, the entire calibration literature that treats temperature as a correction rather than a style choice, and the distinction between not knowing and being random.

Softmax temperature scaling is the formal operation: divide logits by a scalar before the softmax. Documentation describes user-facing tuning rather than the operation. Knowing it explains why the effect is multiplicative rather than additive.

Nucleus sampling is the formal name for top-p, defined by a cumulative probability mass. Texts use informal terms instead. The formal definition makes the interaction with temperature calculable.

Top-k sampling truncates to a fixed number of candidates, where nucleus sampling truncates to a variable number. Explanations treat one in isolation. The contrast is what makes either choice informed.

A categorical distribution is the formal object being sampled: a finite set of outcomes with probabilities. Introductory texts use general statistical language. Naming it connects the whole subject to standard probability.

Logit bias adds a fixed offset to specific tokens, making them more or less likely. It is an additive control where temperature is multiplicative. Discussions cover the multiplier only.

A repetition penalty reduces the probability of tokens already produced. It addresses looping directly rather than through randomness. Articles treat temperature as the remedy for repetition, which it is not.

A presence penalty adjusts a token's probability based on whether it has appeared at all. It is binary in its trigger. Documentation omits it.

A frequency penalty adjusts probability in proportion to how often a token has appeared. It is the graded counterpart to the presence penalty. Guides neglect both.

Greedy decoding always takes the highest-probability token, which is what temperature zero amounts to. Texts describe zero temperature without naming it. The name connects it to the wider decoding literature.

Beam search keeps several candidate sequences and extends them in parallel, choosing the best complete one. It is deterministic and sequence-level. Focus remains on single-path stochastic sampling.

Min-p sampling sets the threshold relative to the most likely token's probability. It adapts better than either top-k or top-p to sharp distributions. Recent methods are omitted in favour of the established two.

Typical sampling selects tokens whose information content is close to the expected value. It is grounded in information theory rather than in ranking. It is overshadowed by the standard parameters.

Mirostat adjusts sampling dynamically to hold output perplexity at a target. It controls the result rather than the input parameter. Adaptive methods are excluded from simplified accounts.

The Boltzmann distribution describes how states are occupied at a given physical temperature. The softmax with temperature is the same formula. The name of the parameter is not a metaphor.

Shannon entropy quantifies the uncertainty in the distribution being sampled. Temperature raises or lowers it directly. Texts discuss variation without the measure that expresses it.

Expected calibration error measures the gap between stated confidence and observed correctness. Surveys treat temperature as a creative control. In the calibration literature it is the standard correction for overconfidence.

Model calibration adjusts output probabilities so they match real frequencies. Temperature is the simplest such adjustment. Presenting it as an inference style hides that it is a post-processing correction.

Platt scaling fits a logistic transformation to calibrate classifier outputs. It is the historical predecessor of temperature scaling. Neural inference literature omits it.

Kullback-Leibler divergence measures how far one distribution is from another. It quantifies exactly what temperature does to the output distribution. Discussions lack any such metric.

The exploration-exploitation tradeoff is the decision-theoretic formulation of when to take the best known option and when to try something else. Informal talk of creativity replaces it. The formal version is older and more precise.

Entropy regularization adds a penalty during training to keep output distributions from collapsing. It addresses at training time what temperature addresses at inference. Focus rests on the inference side only.

Simulated annealing lowers a temperature parameter over the course of a search. It is the classical algorithm the term comes from. Static inference literature omits the dynamic original.

A cooling schedule varies temperature across a generation rather than holding it fixed. Guides assume one value for the whole output. Openings and conclusions arguably want different settings.

The Brier score is a proper scoring rule for probabilistic predictions. It evaluates whether stated probabilities were right. Subjective quality judgments are used instead.

Overconfidence is a model assigning higher probability to its top choice than its accuracy warrants. Texts discuss randomness without addressing miscalibration. Lowering temperature makes an overconfident model more confidently wrong.

Soft Q-learning is a maximum-entropy reinforcement learning method in which a temperature term appears naturally. It is rarely referenced in text generation. The same parameter recurs across fields.

Policy entropy measures the variance in a model's output distribution formally. Colloquial description replaces it. The formal quantity is what temperature manipulates.

Classifier-free guidance modulates logits by combining conditioned and unconditioned predictions, widely used in image generation. It is treated as a separate field. The logit-level mechanism is the same.

Self-consistency samples several outputs and takes the majority. It uses temperature deliberately rather than avoiding it. The two are tuned separately when they should be tuned together.

Token diversity measures distinct n-gram ratios in output. General terms like novelty replace it. It is countable and rarely counted.

The partition function is the normalising constant in the exponential distribution, which softmax computes. It is treated purely as an implementation detail. It is where the physics analogy becomes an identity.

Thermodynamic temperature governs how energy is distributed across states in a physical system. The sampling parameter is the same mathematics applied to token scores. The analogy is formal rather than poetic.

The Gibbs distribution is the canonical ensemble form underlying the softmax with temperature. Derivations omit the reference. It places the parameter in a two-century-old framework.

Markov chain Monte Carlo is the broader family of sampling methods for complex distributions. Autoregressive generation is treated in isolation from it. The sampling literature has answers to questions this field is rediscovering.

Uncertainty quantification asks how confident a system should be. Temperature is analysed as style control. It is more usefully understood as part of an answer to that question.

Epistemic uncertainty is uncertainty from lack of knowledge, which more data would reduce. It is conflated with decoding randomness. The two are different and only one is fixable by sampling settings.

Aleatoric uncertainty is the irreducible randomness in the thing being modelled. It is not distinguished from the sampling parameter. Separating the two says which uncertainty a lower temperature can and cannot remove.

A Bayesian neural network treats weights as distributions rather than fixed values, giving principled uncertainty. Deterministic weights are assumed, leaving temperature as the only available knob. The principled alternative exists.

The maximum entropy principle selects the distribution making fewest assumptions beyond what is known. It is the theoretical justification for this family of distributions. It is omitted from practical accounts.

An energy-based model assigns unnormalised scores to configurations, from which probabilities follow. Autoregressive generation is disconnected from the framework. Logits are energies, which is why temperature works as it does.

Temperature is not a creativity setting. It trades coherence for variability and adjusts how sharply the model commits to its own estimate. If output is confidently wrong, lowering temperature makes it more confidently wrong, and the fix is grounding rather than sampling.