Temperature

Temperature is one of the most important parameters you can adjust when working with large language models. It controls the randomness in the model's output by adjusting the probability distribution over possible next tokens. Understanding temperature helps you tune AI outputs to match your specific use case.

Technically, temperature scales the logits (raw scores) that a model assigns to each possible next token before sampling. A temperature of 1.0 uses the model's learned probabilities as-is. A low temperature (0.1-0.5) makes the distribution more peaked, concentrating probability on the most likely tokens and making the model more deterministic and conservative. A temperature of 0 always picks the single most likely token, making the model fully deterministic. A high temperature (1.5-2.0) flattens the distribution, giving lower-probability tokens more chance of being selected, making the model more creative and unpredictable.

The right temperature depends on your task. For factual tasks like question answering, code generation, or data extraction, a low temperature (0.0-0.3) is best - you want reliable, accurate, consistent outputs. For creative tasks like brainstorming, story writing, or generating diverse options, a higher temperature (0.7-1.2) is better - it produces more varied and surprising outputs. Most general-purpose applications use a temperature around 0.7-1.0 as a balanced default.

Temperature interacts with other sampling parameters like top-p (nucleus sampling), which restricts sampling to only the tokens whose cumulative probability reaches a threshold, and top-k, which restricts sampling to the k most probable tokens. Together, these parameters give fine-grained control over the character of model outputs.

When using AI writing copilots or other AI tools, you may not directly control temperature - the application developers have typically set appropriate defaults for the use case. But understanding temperature helps you interpret why AI outputs sometimes feel repetitive (low temperature) or go off in unexpected directions (high temperature), and what you might request when you want to adjust the style of AI assistance.