What is Perplexity?
AI EngineeringLast updated:
A metric measuring how well a language model predicts a text sample, with lower values indicating better performance.
Perplexity quantifies model uncertainty by exponentiating the average negative log-likelihood per token. It is widely used to evaluate and compare language models during training, though it does not always correlate with downstream task quality.