Quantitative and qualitative evaluation of LMs

Perplexity vs downstream performance

You use perplexity to measure how well a language model predicts text, but it doesn’t always line up with how well the model does on real tasks.

Perplexity is just a transformed average of the model’s predicted probabilities for the correct next tokens:

  • For a sequence of tokens x_1,…,x_Tx\_1, \dots, x\_T, the model assigns probabilities p(x_t∣x_<t)p(x\_t \mid x\_{<t}).
  • The average negative log-likelihood is: NLL=−1T∑_t=1Tlog⁡p(x_t∣x_<t)\text{NLL} = -\frac{1}{T}\sum\_{t=1}^T \log p(x\_t \mid x\_{<t})
  • Perplexity is: Perplexity=eNLL\text{Perplexity} = e^{\text{NLL}}

Lower perplexity = the model is, on average, more confident in the actual tokens in your dataset.

Why lower perplexity ≠ “better at everything”

Perplexity is about predicting what humans wrote, not about solving your specific task. That distinction matters:

AspectWhat perplexity capturesWhat your task might needWhy they diverge
ObjectiveMatch training data token-by-tokenSolve a goal (answer correctly, follow instruction)Data might contain many styles, not your goal
GranularityLocal next-token predictionsGlobal behavior over long outputsA model can be locally fluent but globally useless
DomainTraining / eval corpusYour deployment domainYour domain may be underrepresented
RewardLikelihood of textTask success / utility“Typical internet text” ≠ “best solution”

Example:

  • Model A: Perplexity 15 on a news corpus.
  • Model B: Perplexity 13 on the same corpus.

On a math word problem dataset:

  • Model A answers 70% correctly.
  • Model B answers 62% correctly.

What happened?

  • Model B is better at mimicking the news text style (lower perplexity), but
  • Math word problems require structured reasoning and following instructions, which might not correlate tightly with news-style language modeling.
  • So B “predicts the next word” better, but reasons worse.

Another example: safety. A model can get lower perplexity by closely imitating training text, including toxic content. But for a safety task, you might want it to not imitate that even if the “typical” next token is harmful.

If you only chase lower perplexity during model development, you can easily deploy a model that’s more fluent but not more accurate, safe, or controllable.

Concrete check you can run

  1. Take two models, model_old and model_new.
  2. On the same held-out corpus, measure their perplexities: suppose 18 vs 15 (new is better).
  3. On your actual task (e.g., question-answer pairs or coding tasks), measure:
    • Accuracy (exact match, pass@k, etc.).
    • User-rated usefulness (e.g., 1–5 scale on a random sample).
  4. Compare.

You might see:

  • model_new has lower perplexity and higher task accuracy → nice, correlation holds.
  • Or lower perplexity but same or worse task accuracy → you’ve just seen the limitation of perplexity directly.
1 / 4