What is Scaling Laws?
AI FundamentalsLast updated:
Empirical relationships predicting AI model performance based on model size, data volume, and compute budget.
Scaling laws show that LLM loss decreases predictably as parameters, training tokens, and compute increase. These relationships guide billion-dollar training decisions and explain why larger models tend to be more capable.