Language models follow scaling laws

Jan 2020

Researchers study predictable relationships among loss, model size, data, and compute.

Scaling becomes a practical lens for planning large-model training.

Sources