Compute Trends Across Three Eras of Machine Learning
Compute Trends Across Three Eras of Machine Learning
Compute, data, and algorithmic advances are the three fundamental factors that drive progress in modern Machine Learning (ML). In this paper we study trends in the most readily quantified factor - compute. We make three novel contributions: (1) we curate a dataset with the training compute of 123 milestone ML …