CS
Chinchilla scaling law
Topic
What experts have said about Chinchilla scaling law
2 statements · 1 negative · 1 mixed
Under Chinchilla scaling, unlimited parameters reduce required data by only roughly tenfold at fixed loss.
“Even if you increase the number of parameters by infinity, that would only decrease by a factor of 10 the amount of data that you need in order to keep the same loss.”
Open the episode · The data black hole at the center of AIListen at 7:14
The discussed frontier model may be trained on roughly 100 times Chinchilla-optimal tokens.
“the amount it's overtrained, which is like a factor of 100 overtrained”
Open the episode · Reiner Pope – The math behind how LLMs are trained and servedListen at 1:32:19
Statements are attributed to the speaker as said on the episode and reflect their view at the time, not PodLume's. They are not advice.