Daan

• DaanJan 20, 2024, 10:07 AM

  1 point

  0

  in reply to: Vladimir_Nesov’s comment on: Es­ti­mat­ing effi­ciency im­prove­ments in LLM pre-training

  Thanks for the comment! I agree that would be a good way to more systematically measure algorithmic efficiency improvements. You won’t be able to infer the effects of differences in data quality though—or are you suggesting you think those are very limited anyway?

Es­ti­mat­ing effi­ciency im­prove­ments in LLM pre-training