LLMs probably enter RL a few months after their pretraining data cutoff. Why not just train the model to forecast events that occurred after the pretraining cutoff date but before the present day, so that you immediately have access to the ground truth? You should be able to find a lot of training examples from real world events and even produce them automatically.
If you want to spread the training out across a longer time (going backwards to ~2022), just use an old pretraining corpus published by some open-source lab somewhere, so you have a few years worth of data that is guaranteed to be unknown to the model. At each training step (as well as in deployment), give the model context up to the “current” date in the training experiment, and ask it to predict a certain “future” date which you still have ground truth on.
This is something Thinking Machines Lab basically did; though they only trained for forecasting binary outcomes and had the model search for context. I’m not sure how this (or training LLMs to forecast in general) speaks to myopic fitness-seeking though
LLMs probably enter RL a few months after their pretraining data cutoff. Why not just train the model to forecast events that occurred after the pretraining cutoff date but before the present day, so that you immediately have access to the ground truth? You should be able to find a lot of training examples from real world events and even produce them automatically.
If you want to spread the training out across a longer time (going backwards to ~2022), just use an old pretraining corpus published by some open-source lab somewhere, so you have a few years worth of data that is guaranteed to be unknown to the model. At each training step (as well as in deployment), give the model context up to the “current” date in the training experiment, and ask it to predict a certain “future” date which you still have ground truth on.
This is something Thinking Machines Lab basically did; though they only trained for forecasting binary outcomes and had the model search for context. I’m not sure how this (or training LLMs to forecast in general) speaks to myopic fitness-seeking though