Let’s call the short term predictor (in the long term predictor circuit) P, so if P tries to predict [what P predicts in 0.3s], then the correct prediction would be to immediately predict the output at whatever point in the future the process terminates (the next ground truth injection?). In particular, P would always predict the same until the ground truth comes in. But if I understand correctly, this is not what’s going on.
So second try: is P really still only trying to predict 0.3s into the future, making it less of a “long term predictor” and more of an “ongoing process predictor”? And then you get, e.g., the behavior of predicting a little less enzyme production with every step?
Or third try, is P just trying to minimize something like the sum of squared differences between adjacent predictions, and is thus trying to minimize the number of ground-truth injections, and we get the above an emergent effect?
I don’t completely get this.
Let’s call the short term predictor (in the long term predictor circuit) P, so if P tries to predict [what P predicts in 0.3s], then the correct prediction would be to immediately predict the output at whatever point in the future the process terminates (the next ground truth injection?). In particular, P would always predict the same until the ground truth comes in. But if I understand correctly, this is not what’s going on.
So second try: is P really still only trying to predict 0.3s into the future, making it less of a “long term predictor” and more of an “ongoing process predictor”? And then you get, e.g., the behavior of predicting a little less enzyme production with every step?
Or third try, is P just trying to minimize something like the sum of squared differences between adjacent predictions, and is thus trying to minimize the number of ground-truth injections, and we get the above an emergent effect?
I’m advocating for the first one—P is trying to predict the next ground-truth injection. Does something trouble you about that?
No; it was just that something about how the post explained it made me think that it wasn’t #1.