DanielFilan comments on What is ambitious value learning?

[DanielFilan]

Also, if you define the state to be the entire history, you lose ergodicity assumptions that are needed to prove that algorithms can learn well.