This is generally agreed upon to be the “right” way to study with AI and Go players often pay lip service to it. In practice people’s boundaries for what counts as on-policy distillation are not well-defined and that dillutes the impact. The mechanism whereby boundaries get weakened goes as follows:
A player finishes a game and usually has a narrative of what happened, which mistakes were pivotal, and what they need to improve. The AI’s evaluation will throw prediction error in the person’s model. Since the AI is understood to be unquestionably correct, the person often ‘has’ to update to the AI’s view. However, people can hack their their prediction-error sensors by retroactively updating what they think they believed in the past as well. This makes it extremely difficult to get useful feedback from the AI because all of it is ‘obvious’ or ‘natural’ after it is pointed out to you.
The above pattern seems extremely common and I personally struggle to overcome it. The best method I have found is to make a concrete, written narrative of my games. This includes recording the exact moves I think are mistakes, why they are mistakes, and how much I expect the AI to dislike them. This creates a more well-defined boundary of what your opinions actually were that helps you maintain some epistemic integrity. I would do this regularly if I were still playing Go (semi)-professionally.
This makes it extremely difficult to get useful feedback from the AI because all of it is ‘obvious’ or ‘natural’ after it is pointed out to you.
If that happens, isn’t a sign that the feedback has been useful? Even if the person doesn’t remember their previous thinking, if they find the feedback obvious, presumably that’s because they’ve updated their model to incorporate it.
There’s a similar thing in therapy where people will fix an emotional issue they used to have and then completely forget that they ever had the problem in the first place. This might make it seem like therapy was less effective (they can’t remember any problems it solved!), when in fact it’s a consequence of it having been very effective.
Feeling that something is obvious in hindsight is a bad predictor that your internal model has updated (hindsight bias). It’s true that your reflective model of the game might update—was this a good move, was this a bad one—but that’s easy because you know the AI is super-human and whatever it tells you must be true. This process is unlikely to have any effect on your future move-generation policy.
It’s like the difference between verification and construction of mathematical proofs. It’s usually trivial to verify a proof in comparison to actually constructing it. Verifying a proof isn’t any indicator you learnt the necessary skills to construct one similar in the future. Arguably Go AI is even worse here because it gives an answer without a proof, so you can trivially recognise a move without learning how to generate it.
This is generally agreed upon to be the “right” way to study with AI and Go players often pay lip service to it. In practice people’s boundaries for what counts as on-policy distillation are not well-defined and that dillutes the impact. The mechanism whereby boundaries get weakened goes as follows:
A player finishes a game and usually has a narrative of what happened, which mistakes were pivotal, and what they need to improve. The AI’s evaluation will throw prediction error in the person’s model. Since the AI is understood to be unquestionably correct, the person often ‘has’ to update to the AI’s view. However, people can hack their their prediction-error sensors by retroactively updating what they think they believed in the past as well. This makes it extremely difficult to get useful feedback from the AI because all of it is ‘obvious’ or ‘natural’ after it is pointed out to you.
The above pattern seems extremely common and I personally struggle to overcome it. The best method I have found is to make a concrete, written narrative of my games. This includes recording the exact moves I think are mistakes, why they are mistakes, and how much I expect the AI to dislike them. This creates a more well-defined boundary of what your opinions actually were that helps you maintain some epistemic integrity. I would do this regularly if I were still playing Go (semi)-professionally.
If that happens, isn’t a sign that the feedback has been useful? Even if the person doesn’t remember their previous thinking, if they find the feedback obvious, presumably that’s because they’ve updated their model to incorporate it.
There’s a similar thing in therapy where people will fix an emotional issue they used to have and then completely forget that they ever had the problem in the first place. This might make it seem like therapy was less effective (they can’t remember any problems it solved!), when in fact it’s a consequence of it having been very effective.
Feeling that something is obvious in hindsight is a bad predictor that your internal model has updated (hindsight bias). It’s true that your reflective model of the game might update—was this a good move, was this a bad one—but that’s easy because you know the AI is super-human and whatever it tells you must be true. This process is unlikely to have any effect on your future move-generation policy.
It’s like the difference between verification and construction of mathematical proofs. It’s usually trivial to verify a proof in comparison to actually constructing it. Verifying a proof isn’t any indicator you learnt the necessary skills to construct one similar in the future. Arguably Go AI is even worse here because it gives an answer without a proof, so you can trivially recognise a move without learning how to generate it.