If I’m taking your point correctly, it seems you’re concerned about Goodharting in imitation learning. I agree, this seems a major issue, and I think people are aware of it and thinking about ways to address it.
I don’t think I’d put it that way (although I’m not saying it’s inaccurate). See my comments RE “safety via myopia” and “inner optimizers”.
If I’m taking your point correctly, it seems you’re concerned about Goodharting in imitation learning. I agree, this seems a major issue, and I think people are aware of it and thinking about ways to address it.
I don’t think I’d put it that way (although I’m not saying it’s inaccurate). See my comments RE “safety via myopia” and “inner optimizers”.