Fascinating stuff. I’ve often thought about ways to do bitter-lesson-pilled interpretability, and this means you need to have a metric to optimize that’s difficult to Goodhart on. A metric related to quantifiable understanding, like “can you use this to successfully predict stuff”, or in this case “can you use this to accurately translate a language”, seems like a great category of this.
To be clear I’m talking about future applications of this stuff where the quantifiable goal is more like “how well can the human predict the results of interactions / intervention experiments with the AI model”, since that seems like a pretty robust thing to optimize hard on (either the human gets more understanding or the AI gets generally more transparent/predictable, both of which seem like good things).
Fascinating stuff. I’ve often thought about ways to do bitter-lesson-pilled interpretability, and this means you need to have a metric to optimize that’s difficult to Goodhart on. A metric related to quantifiable understanding, like “can you use this to successfully predict stuff”, or in this case “can you use this to accurately translate a language”, seems like a great category of this.
To be clear I’m talking about future applications of this stuff where the quantifiable goal is more like “how well can the human predict the results of interactions / intervention experiments with the AI model”, since that seems like a pretty robust thing to optimize hard on (either the human gets more understanding or the AI gets generally more transparent/predictable, both of which seem like good things).