One outstanding question I’ve had about introspection is: what part of the model is doing the introspection? In humans, this might be the prefrontal cortex.
In the extreme case (in which models can introspect arbitrarily well), this component is the only part of the model that needs to be aligned: through introspection, it can clean up the rest. In other words, good introspection could reduce the ‘surface area’ for alignment considerably.
One outstanding question I’ve had about introspection is: what part of the model is doing the introspection? In humans, this might be the prefrontal cortex.
In the extreme case (in which models can introspect arbitrarily well), this component is the only part of the model that needs to be aligned: through introspection, it can clean up the rest. In other words, good introspection could reduce the ‘surface area’ for alignment considerably.