Hmm okay I think I understand! Basically you’re making it more efficient to answer questions that you could ask about a black box, but would be infeasible to answer for a black box. Like for the takeover, you could theoretically answer the investigation question by testing it, but that would be inefficient to do in real time, but with a mechanistic explanation you could do so efficiently.
(Tangentially related: Katago (a game engine for Go) recently switched over to transformers. And like most game engines, it is strongly superhuman (in a narrow domain), but the neural net is fairly small since the neural net is combined with a tree search algorithm. Might be an easier candidate to explain before LLMs!)
Hmm okay I think I understand! Basically you’re making it more efficient to answer questions that you could ask about a black box, but would be infeasible to answer for a black box. Like for the takeover, you could theoretically answer the investigation question by testing it, but that would be inefficient to do in real time, but with a mechanistic explanation you could do so efficiently.
(Tangentially related: Katago (a game engine for Go) recently switched over to transformers. And like most game engines, it is strongly superhuman (in a narrow domain), but the neural net is fairly small since the neural net is combined with a tree search algorithm. Might be an easier candidate to explain before LLMs!)