I thought the assumption in ELK is that the “world-model” was a Bayes net. Presumably it would get queried by message-passing. Arguably “message-passing in a Bayes net” is an optimization algorithm. Does it qualify as a mesa-optimizer, given that the message-passing algorithm was presumably written by humans?
Or do you think there could be a different optimizer somehow encoded in the message-passing procedure??
(Or maybe you don’t expect the world-model of a future AGI to actually be a Bayes net, and therefore you don’t care about that scenario and aren’t thinking about it?)
I thought the assumption in ELK is that the “world-model” was a Bayes net
No, I think the Bayes net frame was just a suggestion on how to concretely think about it. I don’t think the ELK doc assumes that it will literally be a Bayes net, and neither do I.
If there is a training scheme that can produce a generative world-model in the form of a Bayes net, with an explanation for how that training scheme routes around the path dependencies I’ve outlined in the low-end section, I’d like to hear about it.
I thought the assumption in ELK is that the “world-model” was a Bayes net. Presumably it would get queried by message-passing. Arguably “message-passing in a Bayes net” is an optimization algorithm. Does it qualify as a mesa-optimizer, given that the message-passing algorithm was presumably written by humans?
Or do you think there could be a different optimizer somehow encoded in the message-passing procedure??
(Or maybe you don’t expect the world-model of a future AGI to actually be a Bayes net, and therefore you don’t care about that scenario and aren’t thinking about it?)
No, I think the Bayes net frame was just a suggestion on how to concretely think about it. I don’t think the ELK doc assumes that it will literally be a Bayes net, and neither do I.
If there is a training scheme that can produce a generative world-model in the form of a Bayes net, with an explanation for how that training scheme routes around the path dependencies I’ve outlined in the low-end section, I’d like to hear about it.