I really enjoyed reading this story! It was a super cool mix of narrative and technical aspects. While reading, I noticed similarities between the world described in this story and the world model used as a part of Bengio et al.’s Scientist AI proposal. Now, the Terrarium itself isn’t a world model like Bengio describes, but I believe it could rather be a world that the world model generates theories about. Having Scientist AI generate theories about the Terrarium could lead to building intriguing theories about the emergent agent behavior and interactions. Specifically, Scientist AI could offer a way to get some legibility back without sacrificing the efficiency that is gained when agents think in neuralese.
I really enjoyed reading this story! It was a super cool mix of narrative and technical aspects. While reading, I noticed similarities between the world described in this story and the world model used as a part of Bengio et al.’s Scientist AI proposal. Now, the Terrarium itself isn’t a world model like Bengio describes, but I believe it could rather be a world that the world model generates theories about. Having Scientist AI generate theories about the Terrarium could lead to building intriguing theories about the emergent agent behavior and interactions. Specifically, Scientist AI could offer a way to get some legibility back without sacrificing the efficiency that is gained when agents think in neuralese.