I’m a mathematician passionate about AI Safety. Currently, I’m doing agent foundations research with Dovetail. Check out my research on this account!
Winter Cross
Consider a robot that is manipulating a deck of cards. This can be described with a world model with
possible states, corresponding to the possible arrangements of the deckShould this be 52! instead of 13! since there are 52 cards in a deck?
Thanks for the answer! That confirms what I was thinking.
That second case: and surprised me since I initially thought that the IMP implied that there was an additional isomorphism between the controller and the environment. I guess that isomorphism effectively still exists since it can be created through the use of coarse graining over the controller states like you mentioned.
This article is really approachable for someone like me who’s just getting acquainted with mathematical AI safety research, so I appreciate that! This definitely helped me better understand the IMP.
I have a question about this part in the “How is the controller ‘modelling’ the environment?” section:
If the joint system is represented by environment-controller pairs , then being injective means that no two pairs (within ) will have the same environment value or controller value . This means that with appropriate re-labelling, each joint state can be indexed:
I don’t see how this follows. Couldn’t the shape of the joint states be more complicated such as forming a cycle or having multiple joint states evolve to the same joint state? Both these possibilities would break the indexing. Is there something I’m missing that implies this linear structure?
I don’t really see how paying students can allow for asynchronous learning. It seems to me that would be more related to changes in how daily agendas are structured and how grades are determined rather than the incentive structure