So, again, how can an AI know which child AI will best fulfill its utility function without being able to observe them in action, which for any sort of long-term goal is necessarily going to be slow?
If the goal is “make the most paperclips per day” the minimum cycle time to evaluate the performance of a child AI is one day. Maybe more if it wants an n > 1 to sample from each AI. And where is this AI getting the resources to try multiple AIs worth of systems? And also, too, if the goal is something complex like “acquire power and/or money” you now have multiple child AIs competing with each-other to better extract resources, which is a different environment than a single AI working to maximize its power/$.
So, again, how can an AI know which child AI will best fulfill its utility function without being able to observe them in action, which for any sort of long-term goal is necessarily going to be slow? If the goal is “make the most paperclips per day” the minimum cycle time to evaluate the performance of a child AI is one day. Maybe more if it wants an n > 1 to sample from each AI. And where is this AI getting the resources to try multiple AIs worth of systems? And also, too, if the goal is something complex like “acquire power and/or money” you now have multiple child AIs competing with each-other to better extract resources, which is a different environment than a single AI working to maximize its power/$.