I agree that it may be a useful source of insight, since it may tell you some learning techniques like this one, but I find it unlikely that this will end up involving giving the AI a “human caregiver”.
Maybe it is not the most likely scenario but a lot of mediocre AIs trained “on the job” in a close loop with humans that are not just overseers but provide a lot of real-world context doesn’t seem so unlikely in a Robin Hanson style slow takeoff.
I agree that it may be a useful source of insight, since it may tell you some learning techniques like this one, but I find it unlikely that this will end up involving giving the AI a “human caregiver”.
Maybe it is not the most likely scenario but a lot of mediocre AIs trained “on the job” in a close loop with humans that are not just overseers but provide a lot of real-world context doesn’t seem so unlikely in a Robin Hanson style slow takeoff.