I feel quite strongly that the powerful minds we create will have curiosity drives, at least by default, unless we make quite a big effort to create one without them for alignment reasons.
The reason is that yes — if you’re superintelligent you can plan your way into curiosity behaviors instrumentally, but how do you get there?
Curiosity drives are a very effective way to “augment” your reward signals, allowing you to improve your models and your abilities by free self-play.
I feel quite strongly that the powerful minds we create will have curiosity drives, at least by default, unless we make quite a big effort to create one without them for alignment reasons.
The reason is that yes — if you’re superintelligent you can plan your way into curiosity behaviors instrumentally, but how do you get there?
Curiosity drives are a very effective way to “augment” your reward signals, allowing you to improve your models and your abilities by free self-play.