I think there are some problems we get into thinking about superintelligences. I think probably trying to have humans control as superintelligence is a way harder problem than having humans control a super-human system made of non-superhuman components. We just need a way to bridge ourselves to the long reflection. I think it’s plausible that a large corporation/ think-tank staffed by human genius level AIs running at 1000x human speed could be powerful enough. There’s no need to jump to “an intelligence so far beyond us it is like we are rats trying to communicate with human geniuses”.
I think there are some problems we get into thinking about superintelligences. I think probably trying to have humans control as superintelligence is a way harder problem than having humans control a super-human system made of non-superhuman components. We just need a way to bridge ourselves to the long reflection. I think it’s plausible that a large corporation/ think-tank staffed by human genius level AIs running at 1000x human speed could be powerful enough. There’s no need to jump to “an intelligence so far beyond us it is like we are rats trying to communicate with human geniuses”.