I might retract the exact phrasing of my reply comment.
I think I was originally using overwhelmingly basically the way you’re using “galaxy brained”, and I feel like I have quibbles about the exact semantics of that phrase that feel about as substantial as your concern about overwhelming. (i.e. there is also a substantive difference between a very powerful brain hosted in a datacenter on Earth, and an AI that with a galaxy of resources)
What I mean by “overwhelmingly superintelligent” is “so fucking smart that humanity would have to have qualitatively changed in a similar orders-of-magnitude degree”, which probably in practice means humans also have to have augmented their own intelligence, or have escalated their AI control schemes pretty far, carefully wielding significantly-[but-not-overwhelming/galaxy-brained]-AI that oversees all of Earth’s security and is either aligned or the humans are really at threading the needle on control for quite powerful systems.
I might retract the exact phrasing of my reply comment.
I think I was originally using overwhelmingly basically the way you’re using “galaxy brained”, and I feel like I have quibbles about the exact semantics of that phrase that feel about as substantial as your concern about overwhelming. (i.e. there is also a substantive difference between a very powerful brain hosted in a datacenter on Earth, and an AI that with a galaxy of resources)
What I mean by “overwhelmingly superintelligent” is “so fucking smart that humanity would have to have qualitatively changed in a similar orders-of-magnitude degree”, which probably in practice means humans also have to have augmented their own intelligence, or have escalated their AI control schemes pretty far, carefully wielding significantly-[but-not-overwhelming/galaxy-brained]-AI that oversees all of Earth’s security and is either aligned or the humans are really at threading the needle on control for quite powerful systems.