My view is that we probably aren’t anywhere remotely close to knowing how to align ASI, and also that the current people don’t seem to be capable of making progress, so we should be working on augmenting human intelligence instead while halting AGI research. So the two things I’d want to suggest they work on are:
Figuring out how to to prevent anyone from building ASI anytime soon
Augmenting human intelligence
(I believe this is essentially the position of IABIED)
Both of these endeavors could absorb a lot of brain power, maybe especially the augmentation. I’d expect mathematicians to have an appreciation for how much humans vary in cognitive ability, and have benefited from high cognitive ability themselves; maybe this helps with pitching them on how valuable it would be to augment human intelligence.
On Copenhagen Interpretation of Ethics grounds I precommitted to not responding to queries like this, or else it would look like anyone who tried to solve any problem had an obligation to consider every related problem.
My intuition is that “the smart shot“ (whatever that is) plus a neural lace in an exponentially expanding number of people looks very similar to an RSI, except that a datacenter is unneeded, the nodes of the ASI have human rights, and it will never actually happen because it requires human experimentation, which would be banned.
My view is that we probably aren’t anywhere remotely close to knowing how to align ASI, and also that the current people don’t seem to be capable of making progress, so we should be working on augmenting human intelligence instead while halting AGI research. So the two things I’d want to suggest they work on are:
Figuring out how to to prevent anyone from building ASI anytime soon
Augmenting human intelligence
(I believe this is essentially the position of IABIED)
Both of these endeavors could absorb a lot of brain power, maybe especially the augmentation. I’d expect mathematicians to have an appreciation for how much humans vary in cognitive ability, and have benefited from high cognitive ability themselves; maybe this helps with pitching them on how valuable it would be to augment human intelligence.
Obvious followup to this: would you be interesting/willing to work on human cognitive enhancement? If not, why?
On Copenhagen Interpretation of Ethics grounds I precommitted to not responding to queries like this, or else it would look like anyone who tried to solve any problem had an obligation to consider every related problem.
Fair enough. I guess I want to get a sense of how hopeless an endeavor what I’ve proposed above is.
My intuition is that “the smart shot“ (whatever that is) plus a neural lace in an exponentially expanding number of people looks very similar to an RSI, except that a datacenter is unneeded, the nodes of the ASI have human rights, and it will never actually happen because it requires human experimentation, which would be banned.