That’s the alignment problem, the primary topic of this site. Opinions vary and arguments are plentiful. The general consensus is that there’s tons of reasons it might wipe us, the informed average something like a 50‰ estimate, and overconfidence is usually from ignorance of those arguments. I won’t try to restate them all, and I don’t know of a place they’re all collected, but they’re all over this site.
If it wiped us out, it will probably wipe them out too.
But what is the probability that AGI wipes us? Why would AGI be more aggresive than humans? Specially if we carefully nurture her to be our Queen!
That’s the alignment problem, the primary topic of this site. Opinions vary and arguments are plentiful. The general consensus is that there’s tons of reasons it might wipe us, the informed average something like a 50‰ estimate, and overconfidence is usually from ignorance of those arguments. I won’t try to restate them all, and I don’t know of a place they’re all collected, but they’re all over this site.