“Overall I don’t see a clear argument about why should we worried about AGI.”
I see many, perhaps the most simple and most convincing is the mere question of “what happens when AI becomes better than humans at coding?”, aka it can re-write its own code, aka it becomes totally unpredictable.
“Quite the contrary, building AGI is still an active area of research with no clear solution.”
There was also no clear solution to many other transformative technologies a couple years before their invention, from atomic bombs to airplanes. With both these examples, many, if not most specialists also said they were impossible.
Either the world wakes up, or we’re screwed. No one can now for sure whether AGI will come soon or even if it’s possible. But the high probability of it should make anyone scared. And even higher is the probability that if AI progress continues we’ll end up with dangerous AI, not necessarily AGI. Aka something difficult or impossible to control.
“Overall I don’t see a clear argument about why should we worried about AGI.”
I see many, perhaps the most simple and most convincing is the mere question of “what happens when AI becomes better than humans at coding?”, aka it can re-write its own code, aka it becomes totally unpredictable.
“Quite the contrary, building AGI is still an active area of research with no clear solution.”
There was also no clear solution to many other transformative technologies a couple years before their invention, from atomic bombs to airplanes. With both these examples, many, if not most specialists also said they were impossible.
Either the world wakes up, or we’re screwed. No one can now for sure whether AGI will come soon or even if it’s possible. But the high probability of it should make anyone scared. And even higher is the probability that if AI progress continues we’ll end up with dangerous AI, not necessarily AGI. Aka something difficult or impossible to control.
Oh no, we know it’s possible. We ourselves prove by our existence that a generally capable reasoning agent can be constructed out of atoms.
Worse, we know that it’s easy. Evolution did it by massive trial and error with no intelligence to guide it.
We are so much cleverer than evolution. And so very far off how intelligent it is possible to be.