The style strikes me as very AI-written. Maybe it isn’t—but the very repetitive structure looks exactly like the type of text I tend to get out of ChatGPT much of the time. Which makes it very hard to read.
There are many highly superficial claims here without much reasoning to back them up. Many claims of what AGI “would” do without elaboration. “AGI approaches challenges as problems to be solved, not battles to be won.”—first, why? Second, how does this help us when the best way to solve the problem involves getting rid of humans?
Lastly, I don’t get the feeling this post engages with the most common AI safety arguments at all. Neither does it with evidence from recent AI developments. How do you expect “international agreements” with any teeth in the current arms race? When we don’t even get national or state level agreements. While Bing/Sydney was not an AGI, it clearly showed that much of what this post dismisses as anthropocentric projections is realistic, and, currently, maybe even the default of what we can expect of AGI as long as it’s LLM-based. And even if you dismiss LLMs and think of more “Bostromian” AGIs, that still leaves you with instrumental convergence, which blows too many holes into this piece to leave anything of much substance.
Downvoted for 3 reasons:
The style strikes me as very AI-written. Maybe it isn’t—but the very repetitive structure looks exactly like the type of text I tend to get out of ChatGPT much of the time. Which makes it very hard to read.
There are many highly superficial claims here without much reasoning to back them up. Many claims of what AGI “would” do without elaboration. “AGI approaches challenges as problems to be solved, not battles to be won.”—first, why? Second, how does this help us when the best way to solve the problem involves getting rid of humans?
Lastly, I don’t get the feeling this post engages with the most common AI safety arguments at all. Neither does it with evidence from recent AI developments. How do you expect “international agreements” with any teeth in the current arms race? When we don’t even get national or state level agreements. While Bing/Sydney was not an AGI, it clearly showed that much of what this post dismisses as anthropocentric projections is realistic, and, currently, maybe even the default of what we can expect of AGI as long as it’s LLM-based. And even if you dismiss LLMs and think of more “Bostromian” AGIs, that still leaves you with instrumental convergence, which blows too many holes into this piece to leave anything of much substance.