I see this is new and getting downvotes. I just want to briefly speculate on why that is, since you might want to edit your abstract if you address these issues in the text.
I’m always excited to see new people putting real effort into the conversation on AI safety, so I want to be supportive. But at this point I can’t actually upvote it, or even take the time to read the whole thing.
I read the abstract. It has a major flaw that might be drawing downvotes. It doesn’t reference previous work on this topic, which makes it likely that it will mostly rehash previous discussions on this topic that most LW readers will be familiar with. There’s probably a new perspective or insight somewhere in there, but reading the whole things in hopes of that doesn’t make sense for a long piece and busy people.
The discussion to date on LW has produced what I think to say is a rough consensus that your central thesis is wrong. This is known as the orthogonality thesis, and by the term moral realism.
To some extent you may mean “cooperation works”. That’s true for humans and similar things stuck at a similar level of effectiveness. If an AI can multiply itself and build robots to do its projects, cooperation would seem to have more downsides than upsides.
Well, it’s true that this post is completely ignorant of literally decades of discussion about this—from mailing list posts to blog posts to articles to books (including an entire chapter in R:AZ, and of course Superintelligence, etc.). The author seems to have made no effort at all, not only to review, but to even acknowledge the existence, of previous work on the subject.
But it’s also AI slop:
About the authors: Max Abecassis in collaboration with Claude Sonnet 4.
This, of course, also (at least partially) explains the first point.
(As the next paragraph tells us, the author seems to have something of a specialty in writing and posting this sort of “written in collaboration with [some LLM]” stuff.)
So what possible reason is there to engage with this stuff?
I had decided to not even read enough to figure out if it was AI written.
I don’t discount AI collaboration if the ideas and all final judgments are human; Jan Kulveit wrote a couple of great posts by having conversations with Claude and letting it do most of the final writing.
But yeah, without even a claim that the human was in charge of the ideas and claims, I’m going to not bother reading it even if it does acknowledge previous work.
I see this is new and getting downvotes. I just want to briefly speculate on why that is, since you might want to edit your abstract if you address these issues in the text.
I’m always excited to see new people putting real effort into the conversation on AI safety, so I want to be supportive. But at this point I can’t actually upvote it, or even take the time to read the whole thing.
I read the abstract. It has a major flaw that might be drawing downvotes. It doesn’t reference previous work on this topic, which makes it likely that it will mostly rehash previous discussions on this topic that most LW readers will be familiar with. There’s probably a new perspective or insight somewhere in there, but reading the whole things in hopes of that doesn’t make sense for a long piece and busy people.
The discussion to date on LW has produced what I think to say is a rough consensus that your central thesis is wrong. This is known as the orthogonality thesis, and by the term moral realism.
To some extent you may mean “cooperation works”. That’s true for humans and similar things stuck at a similar level of effectiveness. If an AI can multiply itself and build robots to do its projects, cooperation would seem to have more downsides than upsides.
Well, it’s true that this post is completely ignorant of literally decades of discussion about this—from mailing list posts to blog posts to articles to books (including an entire chapter in R:AZ, and of course Superintelligence, etc.). The author seems to have made no effort at all, not only to review, but to even acknowledge the existence, of previous work on the subject.
But it’s also AI slop:
This, of course, also (at least partially) explains the first point.
(As the next paragraph tells us, the author seems to have something of a specialty in writing and posting this sort of “written in collaboration with [some LLM]” stuff.)
So what possible reason is there to engage with this stuff?
I had decided to not even read enough to figure out if it was AI written.
I don’t discount AI collaboration if the ideas and all final judgments are human; Jan Kulveit wrote a couple of great posts by having conversations with Claude and letting it do most of the final writing.
But yeah, without even a claim that the human was in charge of the ideas and claims, I’m going to not bother reading it even if it does acknowledge previous work.