Wait, you didn’t? I agree you didn’t say “basement” but the section of the OP I am responding to is saying:
MIRI’s plan, to build a Friendly AI to take over the world
And then you said:
What about the “Task AGI” and “pivotal act” stuff? [Which is an example of MIRI’s plan to build a Friendly AI to take over the world]
The part in square brackets seems like the very clear Gricean implicature here? Am I wrong? If not, what did you mean to say in that sentence?
All the other stuff you say seems fine. I definitely agree MIRI talked about building AIs that would be very powerful and also considered whether power concentration would be a good thing, as it would reduce race dynamics. But again, I am just talking about the part of the OP says that it was MIRI’s plan to build such a system and take over the world, themselves, “in service of reducing x-risk”. None of the above seems like much evidence for that? If you agree that this was not MIRI’s plan, then sure, we are on the same page.
The part in square brackets seems like the very clear Gricean implicature here? Am I wrong? If not, what did you mean to say in that sentence?
See the two sentences right after.
That was at the very least, advising others to think seriously about using aligned AI to take over the world, on the basis that the world was otherwise doomed without a pivotal act. Then there was the matter of how much leverage MIRI thought they had as an organization, which is complicated by the confidentiality.
The Griecian implicature of this is that I at least don’t think it’s clear that MIRI wanted to build an AI to take over the world themselves. Rather, they were encouraging pivotal acts generally, and there’s ambiguity about how much they were individually trying to do so.
The literal implication of this is that it’s hard for people to know how much leverage MIRI has as an organization, which implies it’s hard for them to know that MIRI wanted to take over the world themselves.
Cool, yeah. I mean, I can’t rule this out confidently, but I do pretty strongly object to summarizing this state of affairs as:
Of course the most central old debate was over whether MIRI’s plan, to build a Friendly AI to take over the world in service of reducing x-risks, was a good one.
Like, at least in my ethics there is a huge enormous gulf between trying to take over the world, and saying that it would be a good idea for someone, ideally someone with as much legitimacy as possible, who is going to build extremely powerful AI systems anyways, to do this:
upload humans and run them at speeds more comparable to those of an AI
prevent the origin of all hostile superintelligences (in the nice case, only temporarily and via strategies that cause only acceptable amounts of collateral damage)
design or deploy nanotechnology such that there exists a direct route to the operators being able to do one of the other items on this list (human intelligence enhancement, prevent emergence of hostile SIs, etc.)
I go around and do the latter all the time, and think more people should do so! I agree I can’t rule out from the above that MIRI was maybe also planning to build such systems themselves, but I don’t currently find it that likely, and object to people referring to it as a fact of common knowledge.
Wait, you didn’t? I agree you didn’t say “basement” but the section of the OP I am responding to is saying:
And then you said:
The part in square brackets seems like the very clear Gricean implicature here? Am I wrong? If not, what did you mean to say in that sentence?
All the other stuff you say seems fine. I definitely agree MIRI talked about building AIs that would be very powerful and also considered whether power concentration would be a good thing, as it would reduce race dynamics. But again, I am just talking about the part of the OP says that it was MIRI’s plan to build such a system and take over the world, themselves, “in service of reducing x-risk”. None of the above seems like much evidence for that? If you agree that this was not MIRI’s plan, then sure, we are on the same page.
See the two sentences right after.
The Griecian implicature of this is that I at least don’t think it’s clear that MIRI wanted to build an AI to take over the world themselves. Rather, they were encouraging pivotal acts generally, and there’s ambiguity about how much they were individually trying to do so.
The literal implication of this is that it’s hard for people to know how much leverage MIRI has as an organization, which implies it’s hard for them to know that MIRI wanted to take over the world themselves.
Cool, yeah. I mean, I can’t rule this out confidently, but I do pretty strongly object to summarizing this state of affairs as:
Like, at least in my ethics there is a huge enormous gulf between trying to take over the world, and saying that it would be a good idea for someone, ideally someone with as much legitimacy as possible, who is going to build extremely powerful AI systems anyways, to do this:
I go around and do the latter all the time, and think more people should do so! I agree I can’t rule out from the above that MIRI was maybe also planning to build such systems themselves, but I don’t currently find it that likely, and object to people referring to it as a fact of common knowledge.