Their platform would be whatever version and framing of AI notkilleveryoneism the candidates personally endorse, plus maybe some other smaller things.
They should be open that they consider the potential human disempowerment or extinction to be the main problem of our time.
As for the concrete policy proposals, I am not sure. The focus could be on international treaties, or banning or heavy regulation of AI models who were trained with more than a trillion quadrillion (10^27) operations. (not sure I understand the intent behind your question).
I guess I’m expressing doubt about the viability of wise or cautious AI strategies, given our new e/acc world order, in which everyone who can, is sprinting uninhibitedly towards superintelligence.
My own best hope at this point is that someone will actually solve the “civilizational superalignment” problem of CEV, i.e. learning how to imbue autonomous AI with the full set of values (whatever they are) required to “govern” a transhuman civilization in a way that follows from the best in humanity, etc. - and that this solution will be taken into account by whoever actually wins the race to superintelligence.
This is a strategy that can be pursued within the technical communities that are actually designing and building frontier AI. But the public discussion of AI is currently hopeless, it’s deeply unrealistic about the most obvious consequences of creating new intelligences that are smarter, faster, and more knowledgeable than any human, namely, that they are going to replace human beings as masters of the Earth, and eventually that they will simply replace the entire human race.
My own best hope at this point is that someone will actually solve the “civilizational superalignment” problem of CEV, i.e. learning how to imbue autonomous AI with the full set of values (whatever they are) required to “govern” a transhuman civilization in a way that follows from the best in humanity, etc. - and that this solution will be taken into account by whoever actually wins the race to superintelligence.
Sounds like post-hoc justification for not even trying to stop something bad by picking a plan with zero percent chance of success, instead of further thought and actually trying to do the impossible.
Do you perceive the irony in telling me my hope has “zero percent chance” of happening, then turning around and telling me to do the impossible? I guess some impossibles are more impossible than others.
In fact I’ve spent 30 years attempting various forms of “the impossible” (i.e. things of unknown difficulty that aren’t getting done), it’s kind of why I’m chronically unemployed and rarely have more than $2000 in the bank. I know how to be audaciously ambitious in unpromising circumstances and I know how to be stubborn about it.
You like to emphasize the contingency of history as a source of hope. Fine. Let me tell you that the same applies to the world of intellectual discovery, which I know a lot better than I know the world of politics. Revolutionary advances in understanding can and do occur, and sometimes on the basis of very simple but potent insights.
Sorry if this was unclear, but there’s a difference between plans which work conditioning on an impossibility, and trying to do the impossible. For example, building a proof that works only if P=NP is true is silly in ways that trying to prove P=NP is not. The second is trying to do the impossible, the first is what I was dismissive of.
Yes, doing those things in ways that a capable alignment research can’t find obvious failure modes for. (Which may not be enough, given that they.aren’t superintelligences—but is still a bar which no proposed plan comes close to passing.)
I think that basically everyone at MIRI, Yampolskiy, and a dozen other people all have related and strong views on this. You’re posting on Lesswrong, and I don’t want to be rude, but I don’t know why I’d need to explain this instead of asking you to read the relevant work.
I asked because I’m talking with you and I wanted to know *your* reasoning as to why a technical solution to the alignment of superintelligence is impossible. It seems to be “lots of people see lots of challenges and they are too many to overcome, take it up with them”.
But it’s just a hard problem, and the foundations are not utterly mysterious. Humanity understands quite a lot about the physical and computational nature of our reality by now.
Maybe it would be more constructive to ask how you envisage achieving the political impossible of stopping the worldwide AI race, since that’s something that you do advocate.
I guess I’m expressing doubt about the viability of wise or cautious AI strategies, given our new e/acc world order, in which everyone who can, is sprinting uninhibitedly towards superintelligence.
e/acc does not poll well and there is widespread popular support for regulating AI (see AIPI polls). If the current government favors minimal regulations, that’s evidence that an AI safety candidate is more likely to succeed, not less.
(Although I’m not sure that follows because I think the non-notkilleveryonism variety of AI safety is more popular. Also Musk’s regulatory plan is polling well and I’m not sure if it differs from e.g. Vance’s plan.)
I don’t know what the regulatory plan is, I was just referring to this poll, which I didn’t read in full, I just read the title. Reading it now, it’s not so much a plan as a vision, and it’s not so much “Musk’s vision” as it is a viewpoint (that the poll claims is associated with Musk) in favor of regulating the risks of AI. Which is very different from JD Vance’s position; Vance’s position is closer to the one that does not poll well.
Their platform would be whatever version and framing of AI notkilleveryoneism the candidates personally endorse, plus maybe some other smaller things. They should be open that they consider the potential human disempowerment or extinction to be the main problem of our time.
As for the concrete policy proposals, I am not sure. The focus could be on international treaties, or banning or heavy regulation of AI models who were trained with more than a trillion quadrillion (10^27) operations. (not sure I understand the intent behind your question).
I guess I’m expressing doubt about the viability of wise or cautious AI strategies, given our new e/acc world order, in which everyone who can, is sprinting uninhibitedly towards superintelligence.
My own best hope at this point is that someone will actually solve the “civilizational superalignment” problem of CEV, i.e. learning how to imbue autonomous AI with the full set of values (whatever they are) required to “govern” a transhuman civilization in a way that follows from the best in humanity, etc. - and that this solution will be taken into account by whoever actually wins the race to superintelligence.
This is a strategy that can be pursued within the technical communities that are actually designing and building frontier AI. But the public discussion of AI is currently hopeless, it’s deeply unrealistic about the most obvious consequences of creating new intelligences that are smarter, faster, and more knowledgeable than any human, namely, that they are going to replace human beings as masters of the Earth, and eventually that they will simply replace the entire human race.
Sounds like post-hoc justification for not even trying to stop something bad by picking a plan with zero percent chance of success, instead of further thought and actually trying to do the impossible.
Do you perceive the irony in telling me my hope has “zero percent chance” of happening, then turning around and telling me to do the impossible? I guess some impossibles are more impossible than others.
In fact I’ve spent 30 years attempting various forms of “the impossible” (i.e. things of unknown difficulty that aren’t getting done), it’s kind of why I’m chronically unemployed and rarely have more than $2000 in the bank. I know how to be audaciously ambitious in unpromising circumstances and I know how to be stubborn about it.
You like to emphasize the contingency of history as a source of hope. Fine. Let me tell you that the same applies to the world of intellectual discovery, which I know a lot better than I know the world of politics. Revolutionary advances in understanding can and do occur, and sometimes on the basis of very simple but potent insights.
Sorry if this was unclear, but there’s a difference between plans which work conditioning on an impossibility, and trying to do the impossible. For example, building a proof that works only if P=NP is true is silly in ways that trying to prove P=NP is not. The second is trying to do the impossible, the first is what I was dismissive of.
So what’s the impossible thing—identifying an adequate set of values? instilling them in a superintelligence?
Yes, doing those things in ways that a capable alignment research can’t find obvious failure modes for. (Which may not be enough, given that they.aren’t superintelligences—but is still a bar which no proposed plan comes close to passing.)
Is there someone you regard as the authority on why it can’t be done? (Yudkowsky? Yampolskiy?)
Because what I see, are not problems that we know to be unsolvable, but rather problems that the human race is not seriously trying to solve.
I think that basically everyone at MIRI, Yampolskiy, and a dozen other people all have related and strong views on this. You’re posting on Lesswrong, and I don’t want to be rude, but I don’t know why I’d need to explain this instead of asking you to read the relevant work.
I asked because I’m talking with you and I wanted to know *your* reasoning as to why a technical solution to the alignment of superintelligence is impossible. It seems to be “lots of people see lots of challenges and they are too many to overcome, take it up with them”.
But it’s just a hard problem, and the foundations are not utterly mysterious. Humanity understands quite a lot about the physical and computational nature of our reality by now.
Maybe it would be more constructive to ask how you envisage achieving the political impossible of stopping the worldwide AI race, since that’s something that you do advocate.
e/acc does not poll well and there is widespread popular support for regulating AI (see AIPI polls). If the current government favors minimal regulations, that’s evidence that an AI safety candidate is more likely to succeed, not less.
(Although I’m not sure that follows because I think the non-notkilleveryonism variety of AI safety is more popular. Also Musk’s regulatory plan is polling well and I’m not sure if it differs from e.g. Vance’s plan.)
What plan are you referring to? Is this something AI safety specific?
I don’t know what the regulatory plan is, I was just referring to this poll, which I didn’t read in full, I just read the title. Reading it now, it’s not so much a plan as a vision, and it’s not so much “Musk’s vision” as it is a viewpoint (that the poll claims is associated with Musk) in favor of regulating the risks of AI. Which is very different from JD Vance’s position; Vance’s position is closer to the one that does not poll well.