Thanks for the podcast link! Mark Miller’s Madisonian system is essentially describing a type of game theory approach, and it’s something I did not know about!
There’s so much more to say about the practical implementation of some sort of game theory framework (Or any other solution-we-haven’t-explored-yet, such as one incorporating Yoshua’s “Scientist AI.”)
It’s quite a puzzle.
But it’s a puzzle worth solving.
For example, the source code verification coordination mechanism is something I had not heard of before, and it’s yet another example of how truly complex this challenge is.
But … are these puzzles unsolvable?
The Difficult Puzzle Vs. …
Maybe.
But here’s what troubles me about the alternative, and please take my next words with a grain of salt and feel free to push back. 🙂
So here’s my two cents:
“Shut it all down” will never happen.
Never. Never never never never. (And if they do, I’ll personally apologize to everyone on LW, plus my friends, family who have never heard of AI alignment, and even my small dog for doubting humanity’s ability to come together for the common good. I mean, we’ve avoided MAD so far, right?)
And I’ll explain why in a moment.
The Importance of If Anyone Builds It
But first, I think Eliezer’s book will do wonders for waking people up.
Right now we have many, many, many people who don’t seem to understand these systems are not simply how they present themselves. They don’t know about the “off-switch” problem, the idea of hidden goals, etc. They believe these AI systems are harmless because the systems tell them they are harmless, which is precisely what they were trained to do.
… The Impossible Solution (Shut It All Down!)
But here is why the “shut it down proposal,” with all its undeniablevalue in raising awareness and hopefully making everyone a little more cautious, can never resolve to a solution.
Because …
Someone will always see the advantage in creating ever more powerful agentic AI
If the United States and China and Russia, etc. sign an agreement, almost certainly they will keep building it in secret, just in case the other has it
If both of those nations honor the commitment, maybe North Korea (or Canada, why not) will build it
If North Korea does not build it, some terrorist group will build it
If the terrorist group does not build it, the lone madman will build it
If the lone madman doesn’t build it, some brilliant teenager tinkering around in their basement with some new quantum computer will find a way to build it
If someone doesn’t build it in 10 years, they will build it in 25 years
If someone doesn’t build it in 25 years, they will build it in 50 years
etc.
So, while we enjoy watching the Overton window move to a more realistic location, where people are finally understanding the danger of these systems, let’s keep plugging away at those actual solutions.
We can all contribute to the puzzle worth solving.
Yudkowsky-Miller Debate / “Madisonian” System
Thanks for the podcast link! Mark Miller’s Madisonian system is essentially describing a type of game theory approach, and it’s something I did not know about!
There’s so much more to say about the practical implementation of some sort of game theory framework (Or any other solution-we-haven’t-explored-yet, such as one incorporating Yoshua’s “Scientist AI.”)
It’s quite a puzzle.
But it’s a puzzle worth solving.
For example, the source code verification coordination mechanism is something I had not heard of before, and it’s yet another example of how truly complex this challenge is.
But … are these puzzles unsolvable?
The Difficult Puzzle Vs. …
Maybe.
But here’s what troubles me about the alternative, and please take my next words with a grain of salt and feel free to push back. 🙂
So here’s my two cents:
“Shut it all down” will never happen.
Never. Never never never never. (And if they do, I’ll personally apologize to everyone on LW, plus my friends, family who have never heard of AI alignment, and even my small dog for doubting humanity’s ability to come together for the common good. I mean, we’ve avoided MAD so far, right?)
And I’ll explain why in a moment.
The Importance of If Anyone Builds It
But first, I think Eliezer’s book will do wonders for waking people up.
Right now we have many, many, many people who don’t seem to understand these systems are not simply how they present themselves. They don’t know about the “off-switch” problem, the idea of hidden goals, etc. They believe these AI systems are harmless because the systems tell them they are harmless, which is precisely what they were trained to do.
… The Impossible Solution (Shut It All Down!)
But here is why the “shut it down proposal,” with all its undeniable value in raising awareness and hopefully making everyone a little more cautious, can never resolve to a solution.
Because …
Someone will always see the advantage in creating ever more powerful agentic AI
If the United States and China and Russia, etc. sign an agreement, almost certainly they will keep building it in secret, just in case the other has it
If both of those nations honor the commitment, maybe North Korea (or Canada, why not) will build it
If North Korea does not build it, some terrorist group will build it
If the terrorist group does not build it, the lone madman will build it
If the lone madman doesn’t build it, some brilliant teenager tinkering around in their basement with some new quantum computer will find a way to build it
If someone doesn’t build it in 10 years, they will build it in 25 years
If someone doesn’t build it in 25 years, they will build it in 50 years
etc.
So, while we enjoy watching the Overton window move to a more realistic location, where people are finally understanding the danger of these systems, let’s keep plugging away at those actual solutions.
We can all contribute to the puzzle worth solving.