Well, no offense, but I’m not sure you are aware of the need for Friendliness in Obedient AI, or rather, just how much F you need in a genie.
If you were to actually figure out how to build a genie you would have figured it out by trying to build a CEV-class AI, intending to tackle all those challenges, tackling all those challenges, having pretty good solutions to all of those challenges, not trusting those solutions quite enough, and temporarily retreating to a mere genie which had ALL of the safety measures one would intuitively imagine necessary for a CEV-class independently-acting unchecked AI, to the best grade you could currently implement them. Anyone who thought they could skip the hard parts of CEV-class FAI by just building a genie instead, would die like a squirrel under a lawnmower. For reasons they didn’t even understand because they hadn’t become engaged with that part of the problem.
I’m not certain that this must happen in reality. The problem might have much kinder qualities than I anticipate in the sense of mistakes naturally showing up early enough and blatantly enough for corner-cutters to spot them. But it’s how things are looking as a default after becoming engaged with the problems of CEV-class AI. The same problems show up in proposed ‘genies’ too, it’s just that the genie-proposers don’t realize it.
I’m… not sure what you mean by this. And I wouldn’t be against putting a whole CEV-ish human morality in an AI, either. My point is that there seems to be a big space between your Outcome Pump fail example and highly paternalistic AIs of the sort that caused Failed Utopia 4-2.
It reminds me a little bit about how modern computers are only occasionally used for computation.
Anything smarter-than-human should be regarded as containing unimaginably huge forces held in check only by the balanced internal structure of those forces, since there is nothing which could resist them if unleashed. The degree of ‘obedience’ makes very little difference to this fact, which must be dealt with before you can go on to anything else.
As I understand it, a AI is expected to make huge, inventive efforts to fulfill its orders as it understands them.
You know how sometimes people cause havoc while meaning well? Imagine something immensely more powerful and probably less clueful making the same mistake.
Well, no offense, but I’m not sure you are aware of the need for Friendliness in Obedient AI, or rather, just how much F you need in a genie.
If you were to actually figure out how to build a genie you would have figured it out by trying to build a CEV-class AI, intending to tackle all those challenges, tackling all those challenges, having pretty good solutions to all of those challenges, not trusting those solutions quite enough, and temporarily retreating to a mere genie which had ALL of the safety measures one would intuitively imagine necessary for a CEV-class independently-acting unchecked AI, to the best grade you could currently implement them. Anyone who thought they could skip the hard parts of CEV-class FAI by just building a genie instead, would die like a squirrel under a lawnmower. For reasons they didn’t even understand because they hadn’t become engaged with that part of the problem.
I’m not certain that this must happen in reality. The problem might have much kinder qualities than I anticipate in the sense of mistakes naturally showing up early enough and blatantly enough for corner-cutters to spot them. But it’s how things are looking as a default after becoming engaged with the problems of CEV-class AI. The same problems show up in proposed ‘genies’ too, it’s just that the genie-proposers don’t realize it.
I’m… not sure what you mean by this. And I wouldn’t be against putting a whole CEV-ish human morality in an AI, either. My point is that there seems to be a big space between your Outcome Pump fail example and highly paternalistic AIs of the sort that caused Failed Utopia 4-2.
It reminds me a little bit about how modern computers are only occasionally used for computation.
Anything smarter-than-human should be regarded as containing unimaginably huge forces held in check only by the balanced internal structure of those forces, since there is nothing which could resist them if unleashed. The degree of ‘obedience’ makes very little difference to this fact, which must be dealt with before you can go on to anything else.
As I understand it, a AI is expected to make huge, inventive efforts to fulfill its orders as it understands them.
You know how sometimes people cause havoc while meaning well? Imagine something immensely more powerful and probably less clueful making the same mistake.