It gives remarkably accurate answers for well-formulated questions. Where there is a theoretical limit to the accuracy of an answer to a particular type of question, its answer usually comes close to that limit, plus or minus some tiny rounding error.
Note: This is rather long. Sorry about that, if it’s too long, but I was making notes on a comparable question recently, and you did say to be specific.
Well, the last time I was thinking of features in an AI, was one that could ask questions that could potentially cleave answer space for any poorly formulated request(Because as a well known problem, there is far more processing time and programmer effort spent on poorly formulated requests then well formulated ones) and show a video demonstration of some likely behaviors prior to your answering assuming what it considered the most plausible potential answers.
1: Incredibly simplistic example: you request the AI: “Make me a Line graph of these numbers.”
The AI responds: “Do you want a Line Graph, with number set one as the X axis, which would look like this, or do you want a Line Graph with number set two as the X axis, which would look like this, or are you thinking of something else?”
Without that feature, a program might take that and just go “ERROR: Insufficient parameters.” Or, worse, they’ll give you the WRONG line graph, and they won’t tell you. Both are annoying.
2: More complicated request example: you request the AI: “Bring a drink inside the fridge to me.”
The AI may respond: “Do you want me to bring an entire two liter bottle of soda, which would look like this, or do you want me to bring a can of soda, which would look like this, or do you want me to drag the entire fridge to your position, which would look like this, or are you thinking of something else?”
This is where the video demonstration comes in handy. Relative to you, the AI doesn’t have common sense. It doesn’t necessarily know that you probably don’t want it to drag the entire fridge to your position to the same level of confidence you do. It just puts together some plausible answers and lets you pick one.
(Note: There is nothing particularly special about video, other than that it shows that the AI shouldn’t start actually doing things until you’ve narrowed the request down, and that some requests have mistakes too complicated to be picked up by text only answers.)
Of course, in the previous case, the AI was mostly on the money. It’s also possible that the AI doesn’t have the slightest idea what you’re are talking about in a dangerous way.
3: You request the AI: “Subdue the crazed man inside the building of innocents.”
The AI may respond: “Do you want me to level the building with TNT, which would look like this, or do you want me to gas the building with Chemicals, which would look like this, or do you want me to concoct an Contagious All purpose, Anti-Human Virus and infect him, which would look like this, or are you thinking of something else?”
Whoops. You forgot to tell the AI to subdue the crazed man WITHOUT hurting the innocents. Thankfully it clearly demonstrated what plans it had put together before it implemented any of them, because that would’ve been bad. You may have also not noted that you wanted the man subdued non-lethally.
This also helps (not prevents entirely, of course) guard against the AI just doing something completely inexplicable relative to you.
4: “What’s 2+2?”
“Is the answer 4, like this, or do you want me to build a solid sphere of Computronium advancing at lightspeed and continue calculating, like this, or are you thinking of something else?”
If you wanted the AI to be better, you could network it to a cloud database so it could learn “In 1000 cases When people ask for a drink inside the fridge, they never mean drag the fridge to them, I should stop giving that as a likely answer, and add ’Would you like me to pour your drink in a glass? which seems to come up quite a bit as an answer selected from Something else.”
Since danger goes up as capability does, I’d probably want to train a dumber AI on a great deal of 1 and 2 before ever getting anywhere near 3, let alone something as existential as 4.
Of course, if the AI is substantially good enough at ANY request, it can respond to a request of:
5: “Increase your computing power.”
With “Do you want me to increase my computing power by building more cores like this, or do you want me to increase my computing power by switching out my existing cores with this design that I came up, like this, or are you thinking of something else?”
In which case you have a Self-Improving AI that, after a great deal of training, gets what humanity wants when they tell it things, so it can be enlisted to save the world and make everyone live happily forever.
Except, that what this DIDN’T do, is it didn’t actually follow the procedure that I laid out above, and give an alternate option under the assumption that this isn’t what you asked for(and it might be very dangerous), and possibly ask if you just wanted something else entirely.
I want to note that those steps should be here, but have mainly been cut for space constraints, except as rough mentions.
Obvious problem 1: the video output or descriptions can contain basilisks, including ones that cause problem 2.
Obvious problem 2: Someone could ask and then verify it to make full UFAI without realizing it.
Obvious problem 3: UFAI could arise inside the simulation used to produce the hypotheticals, and either hack it’s way out directly or cause problem 1 followed by problem 2.
And most oblivious problem of all: Withe being able to repeatedly do the highly dangerous and active step of modifying it’s own source code, it’ll never get smart enough to be useful on 95% of queries.
Fair point. in that case, given an unknown partially complete AI, if the first action you take is “Let me just start reading the contents of these files without running it to see what it even does.” then someone could say “A UFAI put a basilisk in the source code and used it to kill all of humanity, you lose.”
So, in the general case, something that will take a natural language request, turn it into a family of optimizable models, identify the most promising ones, ask the user to choose, and then return an optimized answer?
Notice that it doesn’t actually have to do anything itself—only give answers. This makes much easier to build and creates an extra safeguard for free.
But is there anything more we can pare away? For example, a provably correct natural language parser is impossible because natural language is ambiguous and inconsistent. Humans certainly don’t always parse it correctly. On the other hand it’s easy for a human to learn a machine language and huge numbers of them have already done so.
So in the chain of events below, the AIs responsibility would be limited to the words in all caps and humans do the rest.
[1 articulate a need] → [2 formulate an unambiguous query] → [3 FIND CANDIDATE MODELS] → [4 user chooses a model or revises step 2] → [5 RETURN OPTIMAL MANIPULATIONS TO THE MODEL] → [6 user implements manipulation or revises step 2]
We have generic algorithms that do step 5. They don’t always scale well, but that’s an engineering problem, that a lot of people in fields outside AI are already working to solve. We have domain-specific algorithms some of which can do a decent job of step 3-- spam filters, recommendation engines, autocorrectors.
So, does this mean that what’s really missing is a generic problem-representor?
Well, that and friendliness, but if we can articulate a coherent, unambiguous code of morality, we will still need a generic problem-representer to actually incorporate it into the optimization procedure.
Note: This is rather long. Sorry about that, if it’s too long, but I was making notes on a comparable question recently, and you did say to be specific.
Well, the last time I was thinking of features in an AI, was one that could ask questions that could potentially cleave answer space for any poorly formulated request(Because as a well known problem, there is far more processing time and programmer effort spent on poorly formulated requests then well formulated ones) and show a video demonstration of some likely behaviors prior to your answering assuming what it considered the most plausible potential answers.
1: Incredibly simplistic example: you request the AI: “Make me a Line graph of these numbers.”
The AI responds: “Do you want a Line Graph, with number set one as the X axis, which would look like this, or do you want a Line Graph with number set two as the X axis, which would look like this, or are you thinking of something else?”
Without that feature, a program might take that and just go “ERROR: Insufficient parameters.” Or, worse, they’ll give you the WRONG line graph, and they won’t tell you. Both are annoying.
2: More complicated request example: you request the AI: “Bring a drink inside the fridge to me.”
The AI may respond: “Do you want me to bring an entire two liter bottle of soda, which would look like this, or do you want me to bring a can of soda, which would look like this, or do you want me to drag the entire fridge to your position, which would look like this, or are you thinking of something else?”
This is where the video demonstration comes in handy. Relative to you, the AI doesn’t have common sense. It doesn’t necessarily know that you probably don’t want it to drag the entire fridge to your position to the same level of confidence you do. It just puts together some plausible answers and lets you pick one.
(Note: There is nothing particularly special about video, other than that it shows that the AI shouldn’t start actually doing things until you’ve narrowed the request down, and that some requests have mistakes too complicated to be picked up by text only answers.)
Of course, in the previous case, the AI was mostly on the money. It’s also possible that the AI doesn’t have the slightest idea what you’re are talking about in a dangerous way.
3: You request the AI: “Subdue the crazed man inside the building of innocents.”
The AI may respond: “Do you want me to level the building with TNT, which would look like this, or do you want me to gas the building with Chemicals, which would look like this, or do you want me to concoct an Contagious All purpose, Anti-Human Virus and infect him, which would look like this, or are you thinking of something else?”
Whoops. You forgot to tell the AI to subdue the crazed man WITHOUT hurting the innocents. Thankfully it clearly demonstrated what plans it had put together before it implemented any of them, because that would’ve been bad. You may have also not noted that you wanted the man subdued non-lethally.
This also helps (not prevents entirely, of course) guard against the AI just doing something completely inexplicable relative to you.
4: “What’s 2+2?”
“Is the answer 4, like this, or do you want me to build a solid sphere of Computronium advancing at lightspeed and continue calculating, like this, or are you thinking of something else?”
If you wanted the AI to be better, you could network it to a cloud database so it could learn “In 1000 cases When people ask for a drink inside the fridge, they never mean drag the fridge to them, I should stop giving that as a likely answer, and add ’Would you like me to pour your drink in a glass? which seems to come up quite a bit as an answer selected from Something else.”
Since danger goes up as capability does, I’d probably want to train a dumber AI on a great deal of 1 and 2 before ever getting anywhere near 3, let alone something as existential as 4.
Of course, if the AI is substantially good enough at ANY request, it can respond to a request of:
5: “Increase your computing power.”
With “Do you want me to increase my computing power by building more cores like this, or do you want me to increase my computing power by switching out my existing cores with this design that I came up, like this, or are you thinking of something else?”
In which case you have a Self-Improving AI that, after a great deal of training, gets what humanity wants when they tell it things, so it can be enlisted to save the world and make everyone live happily forever.
Except, that what this DIDN’T do, is it didn’t actually follow the procedure that I laid out above, and give an alternate option under the assumption that this isn’t what you asked for(and it might be very dangerous), and possibly ask if you just wanted something else entirely.
I want to note that those steps should be here, but have mainly been cut for space constraints, except as rough mentions.
Obvious problem 1: the video output or descriptions can contain basilisks, including ones that cause problem 2.
Obvious problem 2: Someone could ask and then verify it to make full UFAI without realizing it.
Obvious problem 3: UFAI could arise inside the simulation used to produce the hypotheticals, and either hack it’s way out directly or cause problem 1 followed by problem 2.
And most oblivious problem of all: Withe being able to repeatedly do the highly dangerous and active step of modifying it’s own source code, it’ll never get smart enough to be useful on 95% of queries.
Fair point. in that case, given an unknown partially complete AI, if the first action you take is “Let me just start reading the contents of these files without running it to see what it even does.” then someone could say “A UFAI put a basilisk in the source code and used it to kill all of humanity, you lose.”
That isn’t even entirely without precedent, using this as an example: http://boingboing.net/2012/07/10/dropped-infected-usb-in-the-co.html Sometimes malicious code really is literally left physically lying around, waiting for someone to pop it into a computer out of curiosity.
So, in the general case, something that will take a natural language request, turn it into a family of optimizable models, identify the most promising ones, ask the user to choose, and then return an optimized answer?
Notice that it doesn’t actually have to do anything itself—only give answers. This makes much easier to build and creates an extra safeguard for free.
But is there anything more we can pare away? For example, a provably correct natural language parser is impossible because natural language is ambiguous and inconsistent. Humans certainly don’t always parse it correctly. On the other hand it’s easy for a human to learn a machine language and huge numbers of them have already done so.
So in the chain of events below, the AIs responsibility would be limited to the words in all caps and humans do the rest.
[1 articulate a need] → [2 formulate an unambiguous query] → [3 FIND CANDIDATE MODELS] → [4 user chooses a model or revises step 2] → [5 RETURN OPTIMAL MANIPULATIONS TO THE MODEL] → [6 user implements manipulation or revises step 2]
Continued from above, to reduce TLDR-ness...
We have generic algorithms that do step 5. They don’t always scale well, but that’s an engineering problem, that a lot of people in fields outside AI are already working to solve. We have domain-specific algorithms some of which can do a decent job of step 3-- spam filters, recommendation engines, autocorrectors.
So, does this mean that what’s really missing is a generic problem-representor?
Well, that and friendliness, but if we can articulate a coherent, unambiguous code of morality, we will still need a generic problem-representer to actually incorporate it into the optimization procedure.