@Felix
I started out trying to make a case that the systems we have that seem most promising are the ones that learn, in one way or another. A general intelligence has to be something that you dump training data into.
It IS easier to run the neural net through trials when you don’t understand the problem and might not even know where to start. As eliezer said “Deep Blue’s programmers had to know the rules of chess—since Deep Blue wasn’t enough of a general AI to learn the rules by observation”
On the one hand, this is fine, we know the rules of physics close enough for the AI to design things we ask for. But it can’t do real autonomous planning or self modification (and still be friendly) without a friendly preference ordering/utility function.
The way Eliezer suggests (if I understand) to create a friendly utility function/preference ordering is to have the AI analyze human brains and prefer outcomes we would prefer if we were that smart.
But since we don’t have the ‘rules of the Brain’, the AI has to learn. (unless it does a full brain simulation every time it needs to make a preference decision, but I’d rank that as a low preference.)
I’m having trouble imagining a way to engineer a learning system where we know what will come out of it without ourselves understanding the problem. True, we only need to know the end result, and not intermediate steps. But how can we know if a learning system will miss something vital on a problem that is also beyond our scope?
If you didn’t know the rules of chess, you could make an AI learn the rules. But when do you bet your life that it hasn’t missed something obscure but important?
@Felix I started out trying to make a case that the systems we have that seem most promising are the ones that learn, in one way or another. A general intelligence has to be something that you dump training data into.
It IS easier to run the neural net through trials when you don’t understand the problem and might not even know where to start. As eliezer said “Deep Blue’s programmers had to know the rules of chess—since Deep Blue wasn’t enough of a general AI to learn the rules by observation”
On the one hand, this is fine, we know the rules of physics close enough for the AI to design things we ask for. But it can’t do real autonomous planning or self modification (and still be friendly) without a friendly preference ordering/utility function.
The way Eliezer suggests (if I understand) to create a friendly utility function/preference ordering is to have the AI analyze human brains and prefer outcomes we would prefer if we were that smart. But since we don’t have the ‘rules of the Brain’, the AI has to learn. (unless it does a full brain simulation every time it needs to make a preference decision, but I’d rank that as a low preference.)
I’m having trouble imagining a way to engineer a learning system where we know what will come out of it without ourselves understanding the problem. True, we only need to know the end result, and not intermediate steps. But how can we know if a learning system will miss something vital on a problem that is also beyond our scope?
If you didn’t know the rules of chess, you could make an AI learn the rules. But when do you bet your life that it hasn’t missed something obscure but important?