IMHO, the idea that wealth can’t usefully be measured is one which is not sufficiently worthwhile to merit further discussion.
The “wealth” idea sounds vulnerable to hidden complexity of wishes. Measure it in dollars and you get hyperinflation. Measure it in resources, and the AI cuts down all the trees and converts them to lumber, then kills all the animals and converts them to oil, even if technology had advanced beyond the point of needing either. Find some clever way to specify the value of all resources, convert them to products and allocate them to humans in the level humans want, and one of the products will be highly carcinogenic because the AI didn’t know humans don’t like that. The only way to get wealth in the way that’s meaningful to humans without humans losing other things they want more than wealth is for the AI to know exactly what we want as well or better than we do. And if it knows that, we can ignore wealth and just ask it to do what it knows we want.
“The counterargument is, in part, that some classifiers are better than others, even when all of them satisfy the training data completely. The most obvious criterion to use is the complexity of the classifier.”
I don’t think “better” is meaningful outside the context of a utility function. Complexity isn’t a utility function and it’s inadequate for this purpose. Which is better, tank vs. non-tank or cloudy vs. sunny? I can’t immediately see which is more complex than the other. And even if I could, I’d want my criteria to change depending on whether I’m in an anti-tank infantry or a solar power installation company, and just judging criteria by complexity doesn’t let me make that change, unless I’m misunderstanding what you mean by complexity here.
Meanwhile, reading the link to Bill Hibbard on the SL4 list:
“Your scenario of a system that is adequate for intelligence in its ability to rule the world, but absurdly inadequate for intelligence in its inability to distinguish a smiley face from a human, is inconsistent.”
I think the best possible summary of Overcoming Bias thus far would be “Abandon all thought processes even remotely related to the ones that generated this statement.”
IMHO, the idea that wealth can’t usefully be measured is one which is not sufficiently worthwhile to merit further discussion.
The “wealth” idea sounds vulnerable to hidden complexity of wishes. Measure it in dollars and you get hyperinflation. Measure it in resources, and the AI cuts down all the trees and converts them to lumber, then kills all the animals and converts them to oil, even if technology had advanced beyond the point of needing either. Find some clever way to specify the value of all resources, convert them to products and allocate them to humans in the level humans want, and one of the products will be highly carcinogenic because the AI didn’t know humans don’t like that. The only way to get wealth in the way that’s meaningful to humans without humans losing other things they want more than wealth is for the AI to know exactly what we want as well or better than we do. And if it knows that, we can ignore wealth and just ask it to do what it knows we want.
“The counterargument is, in part, that some classifiers are better than others, even when all of them satisfy the training data completely. The most obvious criterion to use is the complexity of the classifier.”
I don’t think “better” is meaningful outside the context of a utility function. Complexity isn’t a utility function and it’s inadequate for this purpose. Which is better, tank vs. non-tank or cloudy vs. sunny? I can’t immediately see which is more complex than the other. And even if I could, I’d want my criteria to change depending on whether I’m in an anti-tank infantry or a solar power installation company, and just judging criteria by complexity doesn’t let me make that change, unless I’m misunderstanding what you mean by complexity here.
Meanwhile, reading the link to Bill Hibbard on the SL4 list:
“Your scenario of a system that is adequate for intelligence in its ability to rule the world, but absurdly inadequate for intelligence in its inability to distinguish a smiley face from a human, is inconsistent.”
I think the best possible summary of Overcoming Bias thus far would be “Abandon all thought processes even remotely related to the ones that generated this statement.”