I don’t exactly disagree, but I’m concerned you might be downplaying the bias you mention in an ancestor. My study of the field’s been fairly casual (and focused more on archaeological than historical methodology), but I’ve seen enough to know that academically respectable analyses vary wildly, and generally tend to line up with identity-group membership on the part of their exponents; most of the predictive power of history as a field also seems to lie in interpretation rather than in content. To make matters worse, we don’t have time to verify historical interpretations empirically; few respectable ones make significant predictions that’re valid on timescales less than a few decades.
If we’re interested in making predictions about the future based on the historical record, therefore, we’re left with the problem of choosing an interpretation based on its own internal characteristics. We do have some heuristics to work with, like simplicity and lack of post-facto revisions around major changes in the past, but solving this problem in a reliable way looks to me like it might be Friendliness-complete. And the consequences of failure are scarcely less dire than failing at Friendliness itself, if we’re using it to inform our approach to the latter problem.
I agree with you about how difficult the problem of finding unbiased history—the problem is probably harder than gwern suggested. At best, this problem is Friendliness-complete, in that if Omega gave us a solution to Friendliness, it would include a solution to this problem. And I’m not optimistic that the best case is true.
I think solving the problem is a prerequisite to solving Friendliness. It’s probably a prerequisite for a rigorous understanding of how CEV or its equivalent will work. The fact that the community (and SIAI to a lesser extent) think this type of analysis is irrelevant is terribly disturbing to me.
At best, this problem is Friendliness-complete, in that if Omega gave us a solution to Friendliness, it would include a solution to this problem.
I think solving the problem is a prerequisite to solving Friendliness. It’s probably a prerequisite for a rigorous understanding of how CEV or its equivalent will work. T
The FAI project is about finding the moral theory that is correct,(1) then implementing potential AGIs so that they will implement that process of making decisions. I’m not aware of anything other than history that is a viable candidate to be evidence that a particular moral theory is correct.
Further, a FAI would need the capacity to predict how a human society would react to various circumstances or interventions. Again, history is the only data on how human societies react.
(1) I acknowledge the need to taboo “correct” in this context in order to make progress on this front.
I’m not aware of anything other than history that is a viable candidate to be evidence that a particular moral theory is correct.
Its possible that you’re using correct to mean something completely different than I would use it to mean, but I don’t see how history is supposed to be evidence that a moral theory is correct. Are you saying that historically widespread moral theories are likely to be correct?
Further, a FAI would need the capacity to predict how a human society would react to various circumstances or interventions.
This is something that the AI is supposed to figure out for itself, not something that would be hardcoded in (at least not in currently favored designs).
I don’t exactly disagree, but I’m concerned you might be downplaying the bias you mention in an ancestor. My study of the field’s been fairly casual (and focused more on archaeological than historical methodology), but I’ve seen enough to know that academically respectable analyses vary wildly, and generally tend to line up with identity-group membership on the part of their exponents; most of the predictive power of history as a field also seems to lie in interpretation rather than in content. To make matters worse, we don’t have time to verify historical interpretations empirically; few respectable ones make significant predictions that’re valid on timescales less than a few decades.
If we’re interested in making predictions about the future based on the historical record, therefore, we’re left with the problem of choosing an interpretation based on its own internal characteristics. We do have some heuristics to work with, like simplicity and lack of post-facto revisions around major changes in the past, but solving this problem in a reliable way looks to me like it might be Friendliness-complete. And the consequences of failure are scarcely less dire than failing at Friendliness itself, if we’re using it to inform our approach to the latter problem.
I agree with you about how difficult the problem of finding unbiased history—the problem is probably harder than gwern suggested. At best, this problem is Friendliness-complete, in that if Omega gave us a solution to Friendliness, it would include a solution to this problem. And I’m not optimistic that the best case is true.
I think solving the problem is a prerequisite to solving Friendliness. It’s probably a prerequisite for a rigorous understanding of how CEV or its equivalent will work. The fact that the community (and SIAI to a lesser extent) think this type of analysis is irrelevant is terribly disturbing to me.
Why do you believe this?
The FAI project is about finding the moral theory that is correct,(1) then implementing potential AGIs so that they will implement that process of making decisions. I’m not aware of anything other than history that is a viable candidate to be evidence that a particular moral theory is correct.
Further, a FAI would need the capacity to predict how a human society would react to various circumstances or interventions. Again, history is the only data on how human societies react.
(1) I acknowledge the need to taboo “correct” in this context in order to make progress on this front.
Its possible that you’re using correct to mean something completely different than I would use it to mean, but I don’t see how history is supposed to be evidence that a moral theory is correct. Are you saying that historically widespread moral theories are likely to be correct?
This is something that the AI is supposed to figure out for itself, not something that would be hardcoded in (at least not in currently favored designs).