He may have some model of an AI as a perfect Bayesian reasoner that he uses to justify neglecting this. I am immediately suspicious of any argument invoking perfection.
It may also be that what Eliezer has in mind is that any heuristic that can be represented to the AI, could be assigned priors and incorporated into Bayesian reasoning.
Eliezer has read Judea Pearl, so he knows how computational time for Bayesian networks scales with the domain, particularly if you don’t ever assume independence when it is not justified, so I won’t lecture him on that. But he may want to lecture himself.
(Constructing the right Bayesian network from sense-data is even more computationally demanding. Of course, if you never assume independence, then the only right network is the fully-connected one. I’m pretty certain that suggesting that a non-narrow AI will be reasoning over all of its knowledge with a fully-connected Bayesian network is computationally implausible. So all arguments that require AIs to be perfect Bayesian reasoners are invalid.)
I’d like to know how much of what Eliezer says depends on the AI using Bayesian logic as its only reasoning mechanism, and whether he believes that is the best reasoning mechanism in all cases, or only one that must be used in order to keep the AI friendly.
Kaj: I will restate my earlier question this way: “Would AIs also find themselves in circumstances such that game theory dictates that they act corruptly?” It doesn’t matter whether we say that the behavior evolved from accumulated mutations, or whether an AI reasoned it out in a millisecond. The problem is still there, if circumstances give corrupt behavior an advantage.
Eliezer has read Judea Pearl, so he knows how computational time for Bayesian networks scales with the domain, particularly if you don’t ever assume independence when it is not justified, so I won’t lecture him on that. But he may want to lecture himself.
(Constructing the right Bayesian network from sense-data is even more computationally demanding. Of course, if you never assume independence, then the only right network is the fully-connected one. I’m pretty certain that suggesting that a non-narrow AI will be reasoning over all of its knowledge with a fully-connected Bayesian network is computationally implausible. So all arguments that require AIs to be perfect Bayesian reasoners are invalid.)
I’d like to know how much of what Eliezer says depends on the AI using Bayesian logic as its only reasoning mechanism, and whether he believes that is the best reasoning mechanism in all cases, or only one that must be used in order to keep the AI friendly.
Kaj: I will restate my earlier question this way: “Would AIs also find themselves in circumstances such that game theory dictates that they act corruptly?” It doesn’t matter whether we say that the behavior evolved from accumulated mutations, or whether an AI reasoned it out in a millisecond. The problem is still there, if circumstances give corrupt behavior an advantage.