Very accurate and general Predictors may be based on Solomonoff’s theory of universal induction. Very powerful Predictors are unsafe in a rather surprising way: when given sufficient data about the real world, they exhibit goal-seeking behavior, i.e. they calculate a distribution over future data in a way that brings about certain real-world states. This is surprising, since a Predictor is theoretically just a very large and expensive application of Bayes’ law, not even performing a search over its possible outputs.
I claimed that Solomonoff induction did not do that here. Does anyone disagree? Is this a point of contention?