I think AI should threat value function as probabilistic. I.e. instead of thinking “this world has value of exactly N” it could think something like “I 90% sure that this world has value N+-M, but there is 10% possibility that it could actuall have value -ALOT”. And would avoid that world, because it would give a very low expected value on averager.
I think AI should threat value function as probabilistic. I.e. instead of thinking “this world has value of exactly N” it could think something like “I 90% sure that this world has value N+-M, but there is 10% possibility that it could actuall have value -ALOT”. And would avoid that world, because it would give a very low expected value on averager.
It’s a reasonable idea. See here though: https://arbital.com/p/updated_deference/