I disagree that it’s a flaw. Discounting the future, even asymptotically, is a preference statement, not a logical shortcoming. Consider this situation:
Omega offers you two bets, and you must choose one. Bet #1 says you have a 50% chance of dying immediately, and a 50% chance of living 10 average lifespans. Bet #2 says you have a 100% chance of living a single average lifespan.
Having lived a reasonable part of an average lifespan, I can grok these numbers quite well. Still, I would choose Bet #2. Given the opportunity, I wouldn’t modify myself to prefer Bet #1. Moreover, I hope any AI with the power and necessity to choose one of these bets for me, would choose Bet #2.
Yes, fair enough; I should have said “accept the way the brain currently works” rather than using loaded language—apparently I’m not quite following my own prescription. :)
I disagree that it’s a flaw. Discounting the future, even asymptotically, is a preference statement, not a logical shortcoming. Consider this situation:
Omega offers you two bets, and you must choose one. Bet #1 says you have a 50% chance of dying immediately, and a 50% chance of living 10 average lifespans. Bet #2 says you have a 100% chance of living a single average lifespan.
Having lived a reasonable part of an average lifespan, I can grok these numbers quite well. Still, I would choose Bet #2. Given the opportunity, I wouldn’t modify myself to prefer Bet #1. Moreover, I hope any AI with the power and necessity to choose one of these bets for me, would choose Bet #2.
Yes, fair enough; I should have said “accept the way the brain currently works” rather than using loaded language—apparently I’m not quite following my own prescription. :)