Generalizing, this looks like the gambler’s ruin (even positive EV bets can be bad bets, if the losses would be unrecoverable—“quadruple or nothing at 50%, but you’re betting all you have” predictably ends with you having nothing if you keep playing long enough). Except not with units of money, but units of motivation, or feeling like a good person.
If the bet is “amazingly good impact or ruinously bad impact”, you probably shouldn’t take that bet unless you’re pretty certain it’s not going to turn out ruinously bad. And more generally, you shouldn’t take bets you can’t afford to lose, except in some edge cases.
I think those who have a hubristic level of certainty that their AI impact will be good, are in some cases gambling with other people’s money/lives, and that’s bad. If it’s bad for me to bet everything I have unless I’m pretty sure I’m going to win, it’s extra-bad for someone to take all my money without my consent and make a bet that they are just kinda sure is probably positive EV, but could leave me with nothing. And gambling with my life is worse than gambling with my money.
On the other hand, if you’re betting with your own resources, or resources someone has willingly given you knowing the risks, and the upside is big and the downside is big but not ruinously so, I’d say go for it if you feel like it.
Generalizing, this looks like the gambler’s ruin (even positive EV bets can be bad bets, if the losses would be unrecoverable—“quadruple or nothing at 50%, but you’re betting all you have” predictably ends with you having nothing if you keep playing long enough). Except not with units of money, but units of motivation, or feeling like a good person.
If the bet is “amazingly good impact or ruinously bad impact”, you probably shouldn’t take that bet unless you’re pretty certain it’s not going to turn out ruinously bad. And more generally, you shouldn’t take bets you can’t afford to lose, except in some edge cases.
I think those who have a hubristic level of certainty that their AI impact will be good, are in some cases gambling with other people’s money/lives, and that’s bad. If it’s bad for me to bet everything I have unless I’m pretty sure I’m going to win, it’s extra-bad for someone to take all my money without my consent and make a bet that they are just kinda sure is probably positive EV, but could leave me with nothing. And gambling with my life is worse than gambling with my money.
On the other hand, if you’re betting with your own resources, or resources someone has willingly given you knowing the risks, and the upside is big and the downside is big but not ruinously so, I’d say go for it if you feel like it.