The only argument for the headline claim “$50 million a year for a 10% chance to ban ASI” is a footnote that says “The probabilities are produced mostly by gut feeling.”
rationalists use gut-level probability forecasts as a way to specify confidence all the time. that’s like the number 2 shared characteristic of rat culture, behind “believing that ASI will kill everyone”
sure, but it’s just weird? about as weird as commenting on the post saying “i actually think AI is super overblown and we’re going to be ok”, and equally weird that it would be widely upvoted. my model of rat culture is that “Preventing extinction from ASI on a $50M yearly budget” is actually slightly more of a clickbait title than “$50 million a year for a 10% chance to ban ASI”. and if you’re saying “it’s dumb to use made up probabilities to express your confidence”, then that’s a debate that has been had for decades on this forum, at this point.
A title is supposed to describe the contents. If I saw a post saying “The probability of X is P” I would expect to see an explanation of P in particular, as opposed to P^2 or P^0.5.
rationalists use gut-level probability forecasts as a way to specify confidence all the time. that’s like the number 2 shared characteristic of rat culture, behind “believing that ASI will kill everyone”
That’s not actually responding to the criticism. Rat culture could just be wrong!
sure, but it’s just weird? about as weird as commenting on the post saying “i actually think AI is super overblown and we’re going to be ok”, and equally weird that it would be widely upvoted. my model of rat culture is that “Preventing extinction from ASI on a $50M yearly budget” is actually slightly more of a clickbait title than “$50 million a year for a 10% chance to ban ASI”. and if you’re saying “it’s dumb to use made up probabilities to express your confidence”, then that’s a debate that has been had for decades on this forum, at this point.
A title is supposed to describe the contents. If I saw a post saying “The probability of X is P” I would expect to see an explanation of P in particular, as opposed to P^2 or P^0.5.