It was definitely not common knowledge! I think some people just read Eliezer’s writing on TDT, made some extremely confident conclusions about what that means how you should behave, and decided to go around policing those norms. This seems to still happen with random other things Eliezer writes (c.f. Mikhail trying to enforce a norm of “you have to be the kind of agent that other people never regret telling something to based on a fictional story with gods that are perfectly capable of compartmentalizing information”).
Then I think “MIRI never committed to not paying off blackmail” is a perfectly reasonable argument (as is “it was a returned donation”), but “it’s not paying off blackmail, it’s just giving money to a person who’s threatening me so they won’t follow through” is still bizarre.
No? From an incentive and game theory perspective there are huge differences between a world where people can get rich exploiting others and a world where they can’t. The exact lines are blurry but there are obviously some ethical principles here that seem reasonable to try to hold people accountable by.
The right level of accountability is unclear and how to set the right incentives to not create terrorists is a tricky question, but trying to argue that there isn’t a very important difference here seems absurd.
It was definitely not common knowledge! I think some people just read Eliezer’s writing on TDT, made some extremely confident conclusions about what that means how you should behave, and decided to go around policing those norms. This seems to still happen with random other things Eliezer writes (c.f. Mikhail trying to enforce a norm of “you have to be the kind of agent that other people never regret telling something to based on a fictional story with gods that are perfectly capable of compartmentalizing information”).
Then I think “MIRI never committed to not paying off blackmail” is a perfectly reasonable argument (as is “it was a returned donation”), but “it’s not paying off blackmail, it’s just giving money to a person who’s threatening me so they won’t follow through” is still bizarre.
No? From an incentive and game theory perspective there are huge differences between a world where people can get rich exploiting others and a world where they can’t. The exact lines are blurry but there are obviously some ethical principles here that seem reasonable to try to hold people accountable by.
The right level of accountability is unclear and how to set the right incentives to not create terrorists is a tricky question, but trying to argue that there isn’t a very important difference here seems absurd.
The distinctions between crows and ostriches are important for many situations but that doesn’t make either one not a bird.