The extenuating circumstances were significant, and the blackmailer plausibly undeterrable even if they had been predictably unexploitable.
But they were predictably unexploitable. The blackmailer didn’t turn a profit! He was someone who had worked at a startup in the SF Bay Area and likely could get another tech job, going up to 100s of $k of salary per year! The important thing about your payout policy is that people should expect that it is not worth their time to try to extort you. And that’s what happened here.[1]
If someone manages to pay a lunatic $1 to go away after 6-9 months of harassment, we would not say “what a terrible incentive you’ve set up for yourself, now everyone is going to try to come for your money”. We’d say “what a fool that guy was to do all that work for $1″.
It was not a profitable extortion, in the way that means someone is predictably exploitable. It was an expensive way for the lunatic to waste his own time.
Note that we don’t know the exact figure, but I recall looking up their public tax filings that year and there being very little money that could have been spent on this, and concluding it was somewhere in the $20k to $60k range.
what is the technical definition of blackmail? Boring, taboo the term.
What is the right policy to have in regards to paying off aggressors? Maybe it’s fine as long as it’s not net-positive for the aggressor.
What principle did MIRI articulate and commit to before the Louis Helm lawsuit? Did they act in accordance with those principles?
I’m tentatively fine with MIRI or anyone else holding the principle “we’ll pay costs to make lawsuits go away but not enough to make it worth your while”. But AFAIK what they said was more in line with “We’ll never pay off aggressors”. This is a crux for me when judging if they held to their commitments and will hold to their commitments in the future, and that’s true even if “only small pay-offs” is the better policy, because that’s evidence they made a strong commitment without thinking through the edge cases.
I don’t know, I assume LessWrong or their website? I wasn’t there at the time but neither sides’ statements over the last 10 years make sense unless it was common knowledge at the time that MIRI pledged to never pay off blackmail.
It was definitely not common knowledge! I think some people just read Eliezer’s writing on TDT, made some extremely confident conclusions about what that means how you should behave, and decided to go around policing those norms. This seems to still happen with random other things Eliezer writes (c.f. Mikhail trying to enforce a norm of “you have to be the kind of agent that other people never regret telling something to based on a fictional story with gods that are perfectly capable of compartmentalizing information”).
Then I think “MIRI never committed to not paying off blackmail” is a perfectly reasonable argument (as is “it was a returned donation”), but “it’s not paying off blackmail, it’s just giving money to a person who’s threatening me so they won’t follow through” is still bizarre.
No? From an incentive and game theory perspective there are huge differences between a world where people can get rich exploiting others and a world where they can’t. The exact lines are blurry but there are obviously some ethical principles here that seem reasonable to try to hold people accountable by.
The right level of accountability is unclear and how to set the right incentives to not create terrorists is a tricky question, but trying to argue that there isn’t a very important difference here seems absurd.
I don’t think I’ve ever heard it claimed that MIRI said this outright. The usual argument AIUI is: MIRI staff endorse/believe-in TDT (not that MIRI has stated it as organizational policy or anything), TDT endorses not paying off blackmail (and this is obvious/well-known), therefore MIRI staff violated their endorsed (and also, importantly, correct) principles.
If your numbers are right, the blackmailer came out with a reasonable rate of return on his effort. $20k for harassing someone you don’t like (I forget the details, but I understood him to have hostile personal history with several people involved) for half a year, well less than full time, is not a trivial reward. $60k is a pretty solid hourly rate, especially if he was not really in shape to work at the same salary as before, which on priors for someone who does something like this is very very likely.
If your when-healthy market salary is $200k, but you’re not healthy, so you can only get $120k/year from a job you can get and keep? And you get EV $30k from 15 hours a week of harassment for 6-9 months? That’s about your market rate! You’d do better than that working freelance but not that much better, and under many theories of being angry and unwell this is an easier ‘job’ to keep.
Someone told me that there was some dispute about whether he was suing MIRI for a payout, or suing to have a donation he gave returned. If it’s the latter that does feel somewhat different to me, and maybe outside the definition of blackmail.
Yeah in the latter case I would say this is maybe complicatedly blackmail, but definitely not straightforwardly blackmail (and maybe not blackmail at all). The strong nondisparagement clauses that must have been involved make that somewhat implausible to me, but I would need to know more details than I ever bothered to learn to determine how implausible and it doesn’t seem worth the effort.
But they were predictably unexploitable. The blackmailer didn’t turn a profit! He was someone who had worked at a startup in the SF Bay Area and likely could get another tech job, going up to 100s of $k of salary per year! The important thing about your payout policy is that people should expect that it is not worth their time to try to extort you. And that’s what happened here.[1]
If someone manages to pay a lunatic $1 to go away after 6-9 months of harassment, we would not say “what a terrible incentive you’ve set up for yourself, now everyone is going to try to come for your money”. We’d say “what a fool that guy was to do all that work for $1″.
It was not a profitable extortion, in the way that means someone is predictably exploitable. It was an expensive way for the lunatic to waste his own time.
Note that we don’t know the exact figure, but I recall looking up their public tax filings that year and there being very little money that could have been spent on this, and concluding it was somewhere in the $20k to $60k range.
There’s a few different questions here:
what is the technical definition of blackmail? Boring, taboo the term.
What is the right policy to have in regards to paying off aggressors? Maybe it’s fine as long as it’s not net-positive for the aggressor.
What principle did MIRI articulate and commit to before the Louis Helm lawsuit? Did they act in accordance with those principles?
I’m tentatively fine with MIRI or anyone else holding the principle “we’ll pay costs to make lawsuits go away but not enough to make it worth your while”. But AFAIK what they said was more in line with “We’ll never pay off aggressors”. This is a crux for me when judging if they held to their commitments and will hold to their commitments in the future, and that’s true even if “only small pay-offs” is the better policy, because that’s evidence they made a strong commitment without thinking through the edge cases.
What, where would MIRI have said anything like this? What kind of weird commitments are people imagining?
I don’t know, I assume LessWrong or their website? I wasn’t there at the time but neither sides’ statements over the last 10 years make sense unless it was common knowledge at the time that MIRI pledged to never pay off blackmail.
It was definitely not common knowledge! I think some people just read Eliezer’s writing on TDT, made some extremely confident conclusions about what that means how you should behave, and decided to go around policing those norms. This seems to still happen with random other things Eliezer writes (c.f. Mikhail trying to enforce a norm of “you have to be the kind of agent that other people never regret telling something to based on a fictional story with gods that are perfectly capable of compartmentalizing information”).
Then I think “MIRI never committed to not paying off blackmail” is a perfectly reasonable argument (as is “it was a returned donation”), but “it’s not paying off blackmail, it’s just giving money to a person who’s threatening me so they won’t follow through” is still bizarre.
No? From an incentive and game theory perspective there are huge differences between a world where people can get rich exploiting others and a world where they can’t. The exact lines are blurry but there are obviously some ethical principles here that seem reasonable to try to hold people accountable by.
The right level of accountability is unclear and how to set the right incentives to not create terrorists is a tricky question, but trying to argue that there isn’t a very important difference here seems absurd.
The distinctions between crows and ostriches are important for many situations but that doesn’t make either one not a bird.
I don’t think I’ve ever heard it claimed that MIRI said this outright. The usual argument AIUI is: MIRI staff endorse/believe-in TDT (not that MIRI has stated it as organizational policy or anything), TDT endorses not paying off blackmail (and this is obvious/well-known), therefore MIRI staff violated their endorsed (and also, importantly, correct) principles.
If your numbers are right, the blackmailer came out with a reasonable rate of return on his effort. $20k for harassing someone you don’t like (I forget the details, but I understood him to have hostile personal history with several people involved) for half a year, well less than full time, is not a trivial reward. $60k is a pretty solid hourly rate, especially if he was not really in shape to work at the same salary as before, which on priors for someone who does something like this is very very likely.
If your when-healthy market salary is $200k, but you’re not healthy, so you can only get $120k/year from a job you can get and keep? And you get EV $30k from 15 hours a week of harassment for 6-9 months? That’s about your market rate! You’d do better than that working freelance but not that much better, and under many theories of being angry and unwell this is an easier ‘job’ to keep.
Someone told me that there was some dispute about whether he was suing MIRI for a payout, or suing to have a donation he gave returned. If it’s the latter that does feel somewhat different to me, and maybe outside the definition of blackmail.
Yeah in the latter case I would say this is maybe complicatedly blackmail, but definitely not straightforwardly blackmail (and maybe not blackmail at all). The strong nondisparagement clauses that must have been involved make that somewhat implausible to me, but I would need to know more details than I ever bothered to learn to determine how implausible and it doesn’t seem worth the effort.