No. They paid a relatively small sum of money for a lunatic who had been harassing them and fraudulently pretending to be them and calling the police on their houses at random hours of the night to finally shut up and go away (after 6+ months of it).
That’s still giving into blackmail. Much like a settlement that allows an aggressor to not admit wrongdoing is giving into blackmail. Sometimes you, as a practical matter, have to give in to blackmail, if you’re acting with people too irrational to be deterred by predictably not giving in. Sometimes you have not had the opportunity to demonstrate it and do not believe you will have cause to have publicly demonstrated it in the future, and so the pragmatic case says ‘Sure, give in, the cross-time coalition of my selves agrees that it’s not worth it to refuse.’
But I think it does say something negative. They were, then, an organization which was paying a great deal of attention to optimal timeless decision theory that prescribe being shaped across time to be the sort of thing that cannot be usefully blackmailed because it predictably does not give in to it. And when it came to the practical case where they could implement that policy… they gave in. It’s not great to fail to take your beliefs seriously, and this chain of events suggests that they did not. But that’s a venial sin. The extenuating circumstances were significant, and the blackmailer plausibly undeterrable even if they had been predictably unexploitable.
In other words, the substance of the accusation was correct; it was the volume and vehemence of it which was disproportionate and unjustified. (And I still wouldn’t call it morally wrong; that came later.)
The extenuating circumstances were significant, and the blackmailer plausibly undeterrable even if they had been predictably unexploitable.
But they were predictably unexploitable. The blackmailer didn’t turn a profit! He was someone who had worked at a startup in the SF Bay Area and likely could get another tech job, going up to 100s of $k of salary per year! The important thing about your payout policy is that people should expect that it is not worth their time to try to extort you. And that’s what happened here.[1]
If someone manages to pay a lunatic $1 to go away after 6-9 months of harassment, we would not say “what a terrible incentive you’ve set up for yourself, now everyone is going to try to come for your money”. We’d say “what a fool that guy was to do all that work for $1″.
It was not a profitable extortion, in the way that means someone is predictably exploitable. It was an expensive way for the lunatic to waste his own time.
Note that we don’t know the exact figure, but I recall looking up their public tax filings that year and there being very little money that could have been spent on this, and concluding it was somewhere in the $20k to $60k range.
what is the technical definition of blackmail? Boring, taboo the term.
What is the right policy to have in regards to paying off aggressors? Maybe it’s fine as long as it’s not net-positive for the aggressor.
What principle did MIRI articulate and commit to before the Louis Helm lawsuit? Did they act in accordance with those principles?
I’m tentatively fine with MIRI or anyone else holding the principle “we’ll pay costs to make lawsuits go away but not enough to make it worth your while”. But AFAIK what they said was more in line with “We’ll never pay off aggressors”. This is a crux for me when judging if they held to their commitments and will hold to their commitments in the future, and that’s true even if “only small pay-offs” is the better policy, because that’s evidence they made a strong commitment without thinking through the edge cases.
I don’t know, I assume LessWrong or their website? I wasn’t there at the time but neither sides’ statements over the last 10 years make sense unless it was common knowledge at the time that MIRI pledged to never pay off blackmail.
It was definitely not common knowledge! I think some people just read Eliezer’s writing on TDT, made some extremely confident conclusions about what that means how you should behave, and decided to go around policing those norms. This seems to still happen with random other things Eliezer writes (c.f. Mikhail trying to enforce a norm of “you have to be the kind of agent that other people never regret telling something to based on a fictional story with gods that are perfectly capable of compartmentalizing information”).
Then I think “MIRI never committed to not paying off blackmail” is a perfectly reasonable argument (as is “it was a returned donation”), but “it’s not paying off blackmail, it’s just giving money to a person who’s threatening me so they won’t follow through” is still bizarre.
No? From an incentive and game theory perspective there are huge differences between a world where people can get rich exploiting others and a world where they can’t. The exact lines are blurry but there are obviously some ethical principles here that seem reasonable to try to hold people accountable by.
The right level of accountability is unclear and how to set the right incentives to not create terrorists is a tricky question, but trying to argue that there isn’t a very important difference here seems absurd.
I don’t think I’ve ever heard it claimed that MIRI said this outright. The usual argument AIUI is: MIRI staff endorse/believe-in TDT (not that MIRI has stated it as organizational policy or anything), TDT endorses not paying off blackmail (and this is obvious/well-known), therefore MIRI staff violated their endorsed (and also, importantly, correct) principles.
If your numbers are right, the blackmailer came out with a reasonable rate of return on his effort. $20k for harassing someone you don’t like (I forget the details, but I understood him to have hostile personal history with several people involved) for half a year, well less than full time, is not a trivial reward. $60k is a pretty solid hourly rate, especially if he was not really in shape to work at the same salary as before, which on priors for someone who does something like this is very very likely.
If your when-healthy market salary is $200k, but you’re not healthy, so you can only get $120k/year from a job you can get and keep? And you get EV $30k from 15 hours a week of harassment for 6-9 months? That’s about your market rate! You’d do better than that working freelance but not that much better, and under many theories of being angry and unwell this is an easier ‘job’ to keep.
Someone told me that there was some dispute about whether he was suing MIRI for a payout, or suing to have a donation he gave returned. If it’s the latter that does feel somewhat different to me, and maybe outside the definition of blackmail.
Yeah in the latter case I would say this is maybe complicatedly blackmail, but definitely not straightforwardly blackmail (and maybe not blackmail at all). The strong nondisparagement clauses that must have been involved make that somewhat implausible to me, but I would need to know more details than I ever bothered to learn to determine how implausible and it doesn’t seem worth the effort.
There is zero chance that this whole extortion rigamarole turned out net-positive for Louie Helm. He lost much more in opportunity cost, and legal costs and reputation than he remotely gained from the maybe $30k or so he squeezed out of MIRI.
Of those, I’m inclined to place the value of losses for everything but legal costs as zero or near-zero. Someone who cares about their reputation in this respect is not going to go into this in the first place. Someone with a high opportunity cost, likewise. The fact that Helm tried this at all means either he was wildly irrational and undeterrable, or his value on his reputation was nil and his opportunity costs low.
Posit: He was unwell in a way that, durably, prevented him from working at anywhere near his former capacity. This is not particularly unlikely on priors. Severely depressed, something like that. Then, as first-order consequences:
His professional reputation is not worth very much to him. All portions of it that came from skill, he cannot maintain, and they will soon be gone regardless. His social reputation with startup people will soon follow it.
His opportunity cost is much lower than you’d otherwise expect.
Second-order consequence: If legal fees didn’t devour all his payout, this probably came out positive for him, and in expectation (heavy-tailed distribution) better than that.
And the fact that he bothered with it should be a large update in favor of this assumption. It might not push it fully to ‘more likely than not’ but I think it probably does.
Of those, I’m inclined to place the value of losses for everything but legal costs as zero or near-zero.
I… have no idea why you would do that. Of course the primary deterrent you have in almost any circumstance like this is loss of time and effort and reputation. You can’t be immune to anyone squeezing out pennies on the dollar from you. Practically every business out there is “exploitable” if you are willing to put in effort at this level of criminality and cost to yourself.
in expectation (heavy-tailed distribution)
This doesn’t make any sense. Him getting a payout below his break-even point is the topic at discussion. MIRI was not pursuing a stochastic policy that could have somehow caused him to get 10x the payout.
And the fact that he bothered with it should be a large update in favor of this assumption
Literally one of your counter-arguments is “he was unwell in a way that prevented him from working at anywhere near his former capacity” and “severely depressed, something like that”. I don’t see any reason to assume that Louie Helm was pursuing a rational strategy, indeed, I see enormous reasons to assume the opposite.
(Revisiting when my thoughts percolated into more coherent words:) You don’t get to live in the convenient world where most potential blackmailers are people with good alternative options. It’s always going to be an adversarial sample; the people who are most likely to attempt blackmail are those with the worst alternatives and the lowest costs; the ones who have little or no reputation to lose, and poor returns on their effort if it’s directed in other directions. If you can’t deter someone with minimal reputation and low value of their time from blackmailing you, your only defense against blackmail is to prevent anyone meeting that description from having information they could use to blackmail you.
I don’t see any reason to assume that Louie Helm was pursuing a rational strategy, indeed, I see enormous reasons to assume the opposite.
Do you not see how that would also reinforce my point? I don’t even know how to explain that, it’s too obvious to me. (EDIT: I thought of how to explain it and placed it in this new sibling comment.)
MIRI was not pursuing a stochastic policy that could have somehow caused him to get 10x the payout.
Reality was. He didn’t know how much money was available to be extorted, and if it was a lot, it could be quite a lot; heavy-tailed distribution.
That’s still giving into blackmail. Much like a settlement that allows an aggressor to not admit wrongdoing is giving into blackmail. Sometimes you, as a practical matter, have to give in to blackmail, if you’re acting with people too irrational to be deterred by predictably not giving in. Sometimes you have not had the opportunity to demonstrate it and do not believe you will have cause to have publicly demonstrated it in the future, and so the pragmatic case says ‘Sure, give in, the cross-time coalition of my selves agrees that it’s not worth it to refuse.’
But I think it does say something negative. They were, then, an organization which was paying a great deal of attention to optimal timeless decision theory that prescribe being shaped across time to be the sort of thing that cannot be usefully blackmailed because it predictably does not give in to it. And when it came to the practical case where they could implement that policy… they gave in. It’s not great to fail to take your beliefs seriously, and this chain of events suggests that they did not. But that’s a venial sin. The extenuating circumstances were significant, and the blackmailer plausibly undeterrable even if they had been predictably unexploitable.
In other words, the substance of the accusation was correct; it was the volume and vehemence of it which was disproportionate and unjustified. (And I still wouldn’t call it morally wrong; that came later.)
But they were predictably unexploitable. The blackmailer didn’t turn a profit! He was someone who had worked at a startup in the SF Bay Area and likely could get another tech job, going up to 100s of $k of salary per year! The important thing about your payout policy is that people should expect that it is not worth their time to try to extort you. And that’s what happened here.[1]
If someone manages to pay a lunatic $1 to go away after 6-9 months of harassment, we would not say “what a terrible incentive you’ve set up for yourself, now everyone is going to try to come for your money”. We’d say “what a fool that guy was to do all that work for $1″.
It was not a profitable extortion, in the way that means someone is predictably exploitable. It was an expensive way for the lunatic to waste his own time.
Note that we don’t know the exact figure, but I recall looking up their public tax filings that year and there being very little money that could have been spent on this, and concluding it was somewhere in the $20k to $60k range.
There’s a few different questions here:
what is the technical definition of blackmail? Boring, taboo the term.
What is the right policy to have in regards to paying off aggressors? Maybe it’s fine as long as it’s not net-positive for the aggressor.
What principle did MIRI articulate and commit to before the Louis Helm lawsuit? Did they act in accordance with those principles?
I’m tentatively fine with MIRI or anyone else holding the principle “we’ll pay costs to make lawsuits go away but not enough to make it worth your while”. But AFAIK what they said was more in line with “We’ll never pay off aggressors”. This is a crux for me when judging if they held to their commitments and will hold to their commitments in the future, and that’s true even if “only small pay-offs” is the better policy, because that’s evidence they made a strong commitment without thinking through the edge cases.
What, where would MIRI have said anything like this? What kind of weird commitments are people imagining?
I don’t know, I assume LessWrong or their website? I wasn’t there at the time but neither sides’ statements over the last 10 years make sense unless it was common knowledge at the time that MIRI pledged to never pay off blackmail.
It was definitely not common knowledge! I think some people just read Eliezer’s writing on TDT, made some extremely confident conclusions about what that means how you should behave, and decided to go around policing those norms. This seems to still happen with random other things Eliezer writes (c.f. Mikhail trying to enforce a norm of “you have to be the kind of agent that other people never regret telling something to based on a fictional story with gods that are perfectly capable of compartmentalizing information”).
Then I think “MIRI never committed to not paying off blackmail” is a perfectly reasonable argument (as is “it was a returned donation”), but “it’s not paying off blackmail, it’s just giving money to a person who’s threatening me so they won’t follow through” is still bizarre.
No? From an incentive and game theory perspective there are huge differences between a world where people can get rich exploiting others and a world where they can’t. The exact lines are blurry but there are obviously some ethical principles here that seem reasonable to try to hold people accountable by.
The right level of accountability is unclear and how to set the right incentives to not create terrorists is a tricky question, but trying to argue that there isn’t a very important difference here seems absurd.
The distinctions between crows and ostriches are important for many situations but that doesn’t make either one not a bird.
I don’t think I’ve ever heard it claimed that MIRI said this outright. The usual argument AIUI is: MIRI staff endorse/believe-in TDT (not that MIRI has stated it as organizational policy or anything), TDT endorses not paying off blackmail (and this is obvious/well-known), therefore MIRI staff violated their endorsed (and also, importantly, correct) principles.
If your numbers are right, the blackmailer came out with a reasonable rate of return on his effort. $20k for harassing someone you don’t like (I forget the details, but I understood him to have hostile personal history with several people involved) for half a year, well less than full time, is not a trivial reward. $60k is a pretty solid hourly rate, especially if he was not really in shape to work at the same salary as before, which on priors for someone who does something like this is very very likely.
If your when-healthy market salary is $200k, but you’re not healthy, so you can only get $120k/year from a job you can get and keep? And you get EV $30k from 15 hours a week of harassment for 6-9 months? That’s about your market rate! You’d do better than that working freelance but not that much better, and under many theories of being angry and unwell this is an easier ‘job’ to keep.
Someone told me that there was some dispute about whether he was suing MIRI for a payout, or suing to have a donation he gave returned. If it’s the latter that does feel somewhat different to me, and maybe outside the definition of blackmail.
Yeah in the latter case I would say this is maybe complicatedly blackmail, but definitely not straightforwardly blackmail (and maybe not blackmail at all). The strong nondisparagement clauses that must have been involved make that somewhat implausible to me, but I would need to know more details than I ever bothered to learn to determine how implausible and it doesn’t seem worth the effort.
There is zero chance that this whole extortion rigamarole turned out net-positive for Louie Helm. He lost much more in opportunity cost, and legal costs and reputation than he remotely gained from the maybe $30k or so he squeezed out of MIRI.
Of those, I’m inclined to place the value of losses for everything but legal costs as zero or near-zero. Someone who cares about their reputation in this respect is not going to go into this in the first place. Someone with a high opportunity cost, likewise. The fact that Helm tried this at all means either he was wildly irrational and undeterrable, or his value on his reputation was nil and his opportunity costs low.
Posit: He was unwell in a way that, durably, prevented him from working at anywhere near his former capacity. This is not particularly unlikely on priors. Severely depressed, something like that. Then, as first-order consequences:
His professional reputation is not worth very much to him. All portions of it that came from skill, he cannot maintain, and they will soon be gone regardless. His social reputation with startup people will soon follow it.
His opportunity cost is much lower than you’d otherwise expect.
Second-order consequence: If legal fees didn’t devour all his payout, this probably came out positive for him, and in expectation (heavy-tailed distribution) better than that.
And the fact that he bothered with it should be a large update in favor of this assumption. It might not push it fully to ‘more likely than not’ but I think it probably does.
I… have no idea why you would do that. Of course the primary deterrent you have in almost any circumstance like this is loss of time and effort and reputation. You can’t be immune to anyone squeezing out pennies on the dollar from you. Practically every business out there is “exploitable” if you are willing to put in effort at this level of criminality and cost to yourself.
This doesn’t make any sense. Him getting a payout below his break-even point is the topic at discussion. MIRI was not pursuing a stochastic policy that could have somehow caused him to get 10x the payout.
Literally one of your counter-arguments is “he was unwell in a way that prevented him from working at anywhere near his former capacity” and “severely depressed, something like that”. I don’t see any reason to assume that Louie Helm was pursuing a rational strategy, indeed, I see enormous reasons to assume the opposite.
(Revisiting when my thoughts percolated into more coherent words:) You don’t get to live in the convenient world where most potential blackmailers are people with good alternative options. It’s always going to be an adversarial sample; the people who are most likely to attempt blackmail are those with the worst alternatives and the lowest costs; the ones who have little or no reputation to lose, and poor returns on their effort if it’s directed in other directions. If you can’t deter someone with minimal reputation and low value of their time from blackmailing you, your only defense against blackmail is to prevent anyone meeting that description from having information they could use to blackmail you.
Do you not see how that would also reinforce my point? I don’t even know how to explain that, it’s too obvious to me. (EDIT: I thought of how to explain it and placed it in this new sibling comment.)
Reality was. He didn’t know how much money was available to be extorted, and if it was a lot, it could be quite a lot; heavy-tailed distribution.
This honestly feels like it’s getting silly. I am bowing out.