Running Lightcone Infrastructure, which runs LessWrong and Lighthaven.space. You can reach me at habryka@lesswrong.com.
(I have signed no contracts or agreements whose existence I cannot mention, which I am mentioning here as a canary)
habryka
Karma: 65,020
I see a lot better where you are coming from, now. And certainly there are plenty of Austin’s takes here that I disagree with (e.g., I don’t think we should free SBF and put him to work).
Glad I made at least some sense!
This exchange is making me realize that I don’t really know any of you guys all that well, besides for Caroline, and maybe I shouldn’t have waded into this. But… are you saying Austin would do this without consent? You really think Austin would commit some kind of FTX-style fraudulent scheme where user assets get nonconsensually invested into risky investment vehicles? That just… still doesn’t seem like his apparent personality type, to me, at all. And if that was his plan, Caroline would be the worst possible hire. Not only because of increased scrutiny, but also because I’d expect she herself will be severely allergic to any kind of impropriety in any role she takes on for the rest of her life.
I am saying Austin is in my 90th percentile for people likely to do this. I think this still leaves him unlikely to do this, but yeah, I am tracking it as a risk that seems higher for him than most people.
Caroline would be the worst possible hire
I don’t know Caroline well. In general the history of financial fraud strongly suggests that people who previously committed financial fraud have a much higher likelihood of doing so again. You might have special information about Caroline that I do not have that give you appropriate confidence that this base-rate doesn’t apply to Caroline, but it’s at least not clear to me, from where I am standing.
Of course, that’s just my judgment. And it’s fair if you want to be like, “well, Adam publicly endorsed Sam in 2021, so his judgment sucks.” Nevertheless, I think I am a better judge of character than I was in 2021 (though I admit that I have nothing I can point to to convince anyone here), and I disagree with the consensus view in this thread that 2026-Caroline would be likelier than most to commit fraud in her role at Manifund. I’m happy to throw whatever nonexistent social capital I have here behind that.
I appreciate you sharing your take, I do think it’s valuable evidence. I do discount it a good amount given your mixed track record, but not completely.
One of these cases you should be intimately aware of—we loaned Lightcone 300k so that you could make interest payments for Lighthaven.
Yep, and I was very grateful for that! But I strongly reject the implication that hence I should be more hesitant to be concerned. I aspire to not have being a beneficiary influence my judgement either way, which usually means I will err actively on the side of expressing concerns about things I am a beneficiary of, not the other way around.
To be clear, Manifund should feel totally fine to make risky investments with its own unrestricted funds, and should often feel fine to do risky things with more restricted funds, especially when you insure those with personal funds, or solid external funding guarantees. I don’t know the full provenance of the funds you mentioned, but these investments/actions do not immediately strike me as concerning.
And in general I think your transparency greatly reduces my concerns on this dimension, but not enough to leave me unconcerned.
Oops, I didn’t see this before my response to Adam, which covers a lot of the same ground.
Maybe I’m just misinterpreting your wording, or something. But “Austin has a kind of seriously messed up relationship to fraud” sounds like you’re making dark insinuations about Austin’s moral character. Is that not what you meant?
I mean… sure? I think Austin is like 90th percentile likely among people I know to e.g. end up doing something as dumb as investing Manifund funds earmarked for specific projects into volatile assets and then end up not being able to pay out promised grants. I think this is correlated with his takes on FTX.
Like, in my conversations with Austin over the years his general take has been that FTX mostly didn’t really do anything bad, and wasn’t even convinced they did anything illegal until I think a few months ago, and was very sympathetic to Sam’s takes that basically everything would have been fine and no one should be in jail if people hadn’t convinced him to declare bankruptcy and hand over control of the company over to the bankruptcy receivership (not like totally convinced, but my best memory is that he thought it was like 50% likely or something).
I am here definitely not impugning Austin’s decision to be transparent about hiring Caroline. I think that’s great and to his credit. I just also think that the way Austin is making this decision is entangled with the way I think he views various other aspects of financial responsibility, entrepreneurship, risk-taking, and sociopathy on which I have had strong disagreements with Austin since I’ve known him.
And like, it’s those underlying disagreements that make me very concerned about this decision by Austin. I think if Austin believed different things about these topics that seemed to me more calibrated, then I can imagine it being a good idea for him to hire Caroline. But as is, him hiring Caroline is evidence to me that these issues on which I have long disagreed with Austin about are influencing his actions in concerning ways.
Some of this is publicly reflected in this blog post by Austin and his more recent interview with Elizabeth about the topic:
Whether this blowup was caused by intentional deception or an honest mistake, the EA community has been extremely quick to change its tune. In less than a week, everyone has gone from “SBF, golden boy” to “What a criminal, don’t be like that guy”. EA leaders have posted denunciations of fraud, and distanced themselves from Sam; the most upvoted all-time EA Forum post is a community condemnation; the entire Future Fund team up and resigned.
To be fair, the rest of the world is dogpiling on him too. Elon Musk called Sam “full of shit”; Sequoia deleted their glowing profile of SBF published one month ago; Miami took the FTX name off their arena.
But from EA folks, this behavior strikes me as cowardly, coldhearted, opportunistic, bandwagon-y, two-faced and distasteful. I am extremely confused, because these EA leaders are some of the smartest and “good-est” people in the world, whose work I respect and admire, who have shaped the way I think. So it’s very possible that I’m just in the wrong here, but…
Whatever happened to loyalty? To supporting those who have helped you in the past? Are EA folks only fair-weather friends, happy to accept your money in good times but also ready to eviscerate you to maintain deniability and a glossy PR sheen? What distinguishes altruism from selfishness is “being good to people who cannot help you back”. It is extremely suspicious that EA as a whole was happy to laud praise on SBF while the money was flowing, and then turn their backs as soon as it was clear the gravy train dried up.
SBF:
Is a committed vegan
Earned to give while at Jane Street
Led CEA for a few months
Sent money to Ukrainians in time of need
Incubated hundreds of millions worth of good longtermist causes through Future Fund and related spending on eg Anthropic
Take a step back: what are we assessing here? If the question is “should I associate with a person who has this track record, and also once fraudulently misused customer money, but has repented and is trying as hard as possible to fix it”… the answer seems like a clear yes to me. And as a prosaic consideration, I continue to believe that Sam and the rest of the FTX leadership team are extremely talented and aligned people. Even with tarnished reputations today, I expect them to accomplish good and great things. To jump immediately to cutting ties seems like a large strategic error.
And:
Austin [27:23]: “I don’t know, still debatable… everyone made their money back…”
Austin [27:43]: “My sense is that if SBF hadn’t actually declared bankruptcy, as he says, and held onto the assets and tried to resuscitate FTX exchange and maintain the franchise value there, I think he maybe could have done it.”
[...]
Austin [28:09]: “Some things that I do feel to be true, like right now, is… I think we should free SBF and put him to work.”
[...]
Elizabeth [34:52]: “Won by cheating.”
Austin [34:53]: “I don’t think a lot of it was cheating.”
Austin [34:55]: “I think a lot and a lot of it was actually just doing the actual right things, like hiring good people and being good at fundraising and identifying a good market opportunity.”
[...]
Austin [35:11]: “Somebody who understands the whole story of Alameda better might tell me that, ‘Oh, actually the mixing customer funds was very important, and if not for that FTX would never have won compared to Binance,’ whatever. That just seems like not that true to me…”
I agree it could be possible for someone’s decision to hire Caroline in the way Austin did to be positive evidence about their relationship to e.g. financial responsibility and risk-taking. But given my priors it isn’t. And IDK, this really doesn’t seem like that galaxy-brained of a take. I think Austin’s decision to hire Caroline is very substantially “I think the thing she did is much less bad than other people seem to think, so she has less deep of a hole to dig out of”, and this line of reasoning also straightforwardly extends to SBF.
Honestly, I think it’s slightly more likely than not that Austin would work with SBF himself if he was e.g. given a pardon.
Again, I appreciate Austin’s transparency here, and think that part is to his credit. But I think given the evidence available, his actual decision to hire Caroline is mostly evidence of bad judgement of IMO a very concerning kind.
Absolutely! I think this kind of decision should definitely not be treated with the assumption of innocence.
I absolutely do not think we should consider Austin’s statements about his beliefs, or even confident knowledge of his beliefs, sufficient as justification to make a decision like this seem good. I think any context where this hiring decision would be appropriate it would need to come with substantial evidence of a deeper kind of understanding of the relevant tradeoffs, and a history of acting in the right kind of principled way.
Definitely! I don’t know whether that’s true about her current position and think the correct place to draw the line is a bunch earlier than that.
Huh, I think SBF is among the worst people ever. I think Caroline is at least a candidate for it. It requires a lot of skill to be bad. SBF has caused much more harm for civilization and the future of humanity than prolific serial killers. My guess is he makes the top 100?
I agree with that this is a good prior given appropriate operationalization of “positions of power”, which is importantly not what we are discussing! I also think you are underestimating the opportunities people have for genuine redemption and putting people in positions of conditional power where their strengths are leveraged without needing to extend them much trust in domains where their judgement is weak.
E.g. Caroline ending up in a position of power as a result of becoming a really really good forecaster who lots of people defer to because she has an obviously strong forecasting track record, while people are appropriately hesitant to not give her unconditional power, seems like a fine outcome, and if anything it would be inappropriate for institutions like Metaculus to ban Caroline because of a consequentialist argument about her being publicly displayed as a strong forecaster being bad for the world by empowering her.
In this case, elevating Caroline to a position of power in Manifund will predictably cause other people to incorrectly conclude that our community does not think the FTX fraud was wrong[1]. This is a straightforward cost of the action, it’s legitimate to count it as a cost, and the value above replacement of Caroline as an employee seems unlikely to justify that cost.
Sure, but if we want to play the consequences game, being the kind of person who blocks or prevents promoting Caroline to a position of power in Manifund, even though they know (via some hypothetical means) that it will not make future thing FTX fraud less likely, therefore communicates that we do not care about what things actually cause fraud, it communicates that we care about covering our ass whether that helps with fraud or not.
This is a straightforward cost of the action. It’s legitimate to count it as a cost, and the marginal cost of signaling that, instead of just saying “this is bad because it makes fraud more likely” if and only if we believe that, seems likely to be greater than whatever thing you get by signaling affiliation with some generic “FTX was wrong” coalition beyond what is justified.
Like, I think the real analysis here is complicated. Going up a simulacra level in your analysis of a topic is a major major cost and should not be done lightly. Sometimes it’s the right call, but man, it seems to me like a more straightforward cost than the benefits of playing these signaling games marginally better.
I mostly meant by this “we shouldn’t pretend there is at present already a widely accepted rule to not do this under any circumstance”.
I also think that it’s not that hard to imagine situations where you should do this anyway. Most obviously, you should do it anyway when there are other reasons that are more important for helping the relevant person, or from the outside you can’t tell whether this is the actual reason or you are misunderstanding the nature of the reasoning.
(In this case, I know that Austin’s endorsed position is that he is grateful for the FTX Future Fund not for the money they gave to those projects, but for the good judgement that funding those projects displayed, and I think that’s a valid inference! I don’t fully believe him in this case, and think this is a weak enough defense that I think his reasoning here is extremely suspect, but e.g. I updated positively on Nick Beckstead based on some of the things he did at the FTX Future Fund. I would feel fine about someone hiring him because they liked what he did at FTX Future Fund, even if one of the sources of evidence they are using for that is that he was funding part of your own ecosystem. Again, I think Austin’s specific reasoning here is bad and I disagree with him, but that is a fact contingent on the details of his argument, not a super straightforward universal rule that we can just follow and avoid having to look at the details.)
I think you are underestimating the space of possible employment relationships. I agree with you that hiring Caroline is bad, and many of your inferences here are correct, but it’s not because of some super simple rule that “everyone agrees with”.
I think it’s IMO pretty clearly fine for Caroline to work for an organization in a non-executive, non-financial capacity without much ability to cause harm, with compensation less than she could make on the market even given her employment history, and with little to no social authority. The latter is tricky to arrange, but there are many roles for which it would be true.
Caroline was not given the death penalty, she was sentenced to prison, did her time, and while I do not think people should come to trust her readily again, it seems fine for people to work with her in roles that visibly and transparently do not require trust in her.
Whether it’s possible to hire her into a financial role depends on the details of how it is set up. Caroline working as the equivalent of an open-source contributor and contractor to something like the Manifund platform, making pull requests to a repository where other people reliably review the code, seems like a potential good fit for her skillset, and does not give her much power or social authority.
And beyond that, it’s just generally not the community’s business to judge who organizations hire into commoditized non-trust-requiring roles. Yes, leadership roles and roles with social authority, especially ones that have any risk of being mistaken as representative of our extended ecosystem, deserve scrutiny. But contractor-like roles with well-defined outputs that are relatively commoditized do not. I think Martin Shkreli sucks (though not as much as people think he sucks), but if I want to hire him to set up some Airtable automations for me, and people try to get angry at me about that, I will tell them to pound sand and please tell me any actual mechanism by which that could end up causing harm to anyone, and if they don’t have an answer, to please stop bothering me.
This seems like a much better candidate rule to me. I don’t think we should pretend its universal either, but the arguments for it seem substantially stronger than the rule that Elizabeth appears to be proposing above.
I mean, in both cases people seem to somewhat randomly make up optics-oriented rules that they claim are universal and then shut down conversation about whether they are appropriate with “no, even if you were right right, we have this rule, and this rule is how we act”.
In those cases I said “no, but like, it would be good for the world” and they were like “I think we would seem really aggro to the rest of the world if we just accused random prominent figures in our community of being shady and this seems like a clear professional-culture rule, and so even if it is the right choice in this context, society has this rule, and this rule is good and we are better off for it”.
And like, what’s the next step of this conversation? I don’t think this rule was real, I don’t think this rule was real. I am confident there are ways in which Austin could have handled hiring Caroline that involved handling money to some extend that you would have been fine with, and other people would have been fine with. I just don’t believe that there is this clean hard-line rule that people have buy-in for. You can try to make it a rule within our community but then you have to argue for why it’s good!
Sorry, I was unclear. I mean, in the sense that she isn’t allocating a large budget unilaterally and doesn’t control substantial funds directly.
She can literally send money to people, but only in the sense that like dozens of my employees can initiate ACH transfers and credit card charges, which will of course get caught pretty much immediately in our financial records, and then bad things will happen to them. Manifund, as far as I know, does not at the moment involve discretion over large monetary budgets in a way that would enable large-scale fraud, without buy-in from at least you. If I am wrong about that, then I do think it’s your responsibility to add processes to make that difficult, given Caroline’s history (which is part of what I mean by “it just really depends on how well she is supervised”)
Very strong disagree with this. This rule looks totally post-hoc to me, and not the kind of thing one should pretend to be universal, and indeed, the correct judgement of the Manifund situation seems to me highly dependent on how Caroline was hired, and what responsibilities she gets, and how Austin and Caroline relate to what FTX did, and all of those things.
I am generally really frustrated with people making up random universal rules, like this post, which IMO was among the worst things written in the FTX collapse: https://www.lesswrong.com/posts/8wYH4WggxFqT9yhzJ/we-must-be-very-clear-fraud-in-the-service-of-effective
There is so much conflation in the sentences here. Is Caroline “in charge of large amounts of money”? I mean, no, she probably can’t just send money to whoever she likes without getting caught. Whether she can commit more fraud depends on how well she is supervised. There are ways to employ people in vicinity to large amounts of money that comes with substantially less risk of fraud than working in most other jobs (because of substantial transparency requirements).
I can’t see a clear line between what is happening here and what I consider the primary causes of the FTX fallout. When I wanted to talk openly about Sam being shady, people said “it would look really bad if you publicly accused a major leader of EA of being shady without having ironclad evidence of backing that up, and this seems like a pretty clear rule”. Please, can we stop doing this.
To be clear, I think Austin hiring Caroline is a lot of evidence that Austin has a kind of seriously messed up relationship to fraud, and FTX, and ethical and moral responsibility. I think it’s a mistake. But it’s certainly not a mistake because it looks bad! One of the things I’ve heard from the optics-optimizing EAs is that maybe it would be fine to hire Caroline or other ex-FTX employees, but you should do so quietly and on the down-low. This is where this logic leads you! Why that would be much worse is left as an exercise for the reader, but I think it’s sufficiently clear.
Thank you!
We already serve a specialized API for agents that agents get pointed to when they make a request, and we don’t block requests to our general graphQL API. But the agents don’t try that when they run into problems.
God, we have tried so many things to keep LW accessible to the LLMs, but the bots and crawlers are so aggressive these days that it’s actually genuinely hard. We currently have some bot protections on because without those we go down immediately as we get hammered extremely aggressively and go down within minutes. I will nevertheless prioritize doing better on this next week. There must be some way to at least serve some kind of cached version of content.
I was trying to convey that it’s stronger to operate in our desired frame instead of the opponent’s. Explaining feels like it can slide into a defensive posture.
I think this very instinct is probably at the core of like 50% of the dysfunction of modern discourse. It’s just people repeatedly insisting to have everything play out in their specific frame, and repeatedly trying to redirect everything towards that. I have a similar deep distaste for what people like to call “media training” the central lesson of which is to “not answer any question asked straightforwardly but instead use it as a jumping off point for talking about the things you want to talk about”. “Media trained” people are obnoxious to talk to, and make communication and understanding of an issue reliably worse.
If the goal is public persuasion, I’d redirect to what we want people talking about, ideally going on the offensive
No, this is bad. Talk about what will help people understand the situation better. If you need to redirect, first answer the question that is being asked of you straightforwardly, even if it seems to you in bad faith, then explain something in return that you think is relevant. Conversation, including public conversation, is a give and take of different perspectives.
From my experience, the general public will not necessarily internalize the “correct” response over time, no matter how valid the argument (hence the illusory truth effect, though I’d want to read more research on this because I suspect on a longer time horizon you may be right)
People will definitely never all become world experts, but I do think good public communication requires an intention to help the audience genuinely understand. This is not always possible, but the intention is where approximately all the good things in public discourse come from.
I think there is something true in this, but this comment has enough of a missing mood that I think it will hurt its cause more than it helps. I am finding it kind of hard to point to it, but I expect comments like this to put you out of integrity with the very people you claim to want to not alienate.