I thought this says a useful thing in language other than our own. Notably that often we do not actually want the things we say we want and that can be fine.
Nathan Young
Karma: 2,405
Thomas Schelling’s Nobel Prize Speech: An Astonishing Sixty Years: The Legacy of Hiroshima
Why Prayer Is Not Answered by Marion Zimmer Bradley
Be less open-minded: Against uncalibrated open-mindedness
Without spoilers I’d say that the chapters that feature the work’s characters post utopia were part of the reason I liked it also. This is authoritarian, but it’s a picture of heaven that displays what people actually do in heaven. I think how heaven is translated to different people is moving, but I often find incoherences with heaven when described and this had fewer.
Okay so the aim would be to have everything but the papers themselves?
But wouldn’t academics still publish in the journals and waste huge amounts of time doing so?
I often think journals suck and that someone could obviously do better. But it feels like Lighcone already have most of the pieces. A respected, well-used web platform. Connections to donors. A great conference venue. I find myself confused as to what steps are required to start to seriously take on journals.
I struggle to actually think what the relevant model is here?
Is it about where academics pulbish? Do they want somewhere that will reach a large audience without damaging their reputation?
Is it about what donors care about? So if a donor made known that they would look seriously on highly upvoted LessWrong articles would that start a flywheel?Is it about what gives real world prestige? Could there be a conference for some subset of top posters on LessWrong in the last year?
I guess I’m hilariously wrong somehow, but what is going on. If someone wanted to displace journals what would the next step be?
This video of Jackie Chan directing an action scene:
https://x.com/kungfupete/status/2059686222424072569?s=20
What service did you use for this?
I anecdotally had the sense that this was a deeply unpopular opinion; certainly many of the people I talked about these results with thought it would be deeply unpopular.
Comment to post your guesses.
I thought 10% yes, 10% no, 80% not sure.
Billy Perrigo
A suggested update after discussing The Darwinian Honeymoon—Why I am not as impressed by human progress as I used to be by @Elias Schmied
There are two seperate frames around human progress and alignment:
1. What level of minimum human competence does building AGI require?
2. What level of mimum human competenece does AGI alignment require?If the 2 is entirely inside 1, then AGI is likely to be easy. If 2 is large and outside 1, then AGI alignment is likely to be hard.
In this sence there is some base level of difficulty of the universe relating to 1 and 2.
And technological progress can there fore go under 1 or 2. And technology that is part of 1 isn’t evidence for 2, because any AGI in that universe would require that technology.
Type 1 techology:
- Semiconductors
- Coding languages
- LLMsType 2 technologies:
- Nuclear treaties — coordination on catastrophic-risk technology
- Prediction markets — institutional epistemics about uncertain futures
- Animal welfare progress — moral circle expansion to non-reciprocating entities
- Democracy (ambiguous) — constraints on concentrated powerKind of hand wavey, but I think this suggested that some technologies may be evidence for alignment being easier than the universe’s difficulty, and some technologies may just be part of the package needed for AGI in this universe and hence not evidence for alignment being easier than it might have otherwise been.
I think my outside view is something more like: humans tend to solve problems in the interests of humans. That doesn’t feel that damaged by this piece.
Rapid comment:
I agree a bit and think animal welfare (and AI) are a key area to criticise capitalism on.
But I also think that chicken welfare is likely to improve.
I don’t work primarily in prediction markets/forecasting. I work trying to build AI community notes. The forecasting work (maybe like 5-10hrs a week) I do is paid at a rate I am happy to work for it.
[COI: I work at the Swift Centre as a forecaster, I have worked for a prediction market, I am very involved in forecasting. It is not my current work however, which is on community notes]
A few things points attempting to say things other commenters haven’t, though I largely agree with the critical comments and the things they agree with Marcus on:
I agree that the $100M doesn’t seem super well allocated. Not because forecasting is useless, but because the money flowed to big institutions and platforms rather than smaller, weirder, mechanism-design bets. I like Metaculus, but it has absorbed a lot of money in the last 5 years and not clearly changed much. I don’t know if I think FRI has been worth it, I am glad someone has done the research but, again, how much are we talking? I would have preferred smaller projects were funded on the margin. Coefficient’s strategy in forecasting has felt poor to me, often ignoring the community who in my view come up with the most interesting projects and going for marginal spending on incumbents.
Nobody funds mechanism design or institutional epistemics. I recently spoke to someone at a household name enormous tech company who described their institutional process. It was almost unbelievably dysfunctional to me. Who is funding the work to help institutions think better? It doesn’t promise near-term wins and frankly should’t be the priority of any non-research org. So basically no one. Forecasting is an attempt. How much value is there in the joint stock company, or in democracy. To me, that’s what we are talking about. Figuring out fundamentally better ways of making decisions. It is a problem at scale, it is neglected, and given the deregulation of prediction markets, tractable (though maybe bad, more later).
On “feels useful when it isn’t” (point 6). I don’t entirely disagree. I deliberately try not to spend time forecasting unless I’m being paid to. It can be a distraction. Where I disagree is that some forecasting is genuinely mentally sharpening, at least for me as a thinking discipline. And I think it’s a not unreasonable status hierarchy. Do I endorse the status that Peter Wildeford or Eli Leifland have gotten from forecasting? Yes. Frankly, who do I not endorse having got status from being a forecaster?
Why don’t AI 2027 and Ajeya count? Tangible forecasting outputs that demonstrably moved discourse and decision-making. Why don’t these count as valuable forecasting outputs. AI 2027 is clearly informed by judgemental forecasters and was read by (I think) the Vice President. Habryka said something like ‘too much time has been wasted down the resolution criteria mines’ and I disagree but even if one agrees, I’m not sure even he thinks the whole field is a waste of time.Prediction markets may be net-harmful, but not useless. I’ve said publicly I’m less sure PMs are net-positive — bankruptcies and intimate partner violence are real and huge problems that may be as large as any coordinaton benefits. But ‘bad on net’ and ‘useless’ are different claims, and the later seems more obviously incorrect to me. I would be more interested in a post entitled “EA forecasting efforts have caused massive harm”.
Scott Alexander
Sure so disagree vote, I think he’s pretty clearly a journalist.
Zvi Mowshowitz
I agree. But I think the first part is covered pretty well.