I think LessWrong and many other things I’ve built are in a confusing place as it relates to this post. At the present my thinking is roughly:
It does seem like overall the things this broader ecosystem has built are not that federalist, and not that defensible, but I sure think I have made things marginally more federalist and marginally more defensible so maybe that means I shouldn’t quit but others should?
Also, IDK, I don’t think LessWrong is that defensible. It’s not like we have formal membership, and things are quite beholden to quite a lot of random memetic drift and it would if anything be more surprising than not for this site to still be roughly aligned with the culture that I am excited about in 10 years.
The track record of “online communities stay aligned with the interests of its founders or head admins” is really very weak, indeed so weak I have trouble thinking of almost any positive examples. I do think I’ve been doing a decent job in the last decade, but that doesn’t buy me that much confidence for the next (especially as things will probably be pretty crazy with AI).
I think you may be overindexing on the George Washington example, where him quitting exemplified a central part of the principles he was advocating.
Ah, that is actually just a false-positive, I really wasn’t actually intending to analogize the George Washington example of quitting to me quitting. Now that you point it out it sure makes sense as a thing someone would read into the post, but I really didn’t actually intend that!
I struggle to understand the following. Since I don’t believe that anyone could have any mission in an ASI-ruled world, the critical period is likely to be at most 5 years, not 10. Additionally, during the critical period I expect LW to stay the most important AI-related forum[1] where researchers exchange insights like Greenblatt’s impression that most AIs are misaligned, Anthropic’s Persona Selection Model or Harms’ CAST. Finally, I think that Wikipedia is an online community which stayed aligned with the interests of its founders or head admins of creating the encyclopedia… until AI came and made the public lose interest in it.
the critical period is likely to be at most 5 years, not 10
Come on. Yes, timelines appear to be on the shorter side, but clearly it would be extreme hubris to stop planning around >5 year timelines! That really seems very dogmatic to me.
My median timeline is ~7 years until truly transformative AI. And I have quite a lot of probability on things longer than that!
On the meta level: why are there this many net upvotes and agreement votes for planning horizons of “at most 5 years”? This updates me towards thinking that some aspect of collective epistemics is notably worse than I had been tracking.
My guess is most of the agree-votes are for the other parts of the comment. It’s always been tricky to disaggregate things like this (which is why we don’t have agree-voting on posts).
Finally, I think that Wikipedia is an online community which stayed aligned with the interests of its founders or head admins of creating the encyclopedia
I strongly disagree! I think Wikipedia lost the way around 10 years ago.
Additionally, during the critical period I expect LW to stay the most important AI-related forum
Correct (probably unless I go and try to actively build a competing forum or shut down LessWrong). Why this concerns me is I think kind of clearly answered in the post.
I think Wikipedia lost the way around 10 years ago.
I think I agree, though I currently believe it continues to be strongly net positive for the world. My current guess is that it will lose its value to LLMs before it starts to be sufficiently politically captured to be net negative. I am interested to know if you think it is already net negative.
I am interested to know if you think it is already net negative.
It is always very hard to tell what the counterfactual of something would be, but my guess is yes, it’s quite good.
(But IDK, I think Wikipedia has been pretty bad for LessWrong in-particular, and I don’t actually have access to all the other communities similarly effected, and possibly there has been large collateral damage that I am blind to)
Could you explain why you believe that it is Wikipedia which was politically captured? I think that the history of the Russian Branch provides some evidence to the contrary (which, alas, is accessible only to Russian-speaking users like @Mikhail Samin). The attempt of pro-Russian and anti-LGBTQ users to politically capture the branch caused lots of conflicts and eventually caused these users to defect to various clones like Runiversalis.
Alas, Wikipedia’s principles require it to rely on external analysts of news, and if someone politically captured the highest-quality media like the BBC, then Wikipedia’s rules would require it to reflect the media’s position.
Returning to LW and its mission, I don’t understand how a change in culture could undermine it except for causing an onslaught of mechinterp-like slop by new users. But this seems to be more like Hitler’s invasions into many countries than actual corruption.
Part of the difficulty in pointing specifically to how and when Wikipedia was politically captured is that this sort of ideological takeover happens in a pretty abstract and diffuse way without much of a paper trail, which is part of what makes it so hard to defend against.
A good recent example I encountered was the Olympics Boxing scandal around Imane Khelif where having an SRY gene was a very salient point that was reported on and later confirmed, but there was a huge effort to conceal that info from the page, and then later downplay it heavily.
That said, I still think Wikipedia is a pretty substantial net good for providing information.
I am not immediately pulling to mind the reasons. I think I recall seeing a graph of Wikipedia edits going down over time (and serious editors leaving on-net) which is a bad sign for the health. It is also not uncommon for me to hear instances of politically motivated edits.
As one example, this week I was told that Maria Montessori—originator of the Montessori school of education—was an avowed and extreme racist, but that people who are involved in Montessor education reliably edit it out of her Wikipedia page (as can be evidenced by its absence on the page, but repeated presence on the talk page).
(I also heard accusation she was a eugenicist, but I failed to find corroboration of that fact while writing this comment.)
Almost everyone that considered themselves scientifically minded and rational at the dawn of the 20th century was an eugenicist, bad genetics was considered a real concern and danger to address. Doesn’t mean they all were straight up Nazis but it was one of the big fashionable positivist beliefs.
To the extent that the effect is as big as you’re making it out to be (surely it’s “big”, IDK the exact magnitude though), this seems to be mostly explainable by people trying to apply their new great toy to everything around them. A man with a hammer sees everything as a nail etc.
Wikipedia is a race condition in a simulationist context guys, Jesus Christ. Do not talk harshly about far branches of the tree until you have settled earlier branches. Not unless you have forensics.
Finally, I think that Wikipedia is an online community which stayed aligned with the interests of its founders or head admins of creating the encyclopedia...
Both of the cofounders recently said that Wikipedia was biased in the Gaza genocide article. In his comment urging for change, Jimmy Wales suggest that people should do things in the Gaza genocide that violate WP:RS/ReliableSources.
WP:RS/ReliableSources is the central policy that changed in the last ten years that reduced the amount of coverages of viewpoints that diverge from left-wing politics get. It’s not the only change in that direction, but the most important one.
TBF I feel like at most if LessWrong really is drifting (it’s hard for me to say as I’m a relatively late joiner) the worst it’s doing is being less interesting to read. I don’t see a lot of harm coming from it right now; there are parts of the rationalist adjacent/derived community that I find reprehensible or even dangerous, but none of them are particularly represented here. If the worst that can happen is that the big thing simply waters itself down and dies with a whimper, that’s probably as good as it gets.
I see, so this is more about quitting LessWrong specifically and not about quitting Lightcone activities more generally?
Yeah, LessWrong is probably one of the best examples honestly, congrats! I think it’s probably still worth trying but of course I don’t have a good picture of what your opportunity and other costs are.
I see, so this is more about quitting LessWrong specifically and not about quitting Lightcone activities more generally?
I think Lighthaven is also not particularly federalist! Other things we do a bit more. I think in general Lightcone is pretty deeply entwined with this whole ecosystem, which maybe doesn’t quite get federalism (I blame consequentialism).
(Also to be clear, I am using the word “federalism” to point to the thing in my post. I think it overlaps with the general meaning of “federalism”, but I am not at all confident of that. My knowledge of federalism the political philosophy is mostly downstream of reading the SEP entry on federalism)
I think LessWrong and many other things I’ve built are in a confusing place as it relates to this post. At the present my thinking is roughly:
Also, IDK, I don’t think LessWrong is that defensible. It’s not like we have formal membership, and things are quite beholden to quite a lot of random memetic drift and it would if anything be more surprising than not for this site to still be roughly aligned with the culture that I am excited about in 10 years.
The track record of “online communities stay aligned with the interests of its founders or head admins” is really very weak, indeed so weak I have trouble thinking of almost any positive examples. I do think I’ve been doing a decent job in the last decade, but that doesn’t buy me that much confidence for the next (especially as things will probably be pretty crazy with AI).
Ah, that is actually just a false-positive, I really wasn’t actually intending to analogize the George Washington example of quitting to me quitting. Now that you point it out it sure makes sense as a thing someone would read into the post, but I really didn’t actually intend that!
I struggle to understand the following. Since I don’t believe that anyone could have any mission in an ASI-ruled world, the critical period is likely to be at most 5 years, not 10. Additionally, during the critical period I expect LW to stay the most important AI-related forum[1] where researchers exchange insights like Greenblatt’s impression that most AIs are misaligned, Anthropic’s Persona Selection Model or Harms’ CAST. Finally, I think that Wikipedia is an online community which stayed aligned with the interests of its founders or head admins of creating the encyclopedia… until AI came and made the public lose interest in it.
The most important other mission of LW is clear philosophy and practical topics like Daycare illnesses.
Come on. Yes, timelines appear to be on the shorter side, but clearly it would be extreme hubris to stop planning around >5 year timelines! That really seems very dogmatic to me.
My median timeline is ~7 years until truly transformative AI. And I have quite a lot of probability on things longer than that!
On the meta level: why are there this many net upvotes and agreement votes for planning horizons of “at most 5 years”? This updates me towards thinking that some aspect of collective epistemics is notably worse than I had been tracking.
My guess is most of the agree-votes are for the other parts of the comment. It’s always been tricky to disaggregate things like this (which is why we don’t have agree-voting on posts).
I strongly disagree! I think Wikipedia lost the way around 10 years ago.
Correct (probably unless I go and try to actively build a competing forum or shut down LessWrong). Why this concerns me is I think kind of clearly answered in the post.
I think I agree, though I currently believe it continues to be strongly net positive for the world. My current guess is that it will lose its value to LLMs before it starts to be sufficiently politically captured to be net negative. I am interested to know if you think it is already net negative.
It is always very hard to tell what the counterfactual of something would be, but my guess is yes, it’s quite good.
(But IDK, I think Wikipedia has been pretty bad for LessWrong in-particular, and I don’t actually have access to all the other communities similarly effected, and possibly there has been large collateral damage that I am blind to)
Could you explain why you believe that it is Wikipedia which was politically captured? I think that the history of the Russian Branch provides some evidence to the contrary (which, alas, is accessible only to Russian-speaking users like @Mikhail Samin). The attempt of pro-Russian and anti-LGBTQ users to politically capture the branch caused lots of conflicts and eventually caused these users to defect to various clones like Runiversalis.
Alas, Wikipedia’s principles require it to rely on external analysts of news, and if someone politically captured the highest-quality media like the BBC, then Wikipedia’s rules would require it to reflect the media’s position.
Returning to LW and its mission, I don’t understand how a change in culture could undermine it except for causing an onslaught of mechinterp-like slop by new users. But this seems to be more like Hitler’s invasions into many countries than actual corruption.
This seems like a nice example of Wikipedia preventing itself from conquering what it cannot defend.
Part of the difficulty in pointing specifically to how and when Wikipedia was politically captured is that this sort of ideological takeover happens in a pretty abstract and diffuse way without much of a paper trail, which is part of what makes it so hard to defend against.
Tracing Woodgrains documents some interesting cases like a major admin/user that consistently edits in bad faith and generally gets away with it or how figures like Mao get treated rather differently from other big dictators.
A good recent example I encountered was the Olympics Boxing scandal around Imane Khelif where having an SRY gene was a very salient point that was reported on and later confirmed, but there was a huge effort to conceal that info from the page, and then later downplay it heavily.
That said, I still think Wikipedia is a pretty substantial net good for providing information.
I am not immediately pulling to mind the reasons. I think I recall seeing a graph of Wikipedia edits going down over time (and serious editors leaving on-net) which is a bad sign for the health. It is also not uncommon for me to hear instances of politically motivated edits.
As one example, this week I was told that Maria Montessori—originator of the Montessori school of education—was an avowed and extreme racist, but that people who are involved in Montessor education reliably edit it out of her Wikipedia page (as can be evidenced by its absence on the page, but repeated presence on the talk page).
(I also heard accusation she was a eugenicist, but I failed to find corroboration of that fact while writing this comment.)
Almost everyone that considered themselves scientifically minded and rational at the dawn of the 20th century was an eugenicist, bad genetics was considered a real concern and danger to address. Doesn’t mean they all were straight up Nazis but it was one of the big fashionable positivist beliefs.
To the extent that the effect is as big as you’re making it out to be (surely it’s “big”, IDK the exact magnitude though), this seems to be mostly explainable by people trying to apply their new great toy to everything around them. A man with a hammer sees everything as a nail etc.
Sure, my main point was “it doesn’t say much about someone of the time other than they were buying into a fad”.
Wikipedia is a race condition in a simulationist context guys, Jesus Christ. Do not talk harshly about far branches of the tree until you have settled earlier branches. Not unless you have forensics.
Both of the cofounders recently said that Wikipedia was biased in the Gaza genocide article. In his comment urging for change, Jimmy Wales suggest that people should do things in the Gaza genocide that violate WP:RS/ReliableSources.
WP:RS/ReliableSources is the central policy that changed in the last ten years that reduced the amount of coverages of viewpoints that diverge from left-wing politics get. It’s not the only change in that direction, but the most important one.
TBF I feel like at most if LessWrong really is drifting (it’s hard for me to say as I’m a relatively late joiner) the worst it’s doing is being less interesting to read. I don’t see a lot of harm coming from it right now; there are parts of the rationalist adjacent/derived community that I find reprehensible or even dangerous, but none of them are particularly represented here. If the worst that can happen is that the big thing simply waters itself down and dies with a whimper, that’s probably as good as it gets.
I see, so this is more about quitting LessWrong specifically and not about quitting Lightcone activities more generally?
Yeah, LessWrong is probably one of the best examples honestly, congrats! I think it’s probably still worth trying but of course I don’t have a good picture of what your opportunity and other costs are.
Ah, gotcha!
I think Lighthaven is also not particularly federalist! Other things we do a bit more. I think in general Lightcone is pretty deeply entwined with this whole ecosystem, which maybe doesn’t quite get federalism (I blame consequentialism).
(Also to be clear, I am using the word “federalism” to point to the thing in my post. I think it overlaps with the general meaning of “federalism”, but I am not at all confident of that. My knowledge of federalism the political philosophy is mostly downstream of reading the SEP entry on federalism)