Unnamed

Karma: 7,883

Unnamed 14 Oct 2025 22:27 UTC
3 points
0
in reply to: Unnamed’s comment on: Experiment: Test your priors on Bernoulli processes.
Explanation:
Hypothesis 1: The data are generated by a beta-binomial distribution, where first a probability x is drawn from a beta(a,b) distribution, and then 5 experiments are run using that probability x. I had my coding assistant write code to solve for the a,b that best fit the observed data and show the resulting distribution for that a,b. It gave (a,b) = (0.6032,0.6040) and a distribution that was close but still meaningfully off given the million experiment sample size (most notably, only .156 of draws from this model had 2 R’s compared with the observed .162).
Hypothesis 2: With probability c the data points were drawn from a beta-binomial distribution, and with probability 1-c the experiment instead used p=0.5. This came to mind as a simple process that would result in more experiments with exactly 2 R’s out of 4. With my coding assistant writing the code to solve for the 3 parameters a,b,c, this model came extremely close to the observed data—the largest error was .0003 and the difference was not statistically significant. This gave (a,b,c) = (0.5220,0.5227,0.9237).
I could have stopped there, since the fit was good enough so that anything else I’d do would probably only differ in its predictions after a few decimal places, but instead I went on to Hypothesis 3: the beta distribution is symmetric with a=b, so the probability is 0.5 with probability 1-c and drawn from beta(a,a) with probability c. I solved for a,c with more sigfigs than my previous code used (saving the rounding till the end), and found that it was not statistically significantly worse than the asymmetric beta from Hypothesis 2. I decided to go with this one because on priors a symmetric distribution is more likely than an asymmetric distribution that is extremely close to being symmetric. Final result: draw from a beta(0.5223485278, 0.5223485278) distribution with probability 0.9237184759 and use p=0.5 with probability 0.0762815241. This yields the above conditional probabilities out to 6 digits.

Unnamed 13 Oct 2025 23:10 UTC
15 points
3
on: OpenAI #15: More on OpenAI’s Paranoid Lawfare Against Advocates of SB 53
Chris Lehane, the inventor of the original term ‘vast right wing conspiracy’ back in the 1990s to dismiss the (true) allegations against Bill Clinton by Monica Lewinsky
This is inaccurate in a few ways.
Lehane did not invent the term “vast right wing conspiracy”, AFAICT; Hillary Clinton was the first person to use that phrase in reference to criticisms of the Clintons, in a 1998 interview. Some sources (including Lehane’s Wikipedia page) attribute the term to Lehane’s 1995 memo Communication Stream of Conspiracy Commerce, but I searched the memo for that phrase and it does not appear there. Lehane’s Wikipedia page cites (and apparently misreads) this SFGate article, which discusses Lehane’s memo in connection with Clinton’s quote but does not actually attribute the phrase to Lehane.
The memo’s use of the term “conspiracy” was about how the right spread conspiracy theories about the Clintons, not about how the right was engaged in a conspiracy against the Clintons. Its primary example involved claims about Vince Foster which it (like present-day Wikipedia) described as “conspiracy theories” (as you can see by searching the memo for the string “conspirac”).
Also, Lehane’s memo was published in July 1995 which was before the Clinton-Lewinsky sexual relationship began (Nov 1995), and so obviously wasn’t a response to allegations about that relationship.
Lehane’s memo did include some negatives stories about the Clintons that turned out to be accurate, such as the Gennifer Flowers allegations. So there is some legitimate criticism about Lehane’s memo, including how it presented all of these negative stories as part of a pipeline for spreading unreliable allegations about the Clintons, and didn’t take seriously the possibility that they might be accurate. But it doesn’t look like his work was mainly focused on dismissing true allegations.

Unnamed 13 Oct 2025 8:02 UTC
2 points
1
on: The Most Common Bad Argument In These Parts
Exhaustive Free Association is a step in a chain of reasoning where the logic goes “It’s not A, it’s not B, it’s not C, it’s not D, and I can’t think of any more things it could be!”^[1] Once you spot it, you notice it all the damn time.
This description skips over the fallacy part of the fallacy. On its own, the sentence in quotes sounds like a potentially productive contribution to a discussion.

Unnamed 13 Oct 2025 6:33 UTC
3 points
1
on: Experiment: Test your priors on Bernoulli processes.
[0.111019, 0.324513, 0.5, 0.675487, 0.888981]

Unnamed 5 Oct 2025 22:20 UTC
21 points
18
on: The Counterfactual Quiet AGI Timeline
My leading guess is that a world without Yudkowsky, Bostrom, or any direct replacement looks a lot more similar to our actual world, at least by 2025. Perhaps: the exact individuals and organizations (and corporate structures) leading the way are different, progress is a bit behind where it is in our world (perhaps by 6 months to a year at this point), there is less attention to the possibility of doom and less focus on alignment work.
One thing that Yudkowsky et al. did is to bring more attention to the possibility of superintelligence and what it might mean, especially among the sort of techy people who could play a role in advancing ML/AI. But without them, the possibility of thinking machines was already a standard topic in intro philosophy classes, the Turing test was widely known, Deep Blue was a major cultural event, AI and robot takeover were standard topics in sci-fi, Moore’s law was widely known, people like Kurzweil and Moravec were projecting when computers would pass human capability levels, various people were trying to do what they could with the tech that they had. A lot of AI stuff was in the groundwater, especially for the sort of techy people who could play a role in advancing ML/AI. So in nearby counterfactual worlds, as there are advances in neural nets they still have ideas like trying to get these new & improved computers to be better than humans at Go, or to be much better chatbots.
Yudkowsky was also involved in networking, e.g. helping connect founders & funders. But that seems like a kind of catalyst role that speeds up the overall process slightly, rather than summoning it where it otherwise would be absent. The specific reactions that he catalyzed might not have happened without him, but it’s the sort of thing where many people were pursuing similar opportunities and so the counterfactual involves some other combination of people doing something similar, perhaps a bit later or a bit less well.

Unnamed 15 Sep 2025 21:15 UTC
5 points
0
in reply to: Unnamed’s comment on: High-level actions don’t screen off intent
e.g., Betty could cause one more girl to have a mentor either by volunteering as a Big Sister or by donating money to the Big Sisters program.
In the case where she volunteers and mentors the girl directly, it takes lots of bits to describe her influence on the girl being mentored. If you try to stick to the actions->consequences framework for understanding her influence, then Betty (like a gamer) is engaging in hundreds of actions per minute in her interactions with the girl—body language, word choice, tone of voice, timing, etc. What the girl gets out of the mentoring may not depend on every single one of these actions but it probably does depend on patterns in these micro-actions. So it seems more natural to think about Betty’s fine-grained influence on the girl she’s mentoring in terms of Betty’s personality, motivations, etc., and how well she and the girl she’s mentoring click, rather than exclusively trying to track how that’s mediated by specific actions. If you wanted to know how the mentoring will go for the girl, you’d probably have questions about those sorts of things—“What is Betty like?”, “How is she with kids?”, etc.
In the case where Betty donates the money, the girl being mentored will still experience the mentoring in full detail, but most of those details won’t be coming directly from Betty so Betty’s main role is describable with just a few bits (gave $X which allowed them to recruit & support one more Big Sister). e.g., For the specific girl who got a mentor thanks to Betty’s donation, it probably doesn’t make any difference what facial expression Betty was making as she clicked the “donate” button, or whether she’s kind or bitter at the world. Though there are still some indirect paths to Betty influencing fine-grained details for girls who receive Big Sisters mentoring, as the post notes, since the organization could change its operations to try to appeal to potential donors like Betty.

Unnamed 15 Sep 2025 21:13 UTC
4 points
0
on: High-level actions don’t screen off intent
I think of this as coarse-grained influence vs. fine-grained influence, which basically comes down to how many bits are needed to specify the nature of the influence.

Unnamed 31 Aug 2025 2:37 UTC
5 points
3
on: Epistemic advantages of working as a moderate
I think there is a fair amount of overlap between the epistemic advantages of being a moderate (seeking incremental change from AI companies) and the epistemic disadvantages.
Many of the epistemic advantages come from being more grounded or having tighter feedback loops. If you’re trying to do the moderate reformer thing, you need to justify yourself to well-informed people who work at AI companies, you’ll get pushback from them, you’re trying to get through to them.
But those feedback loops are with that reality as interpreted by people at AI companies. So, to some degree, your thinking will get shaped to resemble their thinking. Those feedback loops will guide you towards relying on assumptions that they see as not requiring justification, using framings that resonate with them, accepting constraints that they see as binding, etc. Which will tend to lead to seeing the problem and the landscape from something more like their perspective, sharing their biases & blindspots, etc.

Unnamed 12 Aug 2025 19:11 UTC
2 points
0
in reply to: Unnamed’s comment on: Negative utilitarianism is more intuitive than you think
Instead, the intuitions at play here are mainly about how to set up networks of coordination between agents. This includes
Voluntary interactions: limiting interaction with those who didn’t consent, limiting effects (especially negative effects) on those who didn’t opt in (e.g. Mill’s harm principle)
Social roles: interacting with people in a particular capacity rather than needing to consider the person in all of their humanity & complexity
Boundaries/membranes: limiting what aspects of an agent and its inner workings others have access to or can influence

Unnamed 12 Aug 2025 7:39 UTC
2 points
6
on: Negative utilitarianism is more intuitive than you think
Seems like a mistake to write these intuitions into your axiology.

Unnamed 30 Jul 2025 19:42 UTC
7 points
0
on: My Empathy Is Rarely Kind
Sounds like this agency thing is so central to how you orient to the world that you’re bringing it with you when you try to empathize with someone (at least if they’re human).
If you want to try to do something more like the sort of empathizing that other people talk about, you could try:
1. Imagining some other scenario where the way that you would relate to that other scenario matches the way that this person is relating to the scenario that they’re in.
2. Finding a case where someone is showing some of the agency thing, in a context where you wouldn’t, and empathize with them in that scenario. (This can also help with doing 1, if you also notice how you relate to this context where you wouldn’t show that agency thing, because it adds to your collection of ‘other scenarios’ which could match how someone else engages with a scenario where you would have more agency.)
3. Empathizing with people in contexts that are orthogonal to the agency thing, where how much agency a person is showing isn’t central to what’s happening. (Perhaps someone liking a book/movie/hobby/etc. that you don’t like?)

Unnamed 22 Jul 2025 18:41 UTC
4 points
0
on: Generalized Hangriness: A Standard Rationalist Stance Toward Emotions
It’s standard for parents to have these sorts of models about their kids’ emotions, e.g. “She’s cranky because she didn’t get her nap.”

Unnamed 22 Jul 2025 18:32 UTC
4 points
0
on: Generalized Hangriness: A Standard Rationalist Stance Toward Emotions
Related: Feeling Rational

Unnamed 22 Jul 2025 18:31 UTC
11 points
2
on: Generalized Hangriness: A Standard Rationalist Stance Toward Emotions
I’ve seen hangriness-style advice circulating on twitter (via Zvi, so perhaps in the rationalist milieu) and tiktok (not in the rationalist milieu afaict).
If you feel like you hate everyone, eat
If you feel like everyone hates you, sleep
If you feel like you hate yourself, shower
If you feel like everyone hates everyone, go outside
If you feel overwhelmed by your thoughts, write them down
If you feel lost and alone, call a friend
If you feel stuck in the past, plan for the future
...

Unnamed 10 Jul 2025 23:27 UTC
21 points
9
on: Lessons from the Iraq War for AI policy
The Iraq war & surrounding debate was a formative political experience for me, so I have lots of thoughts/memories/opinions about it.
I agree with the basic account: shocking events (especially 9/11) gave a source of power (the White House) a lot of freedom of action, they made some bad choices about to do with that freedom of action (in part influenced by the shocking events), and other potential sources of power (e.g. US public opinion, the Senate, the UN) which could have interfered were disinclined to (which is basically what it means for one source of power to have a lot of freedom of action).
So probably the most important takeaway is that it’s important who is in positions of power which might get lots of freedom of action depending on what events take place, what ideas & ability to execute effectively they have, and what views they will have after the key events have happened (as they have reactions to those events and reason to focus on the topic, pay attention to others’ views on it, and think about it themselves).
To go into more detail:
The national mood in the US after 9/11 (and then the DC snipers and anthrax attacks) was that the world is a dangerous place, and we need to rally together to keep ourselves safe, and trust the president to do what’s necessary to keep us safe. 9/11 was the big traumatic event, but the set of 3 attacks made a trend and heightened the sense that future attacks could happen to anyone anywhere at any time (as the DC snipers did) and could be technologically sophisticated (as the anthrax attacks were) and kill lots of people. That gave the Bush administration a ton of freedom of action to choose what to do, especially for things like starting wars to keep America safe from terrorism.
Why did George W. Bush choose war? My sense (in line with this article) is that Bush was a ‘big picture’ kind of guy who wasn’t inclined to think through the nuts & bolts of ‘what is this going to accomplish’ and ‘how else might this course of action turn out, besides going the way we want it to’. He tended to latch onto a broad narrative, and in the wake of 9/11 a narrative he found appealing was of himself as a wartime leader, fighting for freedom & democracy against terror & bad guys. The Afghanistan war on its own wasn’t enough to support this narrative so they were looking for other things to do, and Saddam Hussein was the obvious next bad guy sitting around, someone who we had unfinished business with. That Scholar’s Stage article claims (and I find it plausible) that there wasn’t actually much focused discussion within the administration about whether it was a good idea to go to war with Iraq.
The faction within the administration that had already been pushing for war with Iraq was a big part of what made Iraq the obvious next target, but that wasn’t the whole story. As the Scholar’s Stage article discusses, some people pushing for the war in 2002 had been part of the previous Iraq war in 1991, where they’d decided that it was imprudent to remove Saddam Hussein from power, and hadn’t openly changed their minds and been pushing for war in the interim.
Two of the main potential barriers to the White House’s freedom of action in choosing to go to war in Iraq were getting approval from Congress and the United Nations. I think that the zeitgeist of trusting the president to keep us safe played a huge role in getting Congress’s approval, and a weaker variant of that zeitgeist combined with American power helped with (effectively) getting UN approval. It also helped that both happened before the weapons inspectors were allowed back into Iraq. The case for war looked weaker after the weapons inspectors were allowed in and didn’t find signs of the alleged WMD programs, but by that point there was a lot of momentum towards war and the Bush administration had already gotten the authority they needed. France had pushed back on giving that authority through the UN—they wanted a two-resolution plan where first the UN would demand that Iraq let weapons inspectors in, and then after the weapons inspections the UN would reconsider whether to authorize war—and wound up reaching a compromise with the US where they made the wording of the first UN resolution ambiguous so that the US could say that the resolution authorized the war without requiring a second resolution and France could say that they hadn’t supported that. (This is the dispute that led to french fries being called “freedom fries” and the phrase “cheese-eating surrender monkeys” surging in popularity.)
I also have a lot of thoughts about the Iraq war debate, since that is what I was most immersed in at the time, though in hindsight a lot of that seems mostly epiphenomenal. This is more of a thought dump & less focused, but I’ll share anyways since some of the details & texture seem relevant:
The Bush administration’s public case for the war was more like a marketing campaign than part of a public debate over what to do. They talked a lot about how bad Saddam Hussein was (often citing stuff he’d done in the 1980s); this could be used as part of a coherent argument (and therefore he is likely to do such-and-such) but often it was just presented as an emotionally resonant argument fragment—a suggestive fact which felt like it fit with being in favor of a war to remove him from power. There were lots of mentions of 9/11 and terrorism in the context of talking about Iraq & Saddam Hussein; they weren’t explicitly claiming that Hussein had contributed to the 9/11 attack but they did manage to leave a majority of the US believing that he had been involved in 9/11 according to polls at the time. There was also a lot of talk about WMDs, some of it referencing intelligence that turned out to be inaccurate about Iraq’s weapons programs (especially chemical & biological), but much of it framed more like ‘it would be bad if Saddam Hussein got nuclear weapons’ (but rhetorically punchier) or using sleight of hand with the term “WMD” to basically say ‘we have evidence that Iraq has banned chemical weapons programs, and it would be bad if they had nukes’.
There was something more like an actual debate over whether it was a good idea to go to war with Iraq (the Slate dialogue archived here, beginning with this article, provides a pretty good snapshot of the higher quality portion of that debate). That debate took place on a slanted playing field where the pro-war side had the mantle of the zeitgeist on their side, and the anti-war side was generally on the defensive. e.g., There was often an implicit or explicit question of whether opponents of the war were apologists for Saddam Hussein, and so people making an argument against the war might head it off by including something about how they agree that Saddam Hussein is a bad person who has done some awful things. Whereas it was rarer for war proponents to include something about how Saddam Hussein hadn’t been involved in 9/11, or to get pushback for not including that (though it did sometimes happen, especially in higher quality debates or discussions among people who were left-of-center or more distrusting of the Bush administration).
There were a wide range of pro-war cases, including (more on the left) “liberal internationalists” who wanted there to be more constitutional liberal democracies and thought that toppling a bad government and replacing it with a good one was a feasible and good thing to do, and might spread to more of the middle east and make the world a safer place. Others (more on the right) wanted the US to kick some ass so that people/countries would be afraid of getting America mad at them, and that this would help keep us safe. Or, more viscerally & less strategically, some just wanted to get back at people like the ones who had attacked us. There was some tension between these views, and not much effort to create a single coherent view out of them; they were all welcomed into the pro-war coalition.
I think part of the reason that support for the war faded gradually over the years, rather than evaporating during the first year of the war when it became apparent that Iraq didn’t have meaningful WMD programs (or before the war when the weapons inspections didn’t find anything) is that there were so many different cases for war. Partly this is because there were multiple coherent views on the point of the war that got discredited at different times—getting bogged down in a seemingly endless quagmire eventually undermined the “we’ll kick ass so hard that others will be afraid to cross us because they might be next”, and the developments of the new Iraqi government eventually undermined the liberal internationalist case, but those took longer than the WMD info. But I think a bigger part was that the pro-war attitude was more like vibes and side-taking rather than specific arguments about a concrete purpose for the war—see the post-9/11 rallying around the flag, the backlash against France, the “you’re with us or your against us” talk & attitude, etc. Much of the debate & discussion over the war was more like this kind of side-taking than like arguments over whether we should go to war. Another example: saying that the war wouldn’t succeed at accomplishing its proponents goals was often treated as being “defeatist” and felt to some degree like it was siding against the war and against the US. To some degree this side-taking intensified after the start of the war, because then the pro-war side was the American side.

Unnamed 21 Jun 2025 21:31 UTC
3 points
0
on: the silk pajamas effect
I think that the 2 main factors behind the age distribution in professional sports are:
1. Athleticism declines with age and athleticism is a major contributor to being good at sports. You can see this in the pattern of which players do stick around into their 30s—they are often players who have a role that is less dependent on athleticism (e.g. in the NFL kickers, punters, quarterbacks, and long snappers, in the NBA very tall big men) or star players who were much better than baseline in their prime so they continued to be useful even when they weren’t as good as they used to be. (Injuries are a component of this—one way that a person’s athleticism can decline is if they get an injury that they never fully recover from; also older people tend to recover more slowly from injuries and playing through a nagging injury is another way of having reduced athleticism.)
2. Option value for young players: Young players can get a chance on a professional franchise when there’s uncertainty about how good they are or will become, based on the possibility that they will turn out to be good enough to be useful. Players who are out of the NBA or NFL in 2 years are usually players who were never quite good enough to play at that level, but who got a brief chance to be on a roster (or multiple rosters) based on the hope that they were before it became clear that they weren’t good enough.

Unnamed 9 Jun 2025 0:47 UTC
8 points
0
in reply to: Kaj_Sotala’s comment on: Kaj’s shortform feed
Seems misleading.

”Shared Environment” measures to what extent children raised in the same household wind up more similar than children raised in different households. If tailoring your parenting approach to each child helps children develop more agency, happiness, etc., and some households have parents that do this more/better than others, then it would show up as a Shared Environment effect on measures related to agency, happiness, etc.

Unnamed 23 May 2025 19:07 UTC
2 points
0
on: D&D.Sci: The Choosing Ones
Did a little data exploration and then tried
k nearest neighbors. That seemed good enough to provide answers, though it didn’t provide deep understanding. I’ll pick
candidate 11, and for the bonus questions
Holly has the best ratings in isolation, Colleen is a redundant copycat and Amy is noise, and as backups I like 7 and then 19.

Unnamed 20 May 2025 21:18 UTC
4 points
0
in reply to: Thomas Kwa’s comment on: Thomas Kwa’s Shortform
The ‘regression to the mean’ pattern is striking: domains with a lower starting point have been growing faster, and those that started with a longer horizon have mostly been grower slower.
I wonder if that pattern of catchup growth & drag on leaders will mostly hold up over more time and with a larger set of task types.

Unnamed 20 May 2025 18:35 UTC
5 points
0
in reply to: Warty’s comment on: Warty’s Shortform
That seems like an instance of a general story for why markets are good: if something is priced too low people can buy it up and make a profit. It’s a not very impressive way for markets to be impressive.
If you’d said “not surprising” instead of “not impressive” then maybe I would’ve been on board. It’s not that surprising that prediction markets are good at calibration because we already knew that markets are good at that sort of thing. That seems basically true, for certain groups of “we”. Though my attitude is still more check it out: it works like we thought it would rather than nothing to see here, this is just what we expected.