Dumbledore's Army

Heroic re­spon­si­bil­ity is morally neutral

• Dumbledore's Army 6 Nov 2025 11:58 UTC

  5 points

  2

  on: Heroic Responsibility

  As several commenters here have said, the business owner example isn’t a great fit for heroic responsibility. The core is taking responsibility for things that aren’t your job, that you are not socially expected to be responsible for, because you have decided that the thing needs to be done.

  The archetypal fictional example is the hero who raises the rebellion that overthrows the Evil Empire. A normal sensible peasant whose home has just been burned doesn’t do that, he just tries to survive the winter. The hero decides to do more than that, even though it’s not their job and they’re not (at the start of the book/​movie/​whatever) distinguished from all the other thousands of people in the same position. That choice is why we call them the hero.

  Or, for a real-life example, a factory-owner in 1940s Germany might reasonably feel that he has quite enough to do trying to keep his business running in a war zone. It’s not his job to save Jewish lives, they’re not his relatives or anything. And even trying to save them could be very hazardous. Nevertheless, Oskar Schindler decided he was going save as many lives as he could anyway, and saved about 1,200 people. That’s what heroic responsibility looks like.

  And, the most obvious example to anyone on LessWrong: there is no social expectation that a normal citizen who is not a senior government official will decide that it’s up to him to save the world from the risk that Unfriendly AI kills us all, but Eliezer Yudkowsky saw that no one else was going to do the job, and he wants the world to be saved, so he’s spent his career trying.

  The heroic responsibility described by OP, the determination to focus on whether [goal] is actually happening or whether people are buck-passing or symbolically pretending to do the thing, or whatever other failure mode—all that is downstream of the heroic decision to take responsibility for [goal] in the first place.

• Dumbledore's Army 15 Sep 2025 8:59 UTC

  9 points

  4

  on: Does My Ap­pear­ance Pri­mar­ily Mat­ter for a Ro­man­tic Part­ner?

  I agree with others here that appearance is regrettably important in general.

  You ask about looking good for a romantic partner. I suggest you should additionally think about screening effects. Remember that, as a generalisation, the typical woman faces a different problem than the typical man when looking for a partner: the woman has to screen through a large number of offers, many/​most of which are low-quality, to find a partner she wants. People in general respond to screening problems with shortcuts and heuristics. A person who knows you well will consider your appearance as one aspect of what they know about you; a person who has just met you is likely to make a snap judgement on your appearance. Your aim should be for an appearance good enough that it is not making most potential partners screen you out at the first step.

• Dumbledore's Army 29 Aug 2025 7:20 UTC

  4 points

  0

  in reply to: bfinn’s comment on: The Worst Form Of Govern­ment (Ex­cept For Every­thing Else We’ve Tried)

  I can’t think of such a term, and AI’s best effort is ultracrepidarianism, which isn’t quite the same, and which doesn’t roll off the tongue. Unless someone else has a better idea, I suggest ‘misgeneralising’. You’re right that it’s a relatively common bias and needs a name.

Stop and check! The parable of the prince and the dog

• Dumbledore's Army 18 Apr 2025 8:36 UTC

  2 points

  0

  on: The Rus­sell Con­ju­ga­tion Illuminator

  Thank you for creating this.

• Dumbledore's Army 8 Jan 2025 21:14 UTC

  1 point

  0

  in reply to: espoire’s comment on: The On­line Sports Gam­bling Ex­per­i­ment Has Failed

  Thanks for the comment, and I agree that it would be helpful if the debate on this topic could focus on the question of ‘how do we encourage people to take up a less harmful addiction’—like vaping vs smoking—rather than typically jumping to the question of ban /​ don’t ban.

• Dumbledore's Army 8 Jan 2025 21:11 UTC

  4 points

  0

  in reply to: alexey’s comment on: The On­line Sports Gam­bling Ex­per­i­ment Has Failed

  “Except whatever they got addicted to before the legalization of online sports betting, it apparently led to much lower bankruptcy rates etc.”

  Yes, Zvi gives evidence on bankruptcy rates. However, that is not the only kind of harm. Sports gambling doesn’t have direct health effects the way that drug or alcohol addictions do. Different types of addictions hit relationships, and sports gambling isn’t good for relationships, but it’s plausibly less harmful than a porn addiction. Sports gambling leaves people functional and able to hold down a job, again in a way that not all other addictions do. I don’t think you can assert that banning sports gambling will make people ‘much less harmed’. Not unless you’ve done a deep dive into all the different forms of harm caused by different types of addiction which they might get instead. If someone like Scott Alexander did that kind of analysis and announced that banning sports gambling is still worthwhile because people will addict themselves to cannabis or porn and that is net better, that would be different, but so far as I know, no one has tried to answer the question.

  Separate point: you also can’t make the jump from ‘sports gambling is harmful’ to ‘we should ban sports gambling’. Banning things also causes harm. It moderately reduces takeup, but doesn’t eliminate the thing banned—illegal gambling is a problem everywhere people ban it, Prohibition didn’t eliminate alcohol, and the War on Drugs didn’t magically stop people taking them. So banning things causes a moderate reduction in usage of the thing, with an empirical question mark over how much. Banning things also means criminalising people who would otherwise not be criminalised, ruining their lives. It means devoting more societal resources to police and prisons, to enforce the ban, or else it means diverting existing law enforcement resources away from dealing with other crimes. Even if you accept both that sports gambling is harmful to users (more harmful than whatever other thing they’d get addicted to instead) and that banning it would somewhat reduce usage of gambling, you still can’t come to a firm conclusion that banning it is net beneficial to society.

  On Zvi’s levels of legality, I’m OK with the idea of making sports gambling a 2, ie legal with modest frictions, but I really don’t believe that society can justify going to level 3 and criminalising it. And, again, the whole should-we-ban-it debate is a distraction that doesn’t help with the broader question of how society can treat people with addictive personality disorder.

• Dumbledore's Army 19 Dec 2024 17:02 UTC

  7 points

  1

  in reply to: Julian Bradshaw’s comment on: Dark For­est Theories

  Hard disagree to point 1. The fact that humanity hasn’t tried to hide is not counter-evidence to the Dark Forest theory. If the Dark Forest is correct, the prediction is that all non-hiding civilisations will be destroyed. We don’t see anyone else out there, not because every civilisation decided to hide, but because only hiders survived.

  To be clear: the prediction of the Dark Forest theory is that if humanity keeps being louder and noisier, we will at some point come to the attention of an elder civilisation and be destroyed. I don’t know what probability to put on this theory being correct. I doubt it ranks higher than AI in terms of existential risk.

  I do know that ‘we haven’t been destroyed yet, barely 100 years after inventing radio’ is only evidence that there are no ultra-hostile civilisations within 50 light-years which also have the capability to detect even the very weakest radio signals from an antique Marconi. It is not evidence that we won’t be destroyed in future when signals reach more distant civs and/​or we make more noticeable signals.

• Dumbledore's Army 14 Dec 2024 10:51 UTC

  9 points

  −20

  on: The On­line Sports Gam­bling Ex­per­i­ment Has Failed

  A parable to elucidate my disagreement with parts of Zvi’s conclusion:

  Jenny is a teenager in a boarding school. She starts cutting herself using razors. The school principal bans razors. Now all the other kids can’t shave and have to grow beards (if male) and have hairy armpits. Jenny switches to self-harming with scissors. The school principal bans scissors. Now every time the students receive a package they have to tear it open with their bare hands, and anyone physically weak or disabled has to go begging for someone to help them. Jenny smashes a mirror into glass shards, and self-harms with those. The principal bans mirrors...

  Any sane adult here would be screaming at the principal. “No! Stop banning stuff! Get Jenny some psychological treatment!”

  Yes, I know the parable is not an exact match to sports betting, but it feels like a similar dynamic. There are some people with addictive personality disorder and they will harm themselves by using a product that the majority of the population can enjoy responsibly^[1]. (Per a comment above, 39% of the US population bet online, of whom only a tiny fraction will have a gambling problem.) The product might be gambling, alcohol, online porn, cannabis, or something else. New such products may be invented in future. But, crucially, if one product is not available, then these people will very likely form an addiction to something else^[2]. That is what ‘addictive personality disorder’ means.

  Sometimes the authorities want to ban the product in order to protect addicts. But no one ever asks the question: how can we stop them from wanting to self-harm in the first place? Because just banning the current popular method of self-harming is not a solution if people will go on to addict themselves to something else instead^[3]. I feel that the discourse has quietly assumed a fabricated option: if these people can’t gamble then they will be happy unharmed non-addicts.

  I am a libertarian, and I have great sympathy for Maxwell Tabarrok’s arguments. But in this case, I think the whole debate is missing a very important question. We should stop worrying about whether [specific product] is net harmful and start asking how we can fix the root cause of the problem by getting effective treatment to people with addictive personality disorder. And yes, inventing effective treatment and rolling it out on a population scale is a much harder problem than just banning [target of latest moral panic]. But it’s the problem that society needs to solve.

  1. ^

     In my parable the principal bans ‘useful’ things, while authorities responding to addictive behaviour usually want to ban entertainment. That isn’t a crux for me. Entertainment is intrinsically valuable, and banning it costs utility—potentially large amounts of utility when multiplied across an entire population.

  2. ^

     Or more than one something else. People can be addicted to multiple things, I’m just eliding that for readability.

  3. ^

     Zvi quotes research saying that legalizing sports betting resulted in a 28% increase in bankruptcies, which suggests it might be more financially harmful than whatever other addictions people had before, but that’s about as much as we can say.

• Dumbledore's Army 22 Nov 2024 18:42 UTC

  0 points

  0

  in reply to: dr_s’s comment on: Cost, Not Sacrifice

  I agree with your first paragraph. I think the second is off-topic in a way that encourages readers, and possibly you yourself, to get mind-killed. Couldn’t you use a less controversial topic as an example? (Very nearly any topic is less controversial.) And did you really need to compound the problem by assigning motivations to other people whom you disagree with? That’s a really good way to start a flame war.

• Dumbledore's Army 15 Sep 2024 16:57 UTC

  11 points

  1

  on: The Great Data In­te­gra­tion Schlep

  I spent eighteen months working for a quantitative hedge fund. So we were using financial data—that is accounts, stock prices, things that are inherently numerical. (Not like, say, defining employee satisfaction.) And we got the data from dedicated financial data vendors, the majority from a single large company, who had already spent lots of effort to standardise it and make it usable. We still spent a lot of time on data cleaning.

• Dumbledore's Army 23 Mar 2024 10:27 UTC

  14 points

  4

  in reply to: Seth Herd’s comment on: What does “au­to­di­dact” mean?

  The education system also tells students which topics they should care about and think about. Designing a curriculum is a task all by itself, and if done well it can be exceptionally helpful. (As far as I can tell, most universities don’t do it well, but there are probably exceptions.)

  A student who has never heard of, say, a Nash equilibrium isn’t spontaneously going to Google for it, but if it’s listed as a major topic in the game theory module of their economics course, then they will. And yes, it’s entirely plausible that, once students know what to google for, then they find that YouTube or Wikipedia are more helpful than their official lecture content. Telling people they need to Google for Nash equilibria is still a valuable function.

• Dumbledore's Army 23 Mar 2024 10:24 UTC

  5 points

  2

  on: What does “au­to­di­dact” mean?

  As Richard Kennaway said, there are no essences of words. In addition to the points others have already made, I would add: Alice learns what the university tells her to. She follows a curriculum that someone else sets. Bob chooses his own curriculum. He himself decides what he wants to learn. In practice, that indicates a huge difference in their relative personalities, and it probably means that they end up learning different things.

  While it’s certainly possible that Bob will choose a curriculum similar to a standard university course, most autodidacts end up picking a curriculum wildly different. Maybe the university’s standard chemistry course includes an introduction to medical drugs and biochemistry, and Bob already knows he doesn’t care about that so he can skip that part. Maybe the university’s standard course hardly mentions superconducting materials but Bob unilaterally decides to read everything about them and make that his main focus of study.

• Dumbledore's Army 18 Mar 2024 19:42 UTC

  53 points

  35

  on: The Worst Form Of Govern­ment (Ex­cept For Every­thing Else We’ve Tried)

  The argument given relies on a potted history of the US. It doesn’t address the relative success of UK democracy—which even British constitutional scholars sometimes describe as an elective dictatorship that notoriously doesn’t give a veto to minorities. It doesn’t address the history of France, Germany, Italy, Canada, or any other large successful democracy, none of which use the US system, most of which aren’t even presidential,

  If you want to make a point about US history, fine. If you want to talk about democracy, please try drawing from a sample size larger than one.

• Dumbledore's Army 29 Jan 2024 18:02 UTC

  2 points

  0

  on: How to write bet­ter?

  I second GeneSmith’s suggestion to ask readers for feedback. Be aware that this is something of an imposition and that you’re asking people to spend time and energy critiquing what is currently not great writing. If possible, offer to trade—find some other people with similar problems and offer to critique their writing. For fiction, you can do this on CritiqueCircle but I don’t know of an organised equivalent for non-fiction.

  The other thing you can do is to iterate. When you write something, say to yourself that you are writing the first draft of X. Then go away and do something else, come back to your writing later, and ask how you can edit it to make it better. You already described problems like using too many long sentences. So edit your work to remove them. If possible, aim to edit the day after writing—it helps if you can sleep on it. If you have time constraints, at least go away and get a cup of coffee or something in order to separate writing time from editing time.

• Dumbledore's Army 1 Nov 2023 14:34 UTC

  2 points

  0

  on: Fore­cast­ing Ques­tions: What do you want to pre­dict on AI?

  First, I just wanted to say that this is an important question and thank you for getting people to produce concrete suggestions.

  Disclaimer, I’m not a computer scientist so I’m approaching the question from the point of view of an economist. As such, I found it easier to come up with examples of bad regulation than good regulation.

  Some possible categories of bad regulation:

  1 It misses the point.

  • Example: a regulation only focused on making sure that the AI can’t be made to say racist things, without doing anything to address extinction risk.

  • Example: a regulation that requires AI-developers to employ ethics officers or risk management or similar without any requirement that they be effective. (Something similar to cyber-security today: the demand is that companies devote legible resources to addressing the problem, so they can’t be sued for negligence. The demand is not that the resources are used effectively to reduce societal risk.)

  NB: I am implicitly assuming that a government which misses the point will pass bad regulation and then stop because they feel that they have now addressed ‘AI safety’. That is, passing bad legislation makes it less likely that good legislation is passed.

  2 It creates bad incentives

  • Example: from 2027 the government will cap maximum permissible compute for training at whatever the maximum used by that date was. Companies are incentivised to race to do the biggest training runs they can before that date

  • Example: restrictions or taxes on compute apply to all AI companies unless they’re working on military or national security projects. Companies are incentivised to classify as much of their research as possible as military, meaning the research still goes ahead, but it’s now much harder for independent researchers to assess safety, because now it’s a military system with a security classification.

  • Example: the regulation makes AI developers liable for harms caused by AI but makes an exception for open-source projects. There is now a financial incentive to make models open-source

  3 It is intentionally accelerationist, without addressing safety

  • A government that wants to encourage a Silicon Valley type cluster in its territory offers tax breaks for AI research over and above existing tax credits. Result: they are now paying people for going into capabilities research, so there is a lot more capabilities research

  • Industrial policy, or supply chain friendshoring, that results in a lot of new semiconductor fabs being built (this is an explicit aim of America’s IRA). The result is a global glut of chip capacity, and training AI ends up a lot cheaper than in a free-market situation.

• Dumbledore's Army 22 Oct 2023 15:53 UTC

  13 points

  6

  on: AI Safety is Drop­ping the Ball on Clown At­tacks, and Mind Con­trol in General

  Although clown attacks may seem mundane on their own, they are a case study proving that powerful human thought steering technologies have probably already been invented, deployed, and tested at scale by AI companies, and are reasonably likely to end up being weaponized against the entire AI safety community at some point in the next 10 years.

  I agree that clown attacks seem to be possible. I accept a reasonably high probability (c70%) that someone has already done this deliberately—the wilful denigration of the Covid lab leak seems like a good candidate, as you describe. But I don’t see evidence is that deliberate clown attacks are widespread. And specifically, I don’t see evidence that these are being used by AI companies. (I suspect that most current uses are by governments.)

  I think it’s fair to warn against the risk that clown attacks might be used against the AI-not-kill-everyone community, and that this might have already happened, but you need a lot more evidence before asserting that it has already happened. If anything, the opposite has occurred, as the CEOs of all major AI companies signed onto the declaration stating that AGI is a potential existential risk. I don’t have quantitative proof, but from reading a wide range of media across the last couple of years, I get the impression that the media and general public are increasingly persuaded that AGI is a real risk, and are mostly no longer deriding the AGI-concerned as being low-status crazy sci-fi people.

• Dumbledore's Army 1 Oct 2023 15:36 UTC

  2 points

  1

  in reply to: Dweomite’s comment on: In­side Views, Im­pos­tor Syn­drome, and the Great LARP

  Do you still think your communication was better than the people who thought the line was being towed, and if so then what’s your evidence for that?

  We are way off topic, but I am actually going to say yes. If someone understands that English uses standing-on-the-right-side-of-a-line as a standard image for obeying rules, then they are also going to understand variants of the same idea. For example, “crossing a line” means breaking rules/​norms to a degree that will not be tolerated, as does “stepping out of line”. A person who doesn’t grok that these are all referring to the same basic metaphor of do-not-cross-line=rule is either not going to understand the other expressions or is going to have to rote-learn them all separately. (And even after rote-learning, they will get confused by less common variants, like “setting foot over the line”.) And a person who uses tow not toe the line has obviously not grokked the basic metaphor.

• Dumbledore's Army 1 Oct 2023 15:16 UTC

  5 points

  4

  in reply to: Dweomite’s comment on: In­side Views, Im­pos­tor Syn­drome, and the Great LARP

  To recap:

  1. original poster johnswentsworth wrote a piece about people LARPing their jobs rather than attempting to build deeper understanding or models-with-gears

  2. aysja added some discussion about people failing to notice that words have referents, as a further psychological exploration of the LARPing idea, and added tow/​toe the line as a related phenomenon. They say “LARPing jobs is a bit eerie to me, too, in a similar way. It’s like people are towing the line instead of toeing it. Like they’re modeling what they’re “supposed” to be doing, or something, rather than doing it for reasons.”

  3. You asked for further clarification

  4. I tried using null pointers as an alternative metaphor to get at the same concept.

  No one is debating the question of whether learning etymology of words is important and I’m not sure how you got hung up on that idea. And toe/​tow the line is just an example of the problem of people failing to load the intended image/​concept, while LARPing (and believing?) that they are in fact communicating in the same way as people who do.

  Does that help?