Benquo

Rea­son isn’t magic

Wait vs In­ter­rupt Culture

Hand­shakes, Hi, and What’s New: What’s Go­ing On With Small Talk?

Can crimes be dis­cussed liter­ally?

Si­mu­lacra and Subjectivity

Sab­bath hard and go home

Effec­tive al­tru­ism is self-recommending

Zetetic explanation

Ap­proval Ex­trac­tion Ad­ver­tised as Production

Against responsibility

Be se­cretly wrong

• Benquo 28 Dec 2013 16:09 UTC

  65 points

  on: Why CFAR?

  To positively reinforce CFAR for finally posting this, I’m going to give $750 before the end of 2013. This is separate from my matching funds pledge—treat it like any other donation.

  In addition my employer should match that, for a total of $1,500, or $3,000 when you count the fundraiser’s match of both.

  UPDATE: Donation made. I’ll request the employer match in the next few days.

  UPDATE2: Employer match requested

Doubt, Science, and Mag­i­cal Crea­tures—a Child’s Perspective

Cash trans­fers are not nec­es­sar­ily wealth transfers

• Benquo 13 Mar 2019 15:31 UTC

  58 points

  0

  on: You Have About Five Words

  You’re massively underestimating the upper bound.

  I’ve interacted a bunch recently with members of a group of about 2 million people who recite a 245-word creed twice daily, and assemble weekly to read from an 80,000 word text such that the whole text gets read annually. This is nowhere near a complete accounting of engagement with verbal canon within the group. Each of these practices is preceded and followed by an additional standardized text of substantial length, and many people study full-time a much larger canonical text claiming to interpret the core text.

  They also engage in behavior patterns that, while they don’t necessarily reflect detailed engagement by each person with the content of the core text, do reflect a lot of fine-grained responsiveness to the larger interpretive canon.

  You might be closer for what can be done very quickly (within a single generation) under current conditions. But a political movement plenty of people are newly worried about which likely has thousands of members has a 14-word creed.

• Benquo 17 Jul 2019 7:13 UTC

  54 points

  0

  in reply to: Raemon’s comment on: In­tegrity and ac­countabil­ity are core parts of rationality

  Thanks for checking—I’m trying to say something pretty different.

  It seems like the frame of the OP is lumping together the kind of consistency that comes from using the native architecture to model the deep structure of reality (see also Geometers, Scribes, and the structure of intelligence), and the kind of consistency that comes from trying to perform a guaranteed level of service for an outside party (see also Unreal’s idea of Dependability), and an important special case of the latter is rule-following as a form of submission or blame-avoidance. These are very different mental structures, respond very differently to incentives, and learn very different things from criticism. (Nightmare of the Perfectly Principled is my most direct attempt to point to this distinction.)

  People who are trying to submit or avoid blame will try to alleviate the pressure of criticism with minimal effort, in ways that aren’t connected to their other beliefs. On the other hand, people with structured models will sometimes leapfrog past the critic, or jump in another direction entirely, as Benito pointed out in A Sketch of Good Communication.

  If we don’t distinguish between these cases, then attempts to reason about the “optimal” attitude towards integrity or accountability will end up a lumpy, unsatisfactory linear compromise between the following policy goals:

  • Helping people with structurally integrated models notice tensions in their models that they can learn from.

  • Distinguishing people with structurally integrated models from those who (at least in the relevant domain) are mostly just trying not to stick out as wrong, so we can stop listening to the second group.

  • Establishing and enforcing the norms needed to coordinate actions among equals (e.g. shared expectations about promises).

  • Compelling a complicated performance from inferiors, or avoiding punishment by superiors trying to compel a complicated performance from you.

  • Converting people without structurally integrated models into people with structurally integrated models (or vice versa).

  Depending on what problem you’re trying to solve, habryka’s statement that “if someone changes their stated principles in an unpredictable fashion every day (or every hour), then I think most of the benefits of openly stating your principles disappear” can be almost exactly backwards.

  If your principles predictably change based on your circumstances, that’s reasonably likely to be a kind of adversarial optimization similar to A/​B testing of communication. They don’t mean their literal content, at least.

  But there’s plenty of point in principles consistent with learning new things fast. In that case, change represents noise, which is costly, but much less costly than messaging optimized for extraction. And of course changing principles doesn’t need to imply a change in behavior to match—your new principles can and should take into account the fact that people may have committed resources based on your old stated principles.

  In summary, my objection is that habryka seems to be thinking of beliefs as a special case of promises, while I think that if we’re trying to succeed based on epistemic rationality, we should be modeling promises as a special case of beliefs. For more detail on that, see Bindings and Assurances.

  What links here?

  • Clar­ify­ing Your Principles by Raemon (1 Oct 2022 21:20 UTC; 60 points)
  • Raemon's comment on Drown­ing chil­dren are rare by Benquo (26 Nov 2019 2:53 UTC; 5 points)
  • Raemon's comment on In­tegrity and ac­countabil­ity are core parts of rationality by habryka (13 Aug 2019 2:16 UTC; 4 points)

OpenAI makes hu­man­ity less safe

Ex­cerpts from a larger dis­cus­sion about simulacra

• Benquo 25 Mar 2012 16:54 UTC

  52 points

  in reply to: Daniel_Starr’s comment on: Harry Pot­ter and the Meth­ods of Ra­tion­al­ity dis­cus­sion thread, part 12

  You’re forgetting that Tom Riddle actually did study at the monastery before he destroyed it to deny that training to his enemies.

  Voldemort is especially violent and comes off as stupid, but he’s just one of Tom Riddle’s characters, and if you consider their actions as a whole they’re smarter than they appear, on purpose.

  There is a classic trick that card counting teams use to avoid detection. If one person shows up, and bets conservatively until the cards are in their favor, and then immediately starts making huge bets, then it is obvious that they are a card counter and the casino can throw them out. But, if that one person betting conservatively simply leaves the table once he thinks the deck is in his favor, and then someone else comes in wearing a loud Hawaiian shirt and acting like a “wild and crazy” risk-lover, then it just looks like someone risk-averse has been replaced with someone risk loving, and neither looks like they’re counting cards.

Towards op­ti­mal play as Villager in a mixed game