XelaP

• XelaP 8 Jun 2026 8:57 UTC

  1 point

  0

  in reply to: lukeprog’s comment on: Shut Up and Divide?

  I think I happen to also have a second-order want to not want to help strangers. I don’t act on it (beyond wanting to roughly keep my meta selfishness-altruism allocation the same, or otherwise not want to increase my altruism too much), and have already made my decision basically final; but I’ve definitely considered it before.

  There are also some actively sadistic desires paired with second order desires, though those are about the experience I feel instead of a terminal desire for harming others or for reallocating resources towards selfish gain. These I have never seriously entertained acting upon.

  I sometimes fantasize that I’ll live to see humanity get the good ending and get to revel in simulated torture of non-sentient simulacra (or actors that feel no pain and take no real damage but are expertly pretending to).

• XelaP 8 Jun 2026 8:48 UTC

  1 point

  0

  on: Shut Up and Divide?

  These do not seem contradictory. One can divide at the low end, and multiply at the high end. I don’t think I’ve multiplied as much as Eliezer, but I may have divided as much as you—though I suspect other unknown factors make me generally less empathetic, and so unaffected by random murders or school shootings or individual car crashes far from myself. A large enough harm, if novel, will lead me to double-take and feel real horror.

• XelaP 8 Jun 2026 8:32 UTC

  1 point

  0

  on: How do peo­ple stop spiral­ing about Roko’s Basilisk & acausal ex­tor­tion?

  I never really spiralled, but the counterarguments came readily to me. Hopefully, you can be reasoned away from a position that you’ve been reasoned to. If you deeply understand why it’s silly, then you are less likely to spiral! After all, you had to think through a convoluted argument to even know what to spiral about in the first place!

  Here’s the version my grown-up self comes up with:

  The most fatal flaw: Don’t negotiate with terrorists! Come on, man, how did you end up getting this far into acausal decision theory without realizing that it recommends that you don’t succumb to threats made only because the counterparty believes you’ll react? A basilisk would straightforwardly be making the sort of threat that decision theory seems to say you should ignore, for the same reason you should ignore a smart enough predictor that takes hostages to ransom you with or a dictator that threatens to nuke the world unless he’s given an ivory castle.

  Now, the second most fatal flaw: you cannot actually predict what superintelligent AIs will do in enough detail to do acausal stuff with them. Two humans can barely do this with each other, let alone with aliens that’re waaaaaay smarter and possibly literally using algorithmically near perfect simulations to predict you. You certainly aren’t going to get almost anywhere by thinking about what the distribution of AIs will acausally agree upon amongst themselves. They are, again, superintelligent aliens! First, why don’t you predict what promises (if any) I’ve made with my past romantic partners. And note that I leak way more info about myself than merely the fact that I am a human, which is the closest level of info analogous to what you know about a generic superintelligent alien mind.

  
  As a similar example that may illustrate the absurdity of a specific prediction: I have no hope of figuring out what sorts of universal turing machines make our universe have a lot of weight in the simplicity prior and then reasoning about what sorts of universes and what sorts of minds would be using the simplicity prior, and then trying to manipulate it via the most likely output channels to gain power in the real universe. I’m certainly not going around sorting rocks into shapes resembling the konami code in the hope that a mind considering our world as a hypothesis reads the shape of the rocks and reads the konami code as a prediction that not building me in the real world will lead to horrible outcomes. How would I know what good manipulation strategies would be, given all that computational difficulty?

  This example is, if anything, easier to reason about than acausal decision theory with superintelligences—at least we have math formalizing AIXI! The reason I bring it up is that the (relative—in absolute terms it’s abstract as hell) concreteness provided by our mathematical understanding makes it clear how impossible of a task this is for a human. As in, literally impossible—if you manage it without going full transhuman or offloading it to something at least that powerful, I will literally eat a hat.

• XelaP 8 Jun 2026 8:16 UTC

  1 point

  0

  in reply to: JBlack’s comment on: How do peo­ple stop spiral­ing about Roko’s Basilisk & acausal ex­tor­tion?

  The whole point of the basilisk is that it has large stakes and that you can affect those stakes (acausally). Whether unlikely or not, you have (acausal) control over large outcomes, and so in expectation it matters. While I think humans, with our current foolishness, should have inertia to taking significant mainline-damaging actions for unlikely possibilities (especially if the argument is some weird abstract thing with many steps that’s endorsed by few and that you don’t fully understand well enough to reason about and that is widely acknowledged in the field to have fundamental unsolved confusions (like logical counterfactuals, or just how AIs work, like, at all)), ultimately I endorse acting on low probability high stakes outcomes once you’ve crossed certain reliability-in-reasoning vs cost thresholds.

  The real way that the basilisk privileges the hypothesis is by picking a basilisk in particular. There should be very many different AIs that would all want you to build them but not the other.

  Perhaps some large fraction of them could acausally agree to incentivize creating any AI, or any AI within certain bounds, or with stepped incentives for various AIs depending on how much power they had in the acausal negation. There is, of course, no way in hell for you to actually figure out what they’d bargain—so you don’t know what AI you’re supposed to build!

• XelaP 8 Jun 2026 7:54 UTC

  2 points

  0

  in reply to: mako yass’s comment on: Ex­am­ples of Examples

  ...why not just take Goodstein’s theorem, which can be expressed in Peano Arithmetic but not proven in it—yet it can be proven in ZF. I assume we both agree that the continuum hypothesis isn’t clearly true or false, but if you happen to have an opinion there you could take that. Or, for that matter, if you believe that Peano Arithmetic or ZFC or whatever your favorite formal theory is true with high probability, then a consistency statement in that theory is an unprovable statement that you believe! You can write down a statement that is a translation of “Will this turing machine that searches until it finds a contradiction halt?”

  Secondly, you say

  any other field would have accepted that the evidence was overwhelming and moved on long ago. A physicist would simply declare it a law of reality.

  ...but both math and physics have solved centuries old conjectures, even close to the modern day. The parallel postulate was generally thought to be necessary for consistent geometry for 2 millenia; Fermat’s last theorem took 3 centuries; The 36 officers problem took 2 centuries (and the general case took another 50); Catalan’s conjecture took 1.5; The Poincaré conjecture and the four color theorem, took 1 century. Quantum mechanics and relativity overturned major bodies of work that had been around for centuries. Gravitational waves took a century to discover from when they were proposed; the estimates of the age of the universe had issues that took like 75 years to work out; and the Higgs boson and pentaquarks took like 50 years to discover.

  The mathematical examples are especially impressive because of how old and developed core fields like number fucking theory are. Whereas the entire concept of turing machines has only been around since the 30s (1 century ago) and P vs NP is from 1970 (60 years ago).

  When the entire field is only a century old, and the subfield (complexity theory) and problem in question is only around 50 years old, you really shouldn’t update that much on the problem being unprovable.

  The only examples of unprovable statements that were found ‘in the wild’ I’ve heard of are consistency statements and the continuum hypothesis (mathematical logic probably has plenty but I consider that cheating, in the same way that undecidable problems in computer science are cheating). So the base rate of unprovable statements in non-logic fields is quite low, even if we go ahead and count every 50 year old famous important open problem as unprovable.

  Get back to me in another century (if we’re still alive) and I’ll consider it plausible. But to think there’s more than a small chance of unprovability is to forget history.

• XelaP 8 Jun 2026 7:15 UTC

  1 point

  0

  in reply to: Gurkenglas’s comment on: Au­topi­lot Thinking

  I seem to basically have one verbal ‘stream’, so that I mostly cannot think in words while talking or think in words about two different things. If talking about something that I know werl and have talked about before, I will sometimes find myself doing a little bit of accessory planning (sometimes even in words!) while my mouth moves in coherent ways, but I sorta feel like if I focused on it that the spell would break. It’s an interesting question, though, so I’ll try to notice next time.

  I can walk and talk, or drive and talk, easily. The only examples that come to mind of me talking and visualizing were times where I was describing what I was visualizing or reporting a conclusion from my visual thinking, as opposed to talking about something else.

  I think my autopilot thinking is more like if Claude only had a context window spanning a dozen tokens.

• XelaP 8 Jun 2026 6:22 UTC

  1 point

  0

  in reply to: Gurkenglas’s comment on: Au­topi­lot Thinking

  My memory is generally worse, though I’ll still remember broadstrokes.

  I’m unsure what multitasking would look like outside of certain time-pressured jobs. I think I do not usually multitask, regardless of mental state. But it also looks like other people are like this, too?

  I’d suspect that multitasking should not be made much worse by going on autopilot (beyond the way that individual tasks are), as task-switching is already going to require dumping my working memory. I’d be slower at each step, though, and so if the reason why I’m multitasking is because I need to attend to multiple types of events that need to be processed quickly, then I’d expect to be much worse.

• XelaP 8 Jun 2026 2:44 UTC

  5 points

  2

  in reply to: leogao’s comment on: leogao’s Shortform

  For empirical questions, one litmus test is whether most of the populace doesn’t know some true (like stats reported by large institutions with no disagreement from other reputable institutions) and clearly relevant fact, or repeatedly overestimates the observed risk of something.

  AI water usage is an example. Others are the deaths per TWh of nuclear power, that extreme poverty (≤$3/​day) went down by 3 times in the last 20 years (along with what percent are below that line), observed prediction market accuracy, the annual revenue of the US federal government as compared to e.g. the total net worth of billionaires (it’s about ¹⁄₃ of it), and that the labor share of GDP over time hasn’t dropped much.

  I sometimes wonder if political beliefs would be very different if every child knew a certain set of basic stats—and also what political beliefs you could get if you adversarially optimized that set (while still keeping the stats reliably true and ‘basic’) to lead them to some specific conclusion.

• XelaP 7 Jun 2026 22:08 UTC

  9 points

  0

  in reply to: habryka’s comment on: Some Dat­ing Stories

  Hmm, when reading “fuck”, my first reaction was to feel the vibes of a kind of ~sleazy man that ranges from actively harmful to merely having different tastes than me. Then I thought “Ah, it’s John, he probably didn’t mean that”.

  I’d say I then implicitly got like a fourth of the intended meaning. The main other meaning I got was “casualness”. I tend to use “fuck” a lot as a generic intensifier in casual conversation, though I (ironically) don’t tend to use it to talk about sex, seemingly mostly due to a flinch about coming off as sleazy (I do not feel the same flinch when I imagine saying “Alice fucked me”). I think this is why I thought that that’s most of what was meant.

• XelaP 7 Jun 2026 21:54 UTC

  2 points

  0

  in reply to: habryka’s comment on: xpostah’s Shortform

  Did you also disable samuel’s ability to receive (and send?) DMs on here, as you did with the case mentioned in that linked comment?

Au­topi­lot Thinking

• XelaP 7 Jun 2026 16:31 UTC

  4 points

  1

  on: Good brain­teasers for chil­dren?

  When you look at words in a mirror, how come they’re reversed left to right but not top to bottom? What’s special about the horizontal axis?

  Mirrors don’t reflect left to right, they reflect back to front. If I make an L with my hand as seen by my POV, it’ll still be an L in the mirror. However, if I write an L on a piece of paper, and then pick it up and show it to the mirror, then I’ve flipped the paper. That is, looking at text on a T-shirt through a mirror is like looking at the text from the inside of the shirt

  I look at a restaurants reviews, and pick a 4 star restaurant (the best given my budget). When I go in, I notice that they have great music, seating, and looks. I sigh, and reduce my expectations of their food quality. Why?

  The good other stuff means that the restaurant no longer needs to have unusually good food to get 4 stars, so there’s not so much selection on that. So food quality is closer to average.

  

  Suppose I uniformly randomly pick a point in the square. Whats’s the probability I pick the center?

  0%. It is, of course, still possible.

  A barber shaves the heads of all men in his town who do not shave themselves (and only those men). Does he shave his own head?

  This is known as Russell’s paradox, and it shows that naive set theory is contradictory. You can try explaining a little bit about sets and then ask if the set of all sets that dont’t contain themselves contains itself.

  Why does evolution happen, anyways?

  Things that reproduce more will have more of themselves later. Genes that lead to more children with itself in it will become more common. Nothing magical, no “bettering” force, just what is practically a tautology that the things that copy themselves more in the later generations will have more of their copies in those generations.

  ...so, do, say, bears have children way more than dinos did?

  I think the standard answer comes from Fischer: organisms are in an ‘arms race’, where you have to do all this adapting and evolving just to stay at the same fitness, given that your competitors are doing so too: a Red Queen’s Race.

  I have a set of three 6 sided dice with numbers on the faces. We’ll use them to play a game: you’ll pick one die, and then I’ll pick another. Then we’ll roll our dice, and the highest number will win.

  I must admit, however, that I’m a dirty cheater: my set of dice have the special property that I can always pick a die in response to yours that beats it on average! That is, dice A is beaten by dice B is beaten by dice C is beaten by dice A! How can this be? Can you come up with an example?

  (this took adult me like a couple hours to figure out, but there are probably some kids that could do it in 10 minutes. for the most part prerequisite explicit math skills don’t help for this one).

  These are called intransitive dice, and they’re basically the same thing as having three politicians where for every candidate, a majority always prefers a different candidate.

  Which came first, the chicken or the egg?

  I don’t know of a standard answer to this, but ever since I was a little kid I thought the answer was obvious: at some point, a non-chicken will lay an egg with enough genetic similarity to cross whatever your ‘chicken’ threshold is, and thus be the progenitor of chickens. As I’d state it now: chickenhood is not a property that passes perfectly from hen to egg, for the genome of a species is made of many tiny parts that mix together.

  A tree starts from a tiny seed, and grows into a huge tree. Where does the mass come from?

  Almost all of it is from the air. Plants make their carbohydrates from carbon dioxide.

  A dictator rules an island full of 100 green-eyed people. No person is allowed to discover what their own eye-color is via mirrors, discussion, etc. They can, however, each see that everyone else has green eyes. The only possible eye color is blue.

  Every night, the dictator allows anyone who wants to leave to get into a ferry, which will take them to a refugee camp in a safe country if they have green eyes, but will kill them if they don’t.

  The people of this island will only attempt to leave if they are ‘sure’ they won’t be killed. They are also all ‘perfectly rational’, and they all know these two facts about each other, and know that they all know those facts, and know that they all know that they all know that they know thase facts, etc.

  One day, after pressure from the UN, the dictator agrees to make an announcement to the island that he believes is harmless but that the fussy humanitarians insist on him saying. The dictator, evil as he is, is known to speak only truth, and has never once lied publicly. He announces the following:

  “At least one of you have green eyes”.

  As expected, nothing seems to happen. After all, everyone already knew that at least one person had green eyes!

  However, around three months later, everyone leaves the island, certain that they have green eyes. How come?

  Suppose there were 2 people on the island. If only one had green eyes, then they’d see the other had blue, and leave the first night. So if nobody leaves on the first night, they can conclude that they must have green eyes, and then leave on the second.

  Suppose there were 3 people. If I see two people with green eyes, I’ll wait to see if they’ll leave on the second night. If they do, then I know I have blue eyes; otherwise I know I have green eyes. Thus we all leave on the third night.

  Repeat to get that everyone leaves on the 100th night.

  Rockets don’t go straight up. Why not?

  Unless they went really far, they’d just fall back down. So instead they curve sideways, to then orbit the planet. It’s like falling, but while moving sideways fast enough that the ground curves away from you before you can hit the ground, forever. This is called a gravity turn.

  Suppose Alice and Bob are fruit growers. Alice is slow at making apples, and mediocre speed at making oranges. Bob is fast at making oranges, and really super duper extra fast at making apples.

  Despite Bob being better at everything than Alice, they can trade to both be better off. How come?

  Bob’s time is best spent making apples—every orange made uses up time that could’ve been spent making even more apples. Alice’s time is best spent on oranges. They should specialize in what they are best at, then trade to get whatever of the other fruit they need, so both will have more of each fruit. This is known as the law of comparative advantage.

  Say I’m buying insurance. The insurance company charges a price that’ll make them a profit on average, given however much risk I have of triggering a payout. So I must be losing money, on average.

  How come I still might want to buy insurance, then? Is one of us necessarily being scammed?

  An extra dollar is worth less to you the richer you are, so losing a dollar hurts more than gaining one. Approximately speaking the extra amount that losing hurts is worse when you are poorer than when you are richer—it’s a greater fraction of your total wealth.

  Therefore, poorer people are hurt more by risks than richer people. A 10% chance of losing $50k is much worse than a guarantee of paying $5k if I’m a normal person, but is not that different if I’m a billionaire. A richer person can thus profit on average by selling me insurance, while I am happy to have less risk.

  Additionally, a big company can sell insurance to lots of people, and thus exploit averages to reduce their risk.

  If I flip a coin and get 9 heads in a row, what’s the chance the next flip is tails?

  50%. Beware the gambler’s fallacy—the coin doesn’t have memory!

  Suppose someone takes a mammogram (a breast cancer test). If they have cancer, it’ll detect it most of the time (90%). If they don’t, it’ll still say they have cancer 10% of the time.

  Among 40 year old women, 1 in 1000 have cancer. If a 40 year old woman gets a positive result on her mammogram (as in, it says she has cancer), what probability does she have cancer?

  About 1%: the prior is around 1:1000 odds of cancer. If she has cancer it’ll say so 90% of the time, but if she doesn’t it’ll still say so 10% of the time, so each cancer case becomes 90 positive results and each healthy case becomes 10k positive results, giving us final odds of 90:10k ≈ 100:10k = 1:100 ≈ 1%. Having cancer is so unlikely that the test isn’t enough evidence to make you think you likely have cancer.

• XelaP 7 Jun 2026 14:00 UTC

  1 point

  0

  on: Bear spray ex­piry dates: good news, and stag­ger­ing peer-re­viewed pseudoscience

  I think this is an instructive example of bad science. The principle here is understandable to someone with educated layman knowledge, and yet the paper gets it wrong. Getting it wrong is understandable, but not having anyone realize the mistake by the time you publish is the real problem.

  How should they (or, our future selves) have noticed their mistake? As you point out, they should’ve at least noticed their confusion when spraying steady for 7 seconds, which blatantly contradicts their analysis. Another way they could’ve done better is, perhaps, to do some cursory googling about how such spray cans work?

  As for the post: I wish you also looked at what other papers in the literature say. Is it just this article that’s silly, or is there a whole slew of broadly accepted research that make similarly damning mistakes?

• XelaP 6 Jun 2026 9:39 UTC

  4 points

  0

  on: Xe­laP’s Shortform

  It looks like frontier models should soon beat Metaculus community forecasts, and if the trend holds they’ll be beating pro forecasts by the end of 2027.

  https://​​www.metaculus.com/​​futureeval/​​

  

  (the site’s plot isn’t as blurry as my quick screenshot, sorry).

  Metaculus questions (currently pending review):

  When will AI beat Metaculus Community?

  When Will AI beat Metaculus Pros?

  I wonder how the AI prediction about AI predictions compared to the human predictions compare to the human predictions.

• XelaP 2 Jun 2026 3:28 UTC

  1 point

  0

  on: “Con­ta­gious Hum­ming” to Silence a Room

  Are you humming a short ditty, or is it a monotone drone that slowly grows as members of the crowd are assimilated into the hivemind?

• XelaP 28 May 2026 13:56 UTC

  3 points

  0

  in reply to: DaemonicSigil’s comment on: Dae­mon­icSigil’s Shortform

  Rhymes with Is Clickbait Destroying Our General Intelligence? I suspect that writing a good answer to a silly question on r/​whowouldwin, arguing with a friend about what Light Yagami’s next move should be, and the art of best response are all pointing in a similar direction. Likewise, I suspect some amount of personality and internal ‘self-hood’ is built socially.

• XelaP 22 May 2026 3:16 UTC

  1 point

  0

  on: Unify­ing Bar­gain­ing No­tions (2/​2)

  If the pareto frontier has a “flat face” (subset of a linear hyperplane), then you’ll never be able to ‘rationalize’ a non-extreme point (that is, a non-vertex) as the maximum of a weighted sum of utility functions, because either the maximum will be an extreme point or an entire face/​subface (like the sides of a filled in triangle) will have the same maximum value. As an example, if there are two possible options, each of which benefits only one of the players, then a aggregate agent with VNM preferences must either choose deterministically, or be completely indifferent about how to randomize—that is, to either give all the benefit to one player, or be completely indifferent about how much to benefit one player over the other in the randomization.

  For a Pareto Frontier that’s just a single face, you should be able to just do CoCo bargaining like in the previous post—e.g. if Alice gets $5 or Bob gets $1 or if they can’t agree both get $0, then CoCo says to do the option that produces $5 for Alice and then transfer so that they end up with $3 each.

  In general, I think a ‘flat face’ means that on that face, you have transferable utility.

• XelaP 20 May 2026 10:16 UTC

  16 points

  2

  on: Pythagorean addition

  According to the Wiki page, it looks like if you are doing max(a, .9a + .5b), that the worst case error is more like 2%, as you can see from their second table.

  I’d also note that a + ¹⁄₂ b has worst case error of 12%, which might be good enough if you are doing Fermi estimates. It’s also easily explainable: sqrt(a^2 + b^2) = a sqrt(1 + (b/​a)^2) whose first order taylor series approximation is a(1 + ¹⁄₂ b/​a) = a + ¹⁄₂ b. You’ll want a to be the max and b to be the min for a better approximation. This will always overshoot, as it’s an alternating series.

  If you’ve already used the (1 + x)^r ≈ 1 + rx trick before, then you don’t have to remember anything new. You can then see the alpha=.9 as correcting for the overshoot, and the max(a, .9a + .5b) as correcting for the obviously wrong results when b is small (as the Pythagorean sum can’t be smaller than the maximum of the inputs!).

• XelaP 19 May 2026 0:43 UTC

  1 point

  0

  on: Thoughts on in­ter­view­ing can­di­dates for AI safety fellowships

  Most rationalist fellowships are very good about this (thanks!), but it’s worth saying: The only timezone for deadlines should be anywhere-on-earth.

  Err, what do you mean by this? I’d think the deadline should be tied to some day and time, with the time given along with the timezone it’s in.

• XelaP 19 May 2026 0:32 UTC

  3 points

  0

  in reply to: James Payor’s comment on: leogao’s Shortform

  I think some can have the feelings that go with hope in a cause, without actually believing the cause is likely to succeed. Cultural memes around fighting for hopeless causes (e.g. when heros go for hail mary strategies in movies) help.

  It still matters whether you truly think it’s the best shot at victory, or the best way for you to help. That’s what I see as key to preventing the various problems that you mention.