Steff
What I want to see,
which I predict is maybe more realistic(EDIT: Thinking on it more, I’ve changed my prediction, I think convincing one frontier company to shut down is actually more likely!) --
US and China buying equal stakes in each others’ companies. We sell half of Anthropic and OpenAI and a spun-off Deepmind. They sell half of DeepSeek and spin offs of Alibaba and Baidu.
We work together for a slow down, gesture towards great partnership and profit sharing and economic growth between the two countries yadda yadda, and make a good example for the rest of the world so India et al will also join in.
Over many repetitions (to balance out bad luck streaks), it’s rational to play the game for prices of $20 or $19. If you’re saying that SB should reject those prices when presented it while waking, that’s irrational, because an SB that accepts those prices will make money.
This is assuming that if asked twice, she’ll always make the same decision because her decision-making is deterministic. (We could assume there’s a chance she makes a different decision on Tuesday than on Monday but I think that’s an unnecessary and irrelevant complication?)
If he cancels the invasion when Beauty is sleeping, then that weighs the probability towards worlds where she’s not asleep, i.e. Tails. Which is why Heads goes down from 3⁄4 to 2⁄3.
(Not sure if this is identical to the $30/6/6 EV form of the questions? Not sure which is more useful)
I take back one piece, when I said $18 doesn’t correspond to anything. I believe that corresponds to how much Beauty should expect to be making per day on average: (1/2)(30) + (1/2)(6).
If we’re talking about expectation per iteration, though, then it should be (1/2)(30) + (1/2)(12) = $21. If you’re using the $18 value there, then you can get money pumped.
I’m going to be sort of shitty and not give this comment the time it deserves, and I’ll only respond to parts of it. If there’s anything you’d like to focus on that I ignore, please hit me up with another comment and I’ll try to get to it when I can.
First, I’ll say that some of the confusion probably stems from the fact that the post is defending against an argument against Halferism, so I’m presenting a Halfer point and showing it to not result in any contradictions, not justifying the Halfer point directly. You being a Thirder, you wouldn’t think that P(obs Mon) = 3⁄4, since that’s not the Thirder position.
Secondly, I’d like to note my complete agreement with the statement “SB’s answer really depends on what she believes the Riddler used as an heuristic for invading the experiment”.
>> I notice that you seem to be ignoring that point (that your versions of SB problem that have 1⁄2 as a solution work exactly the same without the whole sleep and amnesia thing)
I’m not sure what you’re saying with this piece, but my guess-of-an-answer-to-what-you-might-be-saying is this: The solution does work exactly the same as asking about the coin without any sleep or amnesia, because it should be the same. That’s part and parcel to the whole Halfer position that in waking, Beauty hasn’t learned anything, and without learning anything, there’s nothing to update.
(Analogously: There’s no need to talk about “self locating” or “self indicating” in anthropics, because being-a-self gives no additional information on top of “there exists 1 of the thing that is me”.)
>> The proportion of time she would spend awake on Monday during many repetitions of the experiment is 2⁄3. She should answer 2⁄3. After finishing writing this reply, I think there’s a chance that the crux of our disagreement is here.
Damn, that’s my mistake. 2⁄3 of the time is spent awake on Monday, that’s true. I can see why that would seem to indicate the Thirder position. Probably most helpful would be to focus on my Riddler example, with the added condition that the Riddler wasn’t doing either of the actions you described. Instead, he didn’t know about Beauty at all until he happened to stumble upon her. (I’ve fixed the post up a bit.)
I’m guessing you’ll say that in this version, Beauty’s belief in Monday should not be 3/4?
>> What if the ASI itself protects liberal democracy?
>> Liberal democracy is not the true target
I think the only feasible solution is one and the same: ASI protects the government, and the government is good government because it replaces liberal democracy because AI BECOMES the new government. To some extent.
You could imagine a world where China still has Xi and America still has a president, but their actions are all constrained by what the ASI will allow.
Then of course, the greatest political matter of the day will be: Who gets to influence the ASI? Either corrigibly, or if the ASI can’t be influenced, then this question is reserved for the period of time leading up to the ASI’s escape-from-our-influence.
I’m thinking there’s a legit chance that the future will come down to one man (Asmodei) and all of our efforts should be focused on trying to teach him and encourage him to be able to make the best choices he can make.
I believe what you’ve described is just the original problem again with an extra possible waking of no question asked on Tuesday, but that doesn’t impact the credences of her waking and observing the question. The answer is still 1⁄2.
>> I ought believe that my guess will be correct if I guess Heads is the same as the degree I ought believe that the outcome of the toss is Heads
This disregards that on Tails, the same toss result will result in an extra guess.
>> if you preface your question with “when you are first awakened”, then the answer will always be 1⁄2 whatever comes next
Ah, yes, the Day 1 Objection. I refute this argument here:
https://ramblingafter.substack.com/p/sleeping-beauty-meets-two-face
Sorry for taking so long!
$14 was a mistake, that’s my bad and I’ve updated the post.
>> This betting scheme is a based on a version of the SB problem where the Monday guess and the Tuesday guess are counted as a single one
This is the piece I don’t understand about thirders (apologies if that sounds like aggressive wording): If you have a credence for the coin, why can’t you use it?
>> (your second variant) SB is paid for both Monday and Tuesday when she bets on either Monday or Tuesday… if you repeat the experiment a large number of times, and then tag each wake-up event with the profit that SB made during the iteration that included that event, then the average of tags will be $18
Not exactly. I laid out a separate $6 on Mon and $6 on Tues to make this easier to think about. Combining into $12, you end up with the same amount of money being equally shared between two days. I don’t think it’s accurate to say that SB made $12 on Mon and also say that SB made $12 on Tues. So in other words, I think this scheme is doing double counting. Which also means to me that the $18 value doesn’t really correspond to anything relevant and useful to the scenario?
Hey there, I was wondering if you’d be able to formulate why my following counter is wrong?
https://www.lesswrong.com/posts/cqdDiMe6K6WcMAF2R/boltzmann-brains-like-doomsday-require-no-explaining
I’ve been getting a lot of downvotes, so I figure I must have made a mistake somewhere, but nobody has explained it yet (well, I’ve received one comment but not one I understood).
I assume you’re too busy and this isn’t important so I’m not actually expecting any response here—either way, cheers! I’m a fan of all your work
Hey, thanks for the comment.
I’m not understanding the problem that needs fixing.
Firstly, any observation I make provides no evidence to either Boltzmann brains or non-Boltzmann brains.
Secondly, let’s say the first was false and observations do lend credence to non-Boltzmann brains. I can still believe in a theory where Boltzmann brains greatly outnumber non-Boltzmann brains, which is what my post is about.
>> A fair coin is flipped. If it lands heads, nothing happens. If it lands tails, an extremely biased towards tails coin is flipped. If that coin lands tails, you are killed. If that coin lands heads, nothing further happens.
That’s the essence of my post: The question that begs itself. Your setup assumes a coin flip. But we have no reason to think that the universe is acting in such a coin flipping way.
I didn’t think it involved cards at all, I assumed the card symbols were being used as shorthand for something else, haha.
Disagreed—I don’t believe that the idea of Boltzmann brains requires theories to explain “why are we observing ourselves to be structured brains” whatsoever.
Short version is this: If both types of brains must exist, then there must be observers of both types. Observers of the minority type haven’t learned anything to update priors on, since it’s guaranteed that they must exist, and there’s no reason to assume P(structured) = K/(K + M) unless you were privy to some sort of random selection system in place that picked you out ahead of you existing.
Longer version explanation here: https://www.lesswrong.com/posts/cqdDiMe6K6WcMAF2R/boltzmann-brains-like-doomsday-require-no-explaining
It’s the exact same assumption that goes into the Doomsday Argument, but it’s an assumption that requires justification. E.g., if you believed that a God exists and He instantiates souls before inserting them into bodies randomly, then the Bayesian math would flow from there and you would indeed believe on a relatively soon Doomsday or that universes must have an explanation for why-aren’t-I-a-Boltzmann-brain-which-was-overwhelmingly-more-likely.
To counter the idea of Boltzmann brains, isn’t it enough to simply observe that order is self-replicating? Therefore ordered brains arising from order are vastly more likely than those arising from chaos.
I’m hoping to find the time for a full response later, but right now:
>> First, I’m still waiting for you to fully describe the random day. Is it chosen from a set of two, three, or four?
It’s a random choice among the days she’ll be awoken. If she’s to awake just Mon, then she’ll be offered on Mon. If she’s to awake on Mon and Tues, then a coin is flipped.
The bet must be offered on each waking day, and evaluated each time SB is asked fro a credence, to correctly represent her credence. If you do this, the experiment is a zero-sum game if and only if 2:1 odds are used. But I suspect you will disagree with me. Which is why I do not use betting arguments.
I actually think this might be useful to drill into further. Why couldn’t SB use her credence of the coin for the bet I described? Are you thinking that SB actually has two different credences about the coin, and which one she applies will depend on the exact question asked?
You have ignored how I point out that the need to consider it is demonstrated by letting SB observe it in a way that distinguishes it from the others: Wake her on Tuesday, after Heads, but take her shopping instead of asking about the coin.
Hoping for a clarification with this one: Do you mean that she goes shopping on Tuesday and is told on Tuesday? Or that a random day was chosen for her to go shopping, and she knew that it’d be a random day? I think I would agree with your presented math depending on exactly what she’s learning.
Okay I’m getting tired of fighting the editor, going to quote differently:
>> That when SB is asked the question, there are exactly three possible faces that could be showing, and one of them is green.
I’m not denying that those are the possibilities, but that they share equal probabilities.
>> Did you note that the problem asks about her credence based on her current circumstance, and that there are circumstances that differ? Circumstances that can exist regardless of whether SB is awake.>> So Pr(Awake) is not 1. “Awake” will not happen on Tuesday, after Heads, which is a possible result.
Unfortunately I’m not following this part. P(Awake on Tues) is not 1, but that’s not what she’s observing, she’s observing being awake, which she already knew would happen.
That depends entirely on how you evaluate the”bet.”
I pick a random day to approach Beauty with an offer of a bet. If she guesses right, on Wednesday I’ll give her $1.5, otherwise she’ll give me $2. If she truly believes that the coin flipped Tails with a probability of 2⁄3, then she’ll take this bet.
What is so difficult to see about this?
You’re not making yourself clear. Yes, from the outside perspective of a randomly chosen day, there are different die rolls associated with whether Beauty is asleep or not on that day. How does that contradict the math I presented? Would you like to share some Bayesian equations of your own? Did you read my post about the Day 1 Objection, which I strongly suspect is very relevant?
Apologies if that comes off as rude, I’m just finding it difficult to respond usefully to this latest comment. (“What is so difficult to see about this?” isn’t a helpful question—I could just as easily pose the same question back to you!)
Thanks for taking a look! I’ll be sure to go over what you’ve written—I ran out of time to spend any more time on this stuff for this weekend / this week so I won’t be able to reply soon, just wanted to let you know that your response is appreciated
What observation are you trying to update on?
P(G) = 1⁄2
P(waking) = 1
P(waking | G) = 1
P(G | waking) = P(waking | G)*P(G)/P(waking) = 1⁄2
Let’s say she learns it’s Monday. Half the time, she’ll be awake on a random day. Half the time, she’ll be awake on 2⁄3 of days.
P(M) = P(T) = P(W) = (1/2)(1/3) + (1/2)(2/3) = 1⁄2
(Note that this is distinct from the probability that a randomly chosen waking is M/T/W—more on that in a second)
P(M | G) = 1⁄3
P(G | M) = P(M | G)*P(G)/P(M) = (1/3)(1/2)/(1/2) = 1⁄3
Again, no updates are occurring.
What did I mean by “a randomly chosen waking is M/T/W”? An example would be the chance of Beauty receiving a knock on the door, if an outside experimenter flips a coin to decide whether to knock on her door on her first red waking or the second (or the only choice of green waking, if it was green).
The chance that she gets a knock on a Monday would be (1/2)(1/3) + (1/2)(2/3)(1/2) = 1⁄3.
The Thirder mistake is to treat different wakings as a random draw, rather than as sequential events that we know will both happen. (Also the same mistake made on the Doomsday Argument.) This is exactly analogous to the “Day 1 objection” to the Halfer position of the original problem, wherein Beauty learns that it’s Monday—similar to the scenario you’re describing with green and red sides of a dice, but simpler to reason about, so I think it would be better to discuss.
And I break down exactly why that objection fails here:
https://ramblingafter.substack.com/p/sleeping-beauty-meets-two-face
—
Meta-note for anyone else reading this: As far as I’ve seen, the Thirder logic results in credences of expected value that cannot be used without leading to losing bets (and depending on how it’s justified, can lead to more extreme obvious wrong conclusions such as the presumptuous philosopher). Where as I’ve never seen any contradiction or absurdity to arise from the Halver position. I used to be a Thirder myself at one point, and for a long time I was highly uncertain about what the true answer should be. But the more and more I’ve looked at objections to the Halfer position, the more and more I’ve found it to be resilient, and the more I believe there is a true sense in which the Halfer position is correct and the Thirder position is incorrect, when you’re talking about actual belief in the coin flip and not belief in something you know will be tallied more often with more wakings (e.g. guesses-over-repeated-experiments), and the that former is more useful thing to Beauty (i.e., more actionable; not able to be money-pumped) when she’s asked unspecifically, “What do you think the coin flipped?”
>> If coordination is possible, why did they choose to shut down unilaterally?
It makes sense to me: One company self-sacrificing in order to create a Schelling Point that all other companies could rally on.
Though I suppose it doesn’t matter what makes sense to me. Rather, what matters is whether this makes sense to the (very profit driven) board members for these companies.