Personal website: https://outsidetheasylum.blog/ Feedback about me: https://www.admonymous.co/isaacking
Isaac King(Isaac King)
Karma: 654
so it includes both asymptomatic cases
I think that “includes” should be “excludes”?
making piece
should be
making peace
“Rational Agents Win”
I don’t find the simulation argument very compelling. I can conceive of many ways for Omega to arrive at a prediction with high probability of being correct that don’t involve a full, particle-by-particle simulation of the actors.
In the case where you find yourself holding the $1,000,000 and the $1000 are still available, sure, you can pick them up. That only happens if either Omega failed to predict what you will do, or if you somehow set things up such that you couldn’t, or had to pay a big price, to break your precommitment.
I don’t think that’s true. The traditional Newcomb’s problem could use the exact setup that I used here, the only difference would be that either the opaque box is empty, or Irene never opens the transparent box. The idea that the $1000 is always “available” to the player is central to Newcomb’s problem.
I don’t find the simulation argument very compelling. I can conceive of many ways for Omega to arrive at a prediction with high probability of being correct that don’t involve a full, particle-by-particle simulation of the actors.
I just did that to be consistent with the traditional formulation of Newcomb’s problem, it’s not relevant to the story. I needed some labels for the boxes, and “box A” and “box B” are not very descriptive and make it easy for the reader to forget which is which.
Some clarifications on my intentions writing this story.
Omega being dead and Irene having taken the money from one box before having the conversation with Rachel are both not relevant to the core problem. I included them as a literary flourish to push people’s intuitions towards thinking that Irene should open the second box, similar to what Eliezer was doing here.
Omega was wrong in this scenario, which departs from the traditional Newcomb’s problem. I could have written an ending where Rachel made the same arguments and Irene still decided against doing it, but that seemed less fun. It’s not relevant whether Omega was right or wrong, because after Irene has made her decision, she always has the “choice” to take the extra money and prove Omega wrong. My point here is that leaving the $1000 behind falls prey to the same “rational agents should win” problem that’s usually used to justify one-boxing. After taking the $1,000,000 you can either have some elaborate justification for why it would be irrational to open the second box, or you could just… do it.
Here’s another version of the story that might demonstrate this more succinctly:
Irene wakes up in her apartment one morning and finds Omega standing before her with $1,000,000 on her bedside table and a box on the floor next to it. Omega says “I predicted your behavior in Newcomb’s problem and guessed that you’d take only one box, so I’ve expedited the process and given you the money straight away, no decision needed. There’s $1000 in that box on the floor, you can throw it away in the dumpster out back. I have 346 other thought experiments to get to today, so I really need to get going.”
Ah, that makes sense.
I think you’re missing my point. After the $1,000,000 has been taken, Irene doesn’t suddenly lose her free will. She’s perfectly capable of taking the $1000; she’s just decided not to.
You seem to think I’m making some claim like “one-boxing is irrational” or “Newcomb’s problem is impossible”, which is not at all what I’m doing. I’m trying to demonstrate that the idea of “rational agents just do what maximizes their utility and don’t worry about having to have a consistent underlying decision theory” appears to result in a contradiction as soon as Irene’s decision has been made.
I think you have to consider what winning means more carefully.
A rational agent doesn’t buy a lottery ticket because it’s a bad bet. If that ticket ends up winning, does that contradict the principle that “rational agents win”?
That doesn’t seem at all analogous. At the time they had the opportunity to purchase the ticket, they had no way to know it was going to win.
An Irene who acts like your model of Irene will win slightly more when omega makes an incorrect prediction (she wins the lottery), but will be given the million dollars far less commonly because Omega is almost always correct. On average she loses. And rational agents win on average.
By average I don’t mean average within a particular world (repeated iteration), but on average across all possible worlds.
I agree with all of this. I’m not sure why you’re bringing it up?
Whether Omega ended up being right or wrong is irrelevant to the problem, since the players only find out if it was right or wrong after all decisions have been made. It has no bearing on what decision is correct at the time; only our prior probability of whether Omega will be right or wrong matters.
An open-ended probability calibration test is something I’ve been planning to build. I’d be curious to hear your thoughts on how the specifics should be implemented. How should they grade their own test in a way that avoids bias and still gives useful results?
I think you’re confusing ignorance with other people’s beliefs about that agent’s ignorance. In your example of the police or the STD test, there is no benefit gained by that person being ignorant of the information. There is however a benefit of other people thinking the person was ignorant. If someone is able to find out whether they have an STD without anyone else knowing they’ve had that test, that’s only a benefit for them. (Not including the internal cognitive burden of having to explicitly lie.)
Question Gravity
Wouldn’t an observed mismatch between assigned probability and observed probability count as Bayesian evidence towards miscalibration?
List of Probability Calibration Exercises
This looks super neat, thank you for sharing. I just did a quick test and can confirm that it is in fact riddled with bugs. If it would help, I can write up a list of what needs fixing.
Questions about a topic that I don’t know about result in me just putting the max entropy distribution on that question, which is fine if it’s rare, but leads to unhelpful results if they make up a large proportion of all the questions. Most calibration tests I found pulled from generic trivia categories such as sports, politics, celebrities, science, and geography. I didn’t find many that were domain-specific, so that might be a good area to focus on.
Some of them don’t tell me what the right answers are at the end, or even which questions I got wrong, which I found unsatisfying. If there’s a question that I marked as 95% and got wrong, I’d like to know what it was so that I can look into that topic further.
It’s easiest to get people to answer small numbers of questions (<50), but that leads to a lot of noise in the results. A perfectly calibrated human answering 25 questions at 70% confidence could easily get 80% or 60% of them right and show up as miscalibrated. Incorporating statistical techniques to prevent that would be good. (For example, calculate the standard deviation for that number of questions at that confidence level, and only tell the user that they’re over/under confident if they fall outside it.) The fifth one in my list above does something neat where they say “Your chance of being well calibrated, relative to the null hypothesis, is X percent”. I’m not sure how that’s calculated though.
This is an interesting question, but I think your hypothesis is wrong.
If it is indeed possible to build some sort of simple and robust “area clearer” pattern as the other comments discuss (related post here), then nothing approaching the complexity of an AI is necessary. Simply initialize the small region with the area clearer pattern facing outwards and a smiley face constructer behind it.
It seems to me that the only reason you’d need an AI for this is if that sort of “area clearer” pattern is not possible. In that case, the only way to clear the large square would be to individually sense and interact with each piece of ash and tailor a response to its structure, and you’d need something somewhat “intelligent” to do that. Even in this case, I’m unconvinced that the amount of “intelligence” necessary to do this would put it on par with what we think of as a general intelligence.