Perhaps I am just contrarian in nature, but I took issue with several parts of her reasoning.
“What you’re saying is tantamount to saying that you want to fuck me. So why shouldn’t I react with revulsion precisely as though you’d said the latter?”
The real question is why should she react with revulsion if he said he wanted to fuck her? The revulsion is a response to the tone of the message, not to the implications one can draw from it. After all, she can conclude with >75% certainty that any male wants to fuck her. Why doesn’t she show revulsion simply upon discovering that someone is male? Or even upon finding out that the world population is larger than previously thought, because that implies that there are more men who want to fuck her? Clearly she is smart enough to have resolved this paradox on her own, and posing it to him in this situation is simply being verbally aggressive.
“For my face is merely a reflection of my intellect. I can no more leave fingernails unchewed when I contemplate the nature of rationality than grin convincingly when miserable.”
She seems to be claiming that her confrontational behavior and unsocial values are inseparable from rationality. Perhaps this is only so clearly false to me because I frequent lesswrong.
“If it was electromagnetism, then even the slightest instability would cause the middle sections to fly out and plummet to the ground… By the end of class, it wasn’t only sapphire donut-holes that had broken loose in my mind and fallen into a new equilibrium. I never was bat-mitzvahed.”
This seems to show an incredible lack of creativity (or dare I say it, intelligence), that she would be unable to come up with a plausible way in which an engineer (never mind a supernatural deity) could fix a piece of rock to appear to be floating in the hole in a secure way. It’s also incredible that she would not catch onto the whole paradox of omnipotence long before this, a paradox with a lot more substance.
“he eventual outcome would most likely be a compromise, dependent, for instance, on whether the computations needed to conceal one’s rationality are inherently harder than those needed to detect such concealment.”
Whoah, whoah, since when did cheating and catching it become a race of computation? Maybe an arms race of finding and concealing evidence, but when does computational complexity enter the picture? Second of all, the whole section about the Darwinian arms race makes the (extremely common) mistake of conflating evolutionary “goals” and individual desires. There is a difference between an action being evolutionarily advantageous, and an individual wanting to do it. Never mind the whole confusion about the nature of an individual human’s goals (see http://lesswrong.com/lw/6ha/the_blueminimizing_robot/).
One side point is that the way she presents it (“Emotions are the mechanisms by which reason, when it pays to do so, cripples itself”) is essentially presenting the situation as Newcomb’s Paradox, and claiming that emotions are the solution, since her idea of “rationality” can’t solve it on its own.
“By contrast, Type-1 thinking is concerned with the truth about which beliefs are most advantageous to hold.”
But wait… the example given is not about which beliefs are most advantageous to hold… it’s about which beliefs it’s most advantageous to act like you hold. In fact, if you examine all of the further Type-X levels, you realize that they all collapse down to the same level. Suppose there is a button in front of you that you can press (or not press). How could it be beneficial to believe that you should push the button, but not beneficial to push the button? Barring of course, supercomputer Omegas which can read your mind. You’re not a computer. You can’t get a core dump of your mind which will show a clearly structured hierarchy of thoughts. There’s no distinction to the outside world between your different levels of recursive thoughts.
I suppose this bothered me a lot more before I realized this was a piece of fiction and that the writer was a paranoid schizophrenic (the former applying to most else of what I am saying).
“Ah, yet is not dancing merely a vertical expression of a horizontal desire?”
No, certainly not merely. Too bad Elliot lacked the opportunity (and probably the quickness of tongue) to respond.
“But perplexities abound: can I reason that the number of humans who will live after me is probably not much greater than the number who have lived before, and that therefore, taking population growth into account, humanity faces imminent extinction?...”
Because I am overly negative in this post, I thought I’d point out the above section, which I found especially interesting.
But the whole “Flowers for Algernon” ending seemed a bit extreme...and out of place.
The exposition of meta-probability is well done, and shows an interesting way of examining and evaluating scenarios. However, I would take issue with the first section of this article in which you establish single probability (expected utility) calculations as insufficient for the problem, and present meta-probability as the solution.
In particular, you say
I do not believe that this is a failure of applying a single probability to the situation, but merely calculating the probability wrongly, by ignoring future effects of your choice. I think this is most clearly illustrated by scaling the problem down to the case where you are handed a green box, and only two coins. In this simplified problem, we can clearly examine all possible strategies.
Strategy 1 would be to hold on to your two dollar coins. There is a 100% chance of a $2.00 payout
Strategy 2 would be to insert both of your coins into the box. There is a 50.5% chance of a $0.00 payout, 40.5% chance of a $4.00 payout and a 9% chance of a $2.00 payout.
Strategy 3 would be to insert one coin, and then insert the second only if the first pays out. There is a 55% chance of $1.00 payout, a 4.5% chance of a $2.00 payout, and a 40.5% chance of a $4.00 payout.
Strategy 4 would be to insert one coin, and then insert the second only if the first doesn’t pay out. There is a 50.5% chance of a 0.00$ payout, a 4.5% chance of a $2.00 payout, and a 45% chance of a $3.00 payout.
When put in these terms, it seems quite obvious that your choice to open the box would depend on more than the expected payoff from only the first box, because quite clearly your choice to open the first box pays off (or doesn’t pay off) when opening (or not opening) the other boxes as well. This seems like an error in calculating the payoff matrix rather than a flaw with the technique of single probability values itself. It ignores the fact that opening the first box not only pays you off immediately, but also pays you off in the future by giving you information about the other boxes.
This problem easily succumbs to standard expected value calculations if all actions are considered. The steps remain the same as always:
Assign a utility to each dollar amount outcome
Calculate the expected utility of all possible strategies
Choose the strategy with the highest expected utility
In the case of two coins, we were able to trivially calculate the outcomes of all possible strategies, but in larger instances of the problem, it might be advisable to use shortcuts in the calculations. However, it still remains true that the best choice will still be the one you would have gotten if you had done out the full expected value calculation.
I think the confusion arises because a lot of the time problems are presented in a way that screens them off from the rest of the world. For example, you are given a box, and it either has $10.00 or $100.00. Once you open the box, the only effect it has on you is the amount of money you got. After you get the money, the box does not matter to the rest of the world. Problems are presented this way so that it is easy to factor out the decisions and calculations you have to make from every other decision you have to make. However, decision are not necessarily this way (in fact in real life, very few decisions are). In the choice of inserting the first coin or not, this is simply not the case, despite having superficial similarities to standard “box” problems.
Although you clearly understand that the payoffs from the boxes are entangled, you only apply this knowledge in your informal approach to the problem. The failure to consider the full effects of your actions in opening the first box may be psychologically encouraged by the technique of “single probability calculations”, but it is certainly not a failure of the technique itself to capture such situations.