Thanks for the beauty, it feels good. Some thinking out loud. I can’t help but feel that the key is in the successive layers of maps and territories : maths is (or contains) the map of which physics is the territory, physics is the map of which ‘the real world’ is the territory, ‘the real world’ is the map our brains create from the sensory input concerning the territory which is the ‘play of energies’ out there, while that in itself is another map. Antony Garrett Lisi’s proposal, as an example, would be the most elegant meta-map yet. What these maps have in commmon is : being created by the human brain, a wet lump of nervous tissue comprising ad-hoc purpose specific modules. It has specific ways of making maps, so small wonder all these layers of maps are coherent. Now if the ‘mathematics’ layer of maps has unforeseen and self-consistent properties, it could be just a manifestation of the nature of our map-making modules : they are rules driven. So, is the Universe a geometric figure corresponding to a Lie E8 group, or does that just happen to be the way the human brain is built to interpret things ?
tcpkac
“only because our fundamental theory tells us quite definitely that different versions of us will see different results”.
EY, on what do you base your ‘quite definitely’ ? David Lewis ?
OK thanks, nice intuition pump.
I just saw an incredibly beautiful sunset. I also see the beauty in some of EY’s stuff. Does that mean the sunset was Bayesian, or indeed subject to underlying lawfulness ? No, it only means my enhanced primate brain has a tendency to see beauty in certain things. Not that there is any more epistemic significance in a sunset than there is in a theorem.
When one got past pre-adolescence, one realised that Heinlein’s writing skills, such as they were, were in the service of a political philosophy somewhat to the right of Attila the Hun. Whatever floats your boat.
Charlie (Colorado), I’d appreciate your thoughts on the difference between ‘hard core libertarian’ and ‘right wing’. For me they map to pretty much the same territory, obviously not for you.
Just to respond to the theme that ‘right wing’ is a meaningless label, not so. It originally arose from the seating arrangements in the French Assembly, where the right wing were the monarchists. Hence right wing became generally accepted as a label for the authoritarian defence of a monarchic, aristocratic, or oligarchic power structure. As these power structure tended to be the ones in place, you have the confusion with Conservatism (e.g. Torys). By a further semantic slide, it came, for some, to mean any authoritarian power structure with power concentrated in the hands of the few, hence the lumping together of the various 20thC dictatorships as right wing. For those who conceive the power of ‘Big Business’ to be oligarchic and oppressive, any political program favourising the large corporations is right wing. One source of confusion between ‘right wing’ and Libertarianism comes from the disingenuous protests that any politics which limit the power of the corporate world are ‘attacking free enterprise’ thus, attacking individual freedom. This is compounded by the myths attached to the notion of private property, where ‘mine’ as in ‘my log cabin and my boots’ is extended to ‘my corporation over which I have Regalian powers’ simply because I invested some bucks in it 30 years ago. Libertarianism as described here seems to be a peculiarly American movement, which would map somewhat but not completely to the European anarchists. Finally, of course individual politics are multi-dimensional. However, all countries which aren’t dictatorships seem to end up with two party systems, so all those dimensions have to projected down, hopefully on a ‘best-fit’ basis, to the single axis most appropriate to the country in question.
Risk aversion, and the degree to which it is felt, is a personality trait with high variance between individuals and over the lifespan. To ignore it in a utility calculation would be absurd. Maurice Allais should have listened to his homonym Alphonse Allais (no apparent relation), humorist and theoretician of the absurd, who famously remarked “La logique mène à tout à condition d’en sortir”. Logic leads to everything, on condition it don’t box you in.
James Bach, your point and EY’s are not incompatible : it is a given that what you care about and give importance to is subjective and irrational, however having chosen what outcomes you care about, your best road to achieving them must be Bayesian.… perhaps. My problem with this whole Bayesian kick is that it reminds me of putting three masts and a full set of square-rigged sails on what is basically a canoe : the masts and sails are the Bayesian edifice, the canoe is our useful knowledge in any given real life situation.
Ben Jones, and Patrick (orthonormal), if you offer me 400$ I’ll say ‘yes, thank you’. If you offer me 500$ I’ll say ‘yes, thank you’. If, from whatever my current position is after you’ve been so generous, you ask me to choose between “a certain loss of $100 or a 20% chance of losing $200”, I’ll choose the 20% chance of losing 200$. That’s my math, and I accept money orders, wire transfers, or cash....
Put baldly, the main underlying question is : how do you compare the value of (a) a unit of work expended now, today, on the well-being of a person alive, now, today, with the value of (b) the same unit of work expended now, today, for the well-being of 500 potential people who might be alive in 500 years’ time, given that units of work are in limited supply. I suspect any attempt at a mathematical answer to that would only be an expression of a subjective emotional preference. What is more, the mathematical answer wouldn’t be a discount function, it would be a compounding function, as it would be the result of comparing all the AI units of work available between now and time t in the future, with the units of work required between now and time t to address all the potential needs of humanity and trans-humanity between now and the end of time, which looks seriously like infinity.
To get back to the ‘human life’ examples EY quotes. Imagine instead the first scenario pair as being the last lifeboat on the Titanic. You can launch it safely with 40 people on board, or load in another 10 people, who would otherwise die a certain, wet, and icy death, and create a 1 in 10 chance that it will sink before the Carpathia arrives, killing all. I find that a strangely more convincing case for option 2. The scenarios as presented combine emotionally salient and abstract elements, with the result that the emotionally salient part will tend to be foreground, and the ‘% probabilities’ as background. After all no-one ever saw anyone who was 10% dead (jokes apart).
The answer to ‘shut up and multiply’ is ‘that’s the way people are, deal with it’. One thing apparent from these exchanges is that ‘inferential distance’ works both ways.
Great New Theorem in color perception : adding together 10 peoples’ perceptions of light pink is equivalent to one person’s perception of dark red. This is demonstrable, as there is a continuous scale between pink and red.
An AGI project would presumably need a generally accepted, watertight, axiom based, formal system of ethics, whose rules can reliably be applied right up to limit cases. I am guessing that that is the reason why Eliezer et al are arguing from the basis that such an animal exists.
If it does, please point to it. The FHI has ethics specialists on its staff, what do they have to say on the subject ?
Based on the current discussion, such an animal, at least as far as ‘generally accepted’ goes, does not exist. My belief is that what we have are more or less consensual guidelines which apply to situations and choices within human experience. Unknown’s examples, for instance, tend to be ‘middle of the range’ ones. When we get towards the limits of everyday experience, these guidelines break down.
Eliezer has not provided us with a formal framework within which summing over single experiences for multiple people can be compared to summing over multiple experiences for one person. For me it stops there.
A Utilitarian should care about the outcomes of Utilitarianism..… and yes, as soon as ends justify means, you do get Stalin, Mao, Pol Pot, who were all striving for good consequences...… Which is relevant, as your arguments mostly involve saving lives (a single type of outcome, so making options intuitively comparable). I’m afraid the ‘rant’ doesn’t add much in terms of content, that I can see.
Eliezer, I don’t read the main thrust of your post as being about Newcomb’s problem per se. Having distinguished between ‘rationality as means’ to whatever end you choose, and ‘rationality as a way of discriminating between ends’, can we agree that the whole specks / torture debate was something of a red herring ? Red herring, because it was a discussion on using rationality to discriminate between ends, without having first defined one’s meta-objectives, or, if one’s meta-objectives involved hedonism, establishing the rules for performing math over subjective experiences. To illustrate the distinction using your other example, I could state that I prefer to save 400 lives certainly, simply because the purple fairy in my closet tells me to (my arbitrary preferred objective), and that would be perfectly legitimate. It would only be incoherent if I also declared it to be a strategy which would maximise the number of lives saved if a majority of people adopted it in similar circumstances (a different arbitrary preferred objective). I could in fact have as preferred meta-objective for the universe that all the squilth in flobjuckstooge be globberised, and that would be perfectly legitimate. An FAI (or a BFG, for that matter (Roald Dahl, not Tom Hall)) could scan me and work towards creating the universe in which my proposition is meaningful, and make sure it happens. If now someone else’s preferred meta-objective for the universe is ensuring that the princess on page 3 gets a fairy cake, how is the FAI to prioritise ?
Under Multiple Worlds, aren’t you condemned, whatever you do or don’t do, to there being a number tending to infinity of worlds where what you want to protect is protected, and a number tending to infinity where it is not ?
Hollerith, if ‘most psychologists are idiots’, I wonder how they discovered all the cognitive biases ?
The nature of 0 & 1 as limit cases seem to be fascinating for the theorists. However, in terms of ‘Overcoming Bias’, shouldn’t we be looking at more mundane conceptions of probability ? EY’s posts have drawn attention to the idea that the amount of information needed to add additional cetainty on a proposition increases exponentially while the probability increases linearly. This says that in utilitarian terms, not many situations will warrant chasing the additional information above 99.9% certainty (outside technical implementations in nuclear physics, rocket science or whatever). 99.9% as a number is taken out of a hat. In human terms, when we say ‘I’m 99.9% sure that 2+2 always =4’, where not talking about 1000 equivalent statements. We’re talking about one statement, with a spatial representation of what ’100% sure’ means with respect to that statement, and 0.1% of that spatial representation allowed for ‘niggling doubts’, of the sort : what have I forgotten ? What don’t I know ? What is inconceivable for me ? The interesting question for ‘overcoming bias’ is : how do we make that tradeoff between seeking additional information on the one hand and accepting a limited degree of certainty on the other ? As an example (cf. the Evil Lords of the Matrix), considering whether our minds are being controlled by magic mushrooms from Alpha Pictoris may someday increase the ‘niggling doubt’ range from 0.1% to 5%, but the evidence would have to be shoved in our faces pretty hard first.