Great New Theorem in color perception : adding together 10 peoples’ perceptions of light pink is equivalent to one person’s perception of dark red. This is demonstrable, as there is a continuous scale between pink and red.
tcpkac
To get back to the ‘human life’ examples EY quotes. Imagine instead the first scenario pair as being the last lifeboat on the Titanic. You can launch it safely with 40 people on board, or load in another 10 people, who would otherwise die a certain, wet, and icy death, and create a 1 in 10 chance that it will sink before the Carpathia arrives, killing all. I find that a strangely more convincing case for option 2. The scenarios as presented combine emotionally salient and abstract elements, with the result that the emotionally salient part will tend to be foreground, and the ‘% probabilities’ as background. After all no-one ever saw anyone who was 10% dead (jokes apart).
Just to respond to the theme that ‘right wing’ is a meaningless label, not so. It originally arose from the seating arrangements in the French Assembly, where the right wing were the monarchists. Hence right wing became generally accepted as a label for the authoritarian defence of a monarchic, aristocratic, or oligarchic power structure. As these power structure tended to be the ones in place, you have the confusion with Conservatism (e.g. Torys). By a further semantic slide, it came, for some, to mean any authoritarian power structure with power concentrated in the hands of the few, hence the lumping together of the various 20thC dictatorships as right wing. For those who conceive the power of ‘Big Business’ to be oligarchic and oppressive, any political program favourising the large corporations is right wing. One source of confusion between ‘right wing’ and Libertarianism comes from the disingenuous protests that any politics which limit the power of the corporate world are ‘attacking free enterprise’ thus, attacking individual freedom. This is compounded by the myths attached to the notion of private property, where ‘mine’ as in ‘my log cabin and my boots’ is extended to ‘my corporation over which I have Regalian powers’ simply because I invested some bucks in it 30 years ago. Libertarianism as described here seems to be a peculiarly American movement, which would map somewhat but not completely to the European anarchists. Finally, of course individual politics are multi-dimensional. However, all countries which aren’t dictatorships seem to end up with two party systems, so all those dimensions have to projected down, hopefully on a ‘best-fit’ basis, to the single axis most appropriate to the country in question.
To summarise : A storm in a teacup between a pot and a kettle.
Reactions to 500lb stripy feline things jumping unexpectedly come from pre-verbal categorisations(the ‘low road’, in Daniel Goleman’s terms), so have nothing to do with word definitions. The same is true for many highly emotionally charged categorisations (e.g. for a previous generation, person with skin colour different from mine....). Words themselves do get their meanings from networks of associations. The content of these networks can drift over time, for an individual as for a culture. Words change their meanings. A deliberate attempt to change the meaning of a word by introducing new associations (e.g. via the media) can be successful. Changes in the meanings of political labels, or the associations with a person’s name, are good examples. Whether the direct amygdala circuit can be reprogramed is a different matter. Certainly not as easily as the neocortex. If you lived in the world of Calvin and Hobbes for six months, would you start to instinctively see large stripy feline things jumping out at you unexpectedly as an invitation to play ?
The answer to ‘shut up and multiply’ is ‘that’s the way people are, deal with it’. One thing apparent from these exchanges is that ‘inferential distance’ works both ways.
You’ve forgotten one important caveat in the phrase “And the way to carve reality at its joints, is to draw your boundaries around concentrations of unusually high probability density in Thingspace.” The important caveat is : ‘boundaries around where concentrations of unusually high probability density lie, to the best of our knowledge and belief’ . All the imperfections in categorisation in existing languages come from that limitation. Other problems in categorisation, like those of Antonio, in ‘Merchant of Venise’, or those of the founding fathers who wrote that it is ‘self evident that all men were created equal’ but at the same time were slave owners, do not come from language problems in categorisation, they would have acknowledged that Shylock or the slaves were human, but from different types of cognitive compromise. Apart from that, it’s an intellectually satisfying approach, and you might, if you persevere, end up with a poor relation to an existing language. Why a poor relation ? because it would lack nuance, ambiguity, and redundance, which are the roots of poetry. It would also lack words for the surprising but significant improbable phenomenon. Like genius, or albino. Then again, once you get around to saying you will have words for significant low hills of probability, the whole argument blows away. Bon courage.
I’ll second Frank Hirsch’s comment and add one point. I don’t get this obsession with ‘dictionary definitions’ either. An etymological dictionary is endlessly fascinating precisely because it shows you the evolution of thought processes, concepts, and word usages, in action. Very much the opposite of the sort of table thumping that dictionaries are here supposed to give rise to. Eliezer’s examples seem to be taken from a pretty toxic discussion environment
Excellent post, however, “But people often don’t realize that their argument about where to draw a definitional boundary, is really a dispute over whether to infer a characteristic shared by most things inside an empirical cluster...” Indeed so, but there are other aspects. Humans also have obsessions with (a) how far your cluster is from mine (kinship or the lack of it) (b) given one empirical cluster, how can I pick a characteristic, however minor, which will allow me to split it into ‘us vs them’ (Robber’s Cave). So when you get to discussing whether an uploaded human brain is part of the cluster ‘human’, those are the considerations which will be foremost.
An AGI project would presumably need a generally accepted, watertight, axiom based, formal system of ethics, whose rules can reliably be applied right up to limit cases. I am guessing that that is the reason why Eliezer et al are arguing from the basis that such an animal exists.
If it does, please point to it. The FHI has ethics specialists on its staff, what do they have to say on the subject ?
Based on the current discussion, such an animal, at least as far as ‘generally accepted’ goes, does not exist. My belief is that what we have are more or less consensual guidelines which apply to situations and choices within human experience. Unknown’s examples, for instance, tend to be ‘middle of the range’ ones. When we get towards the limits of everyday experience, these guidelines break down.
Eliezer has not provided us with a formal framework within which summing over single experiences for multiple people can be compared to summing over multiple experiences for one person. For me it stops there.
Charlie (Colorado), I’d appreciate your thoughts on the difference between ‘hard core libertarian’ and ‘right wing’. For me they map to pretty much the same territory, obviously not for you.
Thanks for the beauty, it feels good. Some thinking out loud. I can’t help but feel that the key is in the successive layers of maps and territories : maths is (or contains) the map of which physics is the territory, physics is the map of which ‘the real world’ is the territory, ‘the real world’ is the map our brains create from the sensory input concerning the territory which is the ‘play of energies’ out there, while that in itself is another map. Antony Garrett Lisi’s proposal, as an example, would be the most elegant meta-map yet. What these maps have in commmon is : being created by the human brain, a wet lump of nervous tissue comprising ad-hoc purpose specific modules. It has specific ways of making maps, so small wonder all these layers of maps are coherent. Now if the ‘mathematics’ layer of maps has unforeseen and self-consistent properties, it could be just a manifestation of the nature of our map-making modules : they are rules driven. So, is the Universe a geometric figure corresponding to a Lie E8 group, or does that just happen to be the way the human brain is built to interpret things ?
We do not know that the territory is single- level. It is conceivable that it is not, and the available evidence does not exclude the possibility.
The territory is single level...… BY DEFINITION ….… waaaahahahahahahahahahaha !!!!!
Psychoh, do not despair. Remember : “The real challenge can be played as a single-player game, without speaking aloud.”. We are looking for the natural joints of reality, and that is a purely subjective assessment. Every single pair of phenomena in the Universe can be the subject of a natural join if the difference in one of their attributes happens to be a salient division for you. So draw the line around Christmas any way you want, just like you can draw the line around ‘food things living in the sea’ any way which is relevant to your way of fishing. Just don’t speak it aloud.
Silas, billswift, Eliezer does say, introducing his diagrams in the Neural Categories post : “Then I might design a neural network that looks something like this:”
The primary categorisation is “Threat / Not a threat”, and the main categorisation bias is “Better safe than sorry”. You’ll find that many of your specific categorisation biases are particular examples of that. Examples are : nervousness about your Great Thing being a cult, Asch experiment situations where you have to join the group or stick out from it. Diagram 1b has ‘Threat’ written all over it.....
When one got past pre-adolescence, one realised that Heinlein’s writing skills, such as they were, were in the service of a political philosophy somewhat to the right of Attila the Hun. Whatever floats your boat.
OK thanks, nice intuition pump.
Ben, Rolf, no problem, I just thought that ‘people who look at dictionnaries’ was starting to be a category subject to sneaky connotations.. :)
While we’re staking out the new language, I want a word for red flowers, because I like red flowers, and that is much more important to me than their genotype or taxonomy. Also, I want a special word for slightly-out-of-focus photos, which is a very important category for reasons I’m not at liberty to disclose. The joints of reality are articulated in a rather large number of dimensions. Carving it correspondingly is going to need one heck of a …. dictionary.