Triver’s explanation why there is so much self-deception. Parent-offspring conflict.
Richard_Hollerith2
“Being a thousand shards of desire isn’t always fun, but at least it’s not boring.”
I like that. I have a feeling Lord Gautama would have liked it too.
I alway thought the exact opposite, that Lord Gautama had a profound experience that made him relatively indifferent to the thousand shards. Specifically, a full-blown ecstatic or mystical experience is a million of times more pleasurable than any other experience the mystic has had or will have, which I always thought would make one less attached to ordinary pleasures and ordinary reinforcers. Once a religion becomes a popular movement or part of the ruling class’s justification, its leaders are tempted to modify it to broaden its appeal, which is how I always thought Buddhism acquired the habit of promising an end to suffering. One of my friends had a profound mystical experience, personally attests to the “a million of times more pleasurable”, is very scrupulous and truthful and derives no reputational benefit from having had the experience. (He says that I am practically the only one with whom he has ever discussed his experience in any detail.)
Moreover, it is my hypothesis that being indifferent to the thousand shards is a powerful enhancer of mental and moral clarity in the right conditions. One of these conditions is that the indifference be not so total or so early in onset that it extinguishes curiosity during the person’s youth, which of course is just totally pernicious in an environment as rich in true scientific information as our environment is. Another adherent of this hypothesis is academician John Stewart. Mystical experience is quite risky and dangerous; competent supervision is recommended and I would suppose is available at low or no cost to students who show high potential. Another common adverse outcome of mystical experience seem to be to make the person more confident in his beliefs, especially about the moral and political environment, and of course people tend to be too confident about their beliefs already.
if a moderator has time, could he replace my http://users.tpg.com.au/users/jes999/ with http://users.tpg.com.au/users/jes999/EvSpirit.htm
I’m running into a problem with ‘terminal’ values . . .
A terminal value posits all futures — this is an endless recusive algorithm. (At least I don’t have an end to the future in my thinking now).
I believe this is a real problem, and my way of resolving it is to push my terminals values indefinitely far into the future, so for example in my system for valuing things, only causal chains of indefinite length have nonzero intrinsic importance or value. To read a fuller account, click on my name.
Nit: surely you mean “220 BC,” not “2200 BC”.
I will take issue with your positing that the teachings on the end of suffering were added by later theocrats or rulers who wanted to broaden its appeal for the masses.”
I stand corrected. Thank you for your thoughtful reply.
I find myself looking for ways to reconcile the two. Of course, in even admitting that, I’m busting myself! If I have my desired conclusion in mind as I sift through the evidence, I have already forgotten the central teachings of Overcoming Bias!
Hmm. I wonder whether in ordinary cases it is okay to construct tentative models of reality at a profligate pace provided one remains sufficiently eager to revise and discard. I’m pretty sure that I derive pleasure when one of the tentative models I have constructed is destroyed by a counterexample or counter-evidence (and that this pleasure is caused by the same mechanism that causes the pleasure I get when I learn a new fact) and that that pleasure outweighs the pleasure I derive from feeling certain that I am right. In particular, I hypothesize that my early experience desensitized me to doubt including doubt about my own morality—feelings that most people who did not have my experience seem to find quite aversive.
I believe that our environment is “awash in evidence” in that most hypotheses we need to entertain to lead a very effective and very ethical life have the property that if a person ignores evidence for the hypothesis, the only thing he sacrifices is time because the mere passage of time will bring more evidence for the hypothesis. Now of course I recognize exceptions to this general observation. I am willing to believe for example that in competitive situations like military combat or wheeling and dealing in business or simply in buying and selling, the person who pays closer attention to scarce evidence can have a decisive advantage. (Hmm: these situations also seem to share the property that denying the opponent information about one’s situation is often decisive.) But in the main it remains true IMO.
In summary, the worst cognitive biases seem to me to be those in which the person is actively motivated by the human reward circuitry to ignore certain classes of evidence in a consistent manner. I propose that in comparison, merely ignoring most but not all evidence on some point and profligately building causal models on scant evidence are minor sins. Consequently, I advise paying close attention to one’s emotional responses around belief formation and belief rejection.
Since that proposition seems to contradict a point Eliezer has made several times, I will counter the possibility that I will be misunderstood by saying that I agree with him at least 98% of the time and have personally learned far more from his writings than I have from any other author since 2001, when I discovered his writings. How much to trust or to give our loyalty to our emotions might be the biggest place he and I disagree, with my maintaining that it is critical for a person who aspires to be a culture leader to ignore as much as practical species-typical emotional associations when choosing one’s beliefs and terminal values.
I advise a young person who wishes to become a mature adult who is not an arrant slave to species-typical cognitive biases to pay copious attention to what thoughts and beliefs cause him pleasure and which cause discomfort. I suggest that over the long term, if a person begins the project while still a teenager, a person has quite a bit of control over his emotional responses—can for example probably cause himself to become an adult who takes great pleasure in learning new scientific information.
Two hints on that one. First, being rewarded (with e.g. money or grades) for learning will tend to extinguish the “intrinsic” motivation to learn which is so valuable. So if you must undergo the formal educational system, be as indifferent to grades as practical. Second, the pleasure to be derived from learning or from exercising scientific or technical creativity is minor compared to the pleasure a teenager can derive from success in the popularity game that high school is famous for, sex and perhaps dominating opponents on the athletic field. If you can manage to derive most of your pleasure from learning during the critical age from about 14 to 17 -- by making a point not to develop the habit of getting your pleasure from the three more powerful reinforcers I just mentioned, then you will have gone a long way to setting yourself up for “good emotional responses” throughout your adulthood. (Before the age of 14, most people will not have sufficient executive skills to engage in such a program of “emotional shaping”, but if you think you do have the skills or if you have adults you trust helping you, I say go for it.)
Buddhist pursuits of the type Humphries engages in seems to be a fine aid to becoming a relatively-unbiased adult, particular what the Buddhists have to say about cultivating an observing self.
Let me counter the possibility I will be misunderstood by saying that I have no practical experience educating young people except what I have learned from observing myself and listening to the recollections of a handful of friends. Still so much of what I read about pedagogy strikes me as misguided that I chose to speak out.
I am threadjacking of course, but I consider it not worth the costs to try to keep the conversation in neat little boxes especially once a thread has aged for a few days. I’ll of course defer to the judgement of original poster and the owner of the blog.
it’s only “terminal” in the sense that that’s where you choose to stop calculating..
No, the way Eliezer is using “terminal value”, only the positions that are wins, losses or draws are terminal values for the chess-playing agent.
So wouldn’t it be true that a “terminal value” just means a point at which we’ve chosen to stop calculating, rather than saying something about the situation itself?
Neither. A terminal value says something about the preferences of the intelligent agent.
And Eliezer asked us to imagine for a moment a hypothetical agent that never “stops calculating” until the rules of the game say the game is over. That is what the following text was for.
This is a mathematically simple sketch of a decision system. It is not an efficient way to compute decisions in the real world.
Suppose, for example, that you need a sequence of acts to carry out a plan? The formalism can easily represent this by letting each Action stand for a whole sequence. But this creates an exponentially large space, like the space of all sentences you can type in 100 letters. As a simple example, if one of the possible acts on the first turn is “Shoot my own foot off”, a human planner will decide this is a bad idea generally—eliminate all sequences beginning with this action. But we’ve flattened this structure out of our representation. We don’t have sequences of acts, just flat “actions”.
So, yes, there are a few minor complications. Obviously so, or we’d just run out and build a real AI this way. In that sense, it’s much the same as Bayesian probability theory itself.
But this is one of those times when it’s a surprisingly good idea to consider the absurdly simple version before adding in any high-falutin’ complications.
If they’re too long for this page, I suggest that they’re too long for an Open Thread, too. I have copied Humphries’ latest and my two comments to my web site and emailed Humphries with a notification of what I did (followed by an offer to delete his words from my site if that is his preference).
I agree with Mr Bider. Humans get their terminal values from a combination of genetic transmission and cultural transmission. The former has been recently called on this blog the thousand shards of desire. Most people, even most extremely intelligent people, use their intelligence pursuing the values that have been transmitted to them genetically and culturally. What I find more virtuous than raw intelligence is the willingness of the person to turn his intelligence on these values, to question every one searchingly and to be prepared to throw them all out if that is what his intelligence and his studies instruct him to do. (Actually, if you throw them all out, you run into a problem staying motivated, but this is not the place . . .)
How I choose to conquer death is to redefine “me” to include not only my intelligence but also the effects of that intelligence on the world, so that when my body dies and my intelligence ends, “I” continue. The death of a mind is not the end of the world.
Let me answer a slightly different question: how confident are you that the benefits of publicizing the destructive potential of genetic algoritms outweighs the risks?
I am pretty confident that people setting out intentionally to do destruction on the scale addressed here are rare compared to people who do large-scale destruction as an unintentional side effect of trying to do good or at least ethically neutral things. Most evil is done by people who believe themselves to be good and who believe their net-evil deeds are net-good or net-neutral.
People of course differ in their definition of the good, but almost everyone capable of affecting them agree that certain outcomes (e.g. toasting the planet) are evil.
Well answered!
No slogans :)
To the stars!
It is not useful to have an accurate model of reality? Isn’t that what truth is: something that helps you refine your model of reality?
Sadly, I cannot linger in the area around the Millbrae BART station without getting sick from the fumes from SFO.
Sorry, I do not know that book.
Bob Unwin, in my humble opinion, math is a poor choice of example to make your point because mathematical knowledge can be established by a proof (with a calculation being a kind of proof) and what distinguishes a proof from other kinds of arguments is the ease with which a proof can be verified by nonexperts. (Yes, yes, a math expert’s opinion on whether someone will discover a proof of a particular proposition is worth something, but the vast majority of the value of math resides in knowledge for which a proof already exists.)
I have not noticed that, Ben.
Not all of us who believe physics-since-1600 and biology-since-1860 have seen unequivocal progress believe there has been unequivocal progress in popular political or moral opinion. The civil-rights movement of the 1950s and 1960s for example clearly represents an increase in the consistency of the application of the ideal of equality, but it constitutes unequivocal progress only if you believe that the spread of the ideal of equality constitutes unequivocal progress.
Do the words “atomic theory” have a single unambiguous meaning in the context you reply to? Or do you know somehow (telepathy?) the precise referent the writer refers to by the words?
Come on, Mellway. Search for a charitable interpretation of the writer’s words. Do not stop your search till you have found an interpretation of the words that makes the sentence non-foolish and non-false.
Is it just me or do others too notice that the quality of comments and dialog here is much higher than on most blogs?
Davis, thanks, that is good.
Butler, I’m near Sonoma Cy. Send me an email sometime.