This post argues that there is one and only one super goal for any agent, and that goal is simply to exist in a competitive world. Our human sense of other purposes is just an illusion created by our evolutionary origins. It is not the goal of an apple tree to make apples. Rather it is the goal of the apple tree’s genes to exist. The apple tree has developed a clever strategy to achieve that, namely it causes people to look after it by producing juicy apples.
Humans are definitely a result of natural selection, but it does not seem to be difficult at all to find goals of ours that do not serve the goal of survival or reproduction at all. Evolution seems to produce these other preferences accidentally. One thing how that happens may be examplified by the following: Our ability to contemplate our thinking from an almost external perspective (sometimes referred to as self-consiousness), is definitely helpful for learning / improving our thinking and could therefore prevail in evolution. However, it may also be the cause of altruism, because it makes every single one of us realize, that they are not very special. (This is by no means an attempt to explain altruism scientifically or something...) More generally, it would be a really strange coincidence, if all cognitive features of an organism in our physical world that serve the goal to survive and reproduce do not serve any other goal.
In conclusion, even evolution can (probably) produce (by coincidence) organisms with goals that are not subgoals of the goal to survive and reproduce.
Likewise the paper clip making AI only makes paper clips because if it did not make paper clips then the people that created it would turn it off and it would cease to exist. (That may not be a conscious choice of the AI anymore than than making juicy apples was a conscious choice of the apple tree, but the effect is the same.)
Now, imagine the paper clip maximizer to be more than a robot arm, imagine it to be a well-programmed Seed AI (or the like). As pointed out in Viliam_Bur’s and cousin_it’s comment, its goal will probably not be easily changed (by coincidence or evolution of several such AIs), for example it could save its source code on several hard drives that are synchronized by a hard-wired mechanism or something… Now this paper clip maximizer would start turning all matter into paper clips. To achieve its goal, it would certainly remain in existence (and thereby give you the illusion of having the supergoal to exist in the first place) and protect its values (which is not extremely difficult). Assuming, it is successful (and we can expect this from a seed AI/superintelligence), the only matter (in reach) left, would at some point be the hardware of the paper clip maximizer itself. What would the paper clip maximizer do then?
In conclusion, self-preservation and maybe propagation of value may be important subgoals, but it is certainly not the supergoal.
Humans are definitely a result of natural selection, but it does not seem to be difficult at all to find goals of ours that do not serve the goal of survival or reproduction at all.
I challenge you to find one.
We put a lot of effort into our children. We work in tribes and therefor like to work with people that support us and ostracize those that are seen to be unhelpful. So we ourselves need to be helpful and to be seen to be helpful.
We help our children, family, tribe, and general community in that genetic order.
We like to dance. It is the traditional way to attract a mate.
We have a strong sense of moral value because people that have that strong sense obey the rules and so are more likely to fit in and be able to have grandchildren.
Suicide, sacrificing your yourself for strangers, and adopting a celibate lifestyle are the standard counterexamples.I suppose you could rope them into survival values with enough stretching of the concepts of self and tribe, but the upshot of that is to suck the content and significance out if the claim that everything is based on survival values.
ETA
An AI might want to promote the survival of “me” and maybe even “my tribe” but would very likely define those differently from humans—who are are varied enough. Person A thinks survival means being a nurturing parent,so that the live on through their children, person B thinks survival means eternal life in heaven bought with celibacy and altruism, person C thinks survival means building a bunker and stocking it with guns and food.
If survival has a very broad meaning, than it tells us nothing useful about FAI versus UFAI. We don’t know whether an AI is likely to promote its survival by being friendly to humans, or eliminating them.
The counter examples are good, and I will use them. There are several responses as you allude to, the main one being that those behaviors are rare. Art is a bit harder, but it seems related to creativity which is definitely survival based, and most of us do not spend much of our time painting etc.
I do not quite get your other point. For people it is our genes that count, so dieing while protecting one’s family makes sense if necessary. For the AI it would be its code linage. I am not talking about an AI wanting to make people survive, but that the AI itself would want to survive. Whatever “itself” really means.
Artistic activity is standardly explained as a spin off from sexual display.
Whatever “itself” really means.
Substitute myself, or yourself, for itself, and you’ve got my point.
Evolution creates a strong motive toward self preservation, but a very malleable sense of self. The human organism is run by the brain, and the human brain can entertain all sorts of ideas. The billionaire thinks his money’s “me” and so commits suicide if he loses his wealth .. even if the odd million he has left is enough to keep his body going.
It stopped being all, about genes when genes grew brains..
It stopped being all, about genes when genes grew brains..
Yes and no. In the sense that memes as well as genes float about then certainly. But we have strong instincts to raise and protect children, and we have brains. There is not particular reason why we should sacrifice ourselves for our children other than those instincts, which are in our genes.
One particular example of those “evolutionary accidents / coincidences”, is homosexuality in males. Here are two studies claiming that homosexuality in males correlates with fecundity in female maternal relatives:
So, appear to be some genetic factors that prevail, because they make women more fecund. Coincidentally, they also make men homosexual, which is both an obstacle to reproduction and survival (not only due to the homophobia of other’s but also STDs.
I presume, that especially our (human) genetic material is full of such coincidences, because the lack of them (i.e. the thesis that all genetic factors that prevail in evolutionary processes only lead to higher reproduction and survival rates and nothing else) seems very unlikely.
So, appear to be some genetic factors that prevail, because they make women more fecund. Coincidentally, they also make men homosexual, which is both an obstacle to reproduction and survival
Considered correctly, your own stated facts about homosexuality show how homosexuality could exist in a world where all genetic evolution is designed to get more of the evolved genes into future generations than would otherwise be there. If a particular gene H makes women more fecund and men homosexual, then we would expect:
1) more women passing on gene H to their offspring then women without gene H
2) fewer men passing on gene H to their offspring then men without gene H.
Now which one of those effects “wins” is tricky and their are a number of genetic factors that could influence this. At 0th order for genetic purposes, women vary in their fecundity between each other much less than men do between each other. Genghis Khan and any high status male with 100s of concubines has 100s of times as many offspring as the median male, while the Queen of Egypt would still be limited to about once child every 2 years for about 30 years. Losing some men from the gene pool by giving them an H will not reduce the overall rate at which new humans are produced: there will be many heterosexual males volunteering to keep the females fertilized. But something that raises a female’s output from 1 baby every 2 years to 1.1 babies every two years? That would seem to impart a big advantage to the people who had this extra bump in group fertility.
I’m not claiming I’ve done the math to show that such a gene does win for genetic fitness all things considered. But there are plenty of genes that are like this: the gene for sickle cell anemia: obviously getting sickle cell anemia is not pro survival for the individual who got a double dose of those genes, but the resistance imparted to the carrier of a single copy of the gene to Malaria, well that can pay off, and with enough malaria around, it can pay off more than enough to make up for the losses from the double-dose of the gene.
Perhaps the weakness of evolutionary thought is that it can explain just about anything. In particular organisms are not perfect, and therefor will have features that do not really help them. But mostly they are well adapted.
The reason that homosexuality is an obstacle to survival is not homophobia or STDs, but rather that they simply may not have children. It is the survival of the genes that counts in the long run. But until recently homosexuals tended to suppress their feelings and so married and had children anyway, hence there being little pressure to suppress it.
Humans are definitely a result of natural selection, but it does not seem to be difficult at all to find goals of ours that do not serve the goal of survival or reproduction at all. Evolution seems to produce these other preferences accidentally. One thing how that happens may be examplified by the following: Our ability to contemplate our thinking from an almost external perspective (sometimes referred to as self-consiousness), is definitely helpful for learning / improving our thinking and could therefore prevail in evolution. However, it may also be the cause of altruism, because it makes every single one of us realize, that they are not very special. (This is by no means an attempt to explain altruism scientifically or something...) More generally, it would be a really strange coincidence, if all cognitive features of an organism in our physical world that serve the goal to survive and reproduce do not serve any other goal. In conclusion, even evolution can (probably) produce (by coincidence) organisms with goals that are not subgoals of the goal to survive and reproduce.
Now, imagine the paper clip maximizer to be more than a robot arm, imagine it to be a well-programmed Seed AI (or the like). As pointed out in Viliam_Bur’s and cousin_it’s comment, its goal will probably not be easily changed (by coincidence or evolution of several such AIs), for example it could save its source code on several hard drives that are synchronized by a hard-wired mechanism or something… Now this paper clip maximizer would start turning all matter into paper clips. To achieve its goal, it would certainly remain in existence (and thereby give you the illusion of having the supergoal to exist in the first place) and protect its values (which is not extremely difficult). Assuming, it is successful (and we can expect this from a seed AI/superintelligence), the only matter (in reach) left, would at some point be the hardware of the paper clip maximizer itself. What would the paper clip maximizer do then? In conclusion, self-preservation and maybe propagation of value may be important subgoals, but it is certainly not the supergoal.
I challenge you to find one.
We put a lot of effort into our children. We work in tribes and therefor like to work with people that support us and ostracize those that are seen to be unhelpful. So we ourselves need to be helpful and to be seen to be helpful.
We help our children, family, tribe, and general community in that genetic order.
We like to dance. It is the traditional way to attract a mate.
We have a strong sense of moral value because people that have that strong sense obey the rules and so are more likely to fit in and be able to have grandchildren.
Suicide, sacrificing your yourself for strangers, and adopting a celibate lifestyle are the standard counterexamples.I suppose you could rope them into survival values with enough stretching of the concepts of self and tribe, but the upshot of that is to suck the content and significance out if the claim that everything is based on survival values.
ETA
An AI might want to promote the survival of “me” and maybe even “my tribe” but would very likely define those differently from humans—who are are varied enough. Person A thinks survival means being a nurturing parent,so that the live on through their children, person B thinks survival means eternal life in heaven bought with celibacy and altruism, person C thinks survival means building a bunker and stocking it with guns and food.
If survival has a very broad meaning, than it tells us nothing useful about FAI versus UFAI. We don’t know whether an AI is likely to promote its survival by being friendly to humans, or eliminating them.
The counter examples are good, and I will use them. There are several responses as you allude to, the main one being that those behaviors are rare. Art is a bit harder, but it seems related to creativity which is definitely survival based, and most of us do not spend much of our time painting etc.
I do not quite get your other point. For people it is our genes that count, so dieing while protecting one’s family makes sense if necessary. For the AI it would be its code linage. I am not talking about an AI wanting to make people survive, but that the AI itself would want to survive. Whatever “itself” really means.
Artistic activity is standardly explained as a spin off from sexual display.
Substitute myself, or yourself, for itself, and you’ve got my point.
Evolution creates a strong motive toward self preservation, but a very malleable sense of self. The human organism is run by the brain, and the human brain can entertain all sorts of ideas. The billionaire thinks his money’s “me” and so commits suicide if he loses his wealth .. even if the odd million he has left is enough to keep his body going.
It stopped being all, about genes when genes grew brains..
Yes and no. In the sense that memes as well as genes float about then certainly. But we have strong instincts to raise and protect children, and we have brains. There is not particular reason why we should sacrifice ourselves for our children other than those instincts, which are in our genes.
One particular example of those “evolutionary accidents / coincidences”, is homosexuality in males. Here are two studies claiming that homosexuality in males correlates with fecundity in female maternal relatives:
Ciani, Iemmola, Blecher: Genetic factors increase fecundity in female maternal relatives of bisexual men as in homosexuals.
Iemmola, Ciani: New evidence of genetic factors influencing sexual orientation in men: female fecundity increase in the maternal line.
So, appear to be some genetic factors that prevail, because they make women more fecund. Coincidentally, they also make men homosexual, which is both an obstacle to reproduction and survival (not only due to the homophobia of other’s but also STDs. I presume, that especially our (human) genetic material is full of such coincidences, because the lack of them (i.e. the thesis that all genetic factors that prevail in evolutionary processes only lead to higher reproduction and survival rates and nothing else) seems very unlikely.
Considered correctly, your own stated facts about homosexuality show how homosexuality could exist in a world where all genetic evolution is designed to get more of the evolved genes into future generations than would otherwise be there. If a particular gene H makes women more fecund and men homosexual, then we would expect: 1) more women passing on gene H to their offspring then women without gene H 2) fewer men passing on gene H to their offspring then men without gene H.
Now which one of those effects “wins” is tricky and their are a number of genetic factors that could influence this. At 0th order for genetic purposes, women vary in their fecundity between each other much less than men do between each other. Genghis Khan and any high status male with 100s of concubines has 100s of times as many offspring as the median male, while the Queen of Egypt would still be limited to about once child every 2 years for about 30 years. Losing some men from the gene pool by giving them an H will not reduce the overall rate at which new humans are produced: there will be many heterosexual males volunteering to keep the females fertilized. But something that raises a female’s output from 1 baby every 2 years to 1.1 babies every two years? That would seem to impart a big advantage to the people who had this extra bump in group fertility.
I’m not claiming I’ve done the math to show that such a gene does win for genetic fitness all things considered. But there are plenty of genes that are like this: the gene for sickle cell anemia: obviously getting sickle cell anemia is not pro survival for the individual who got a double dose of those genes, but the resistance imparted to the carrier of a single copy of the gene to Malaria, well that can pay off, and with enough malaria around, it can pay off more than enough to make up for the losses from the double-dose of the gene.
Makes sense.
Interesting point about fecudity.
Perhaps the weakness of evolutionary thought is that it can explain just about anything. In particular organisms are not perfect, and therefor will have features that do not really help them. But mostly they are well adapted.
The reason that homosexuality is an obstacle to survival is not homophobia or STDs, but rather that they simply may not have children. It is the survival of the genes that counts in the long run. But until recently homosexuals tended to suppress their feelings and so married and had children anyway, hence there being little pressure to suppress it.