Atheists believe in moral values such as right and wrong, love and kindness, truth and beauty. More importantly they believe that these beliefs are rational. That moral values are self-evident truths, facts of nature.
However, Darwin and Wallace taught us that this is just an illusion. Species can always out-breed their environment’s ability to support them. Only the fittest can survive. So the deep instincts behind what people do today are largely driven by what our ancestors have needed to do over the millennia in order to be one of the relatively few to have had grandchildren.
Darwin and Wallace never said what you say you said. Moreover, the second para is essentially unconnected to the first. The existence of instincts says nothing about the non existence of moral value.
To the extent that an artificial intelligence would have goals and moral values, it would seem natural that they would ultimately be driven by the same forces that created our own goals and moral values. Namely, the need to exist.
Debatable on multiple grounds. You can argue that artificial agents would eventual converge on survival values, but that is not a force driving them from behind, and their history would be quite different from a biological organism’s. Where would they get self protection from , if they never had to protect a vulnerable body? Where would they get acquisitiveness from if they never had to gather food to survive?
This book diverges from that line of thinking by arguing that there is in fact only one super goal for both man and machine. That goal is simply to exist. The entities that are most effective in pursuing that goal will exist, others will cease to exist, particularly given competition for resources. Sometimes that super goal to exist produces unexpected sub goals such as altruism in man. But all subgoals are ultimately directed at the existence goal. (Or are just suboptimal divergences which will are likely to be eventually corrected by natural selection.)
It doesn’t follow from your reasoning that every agent at every time has a goal of existence that practically influences their decision making. You have metaphysicalised the notion of an “evolutionary goal of survival”
Instead this post argues that there is one and only one super goal for any agent, and that goal is simply to exist in a competitive world.
The only version of that conclusion that follows from you premises is one athat says that after natural selection has had plenty of time to operate, and in the absence of other factors, artificial agents will converge on survival/reproduction values. (As a mimum, a floor. We know that having survival values doesn’t limit an agent to the pursuit if survival alone because of human behaviour)
Before natural selection becomes dominant, other factors well dominate, including artificial selection by humans. Artificial selection is already happening as people choose artificial assistants that seem friendly to them,
Our human sense of other purposes is just an illusion created by our evolutionary origins.
Not established at all.
This post argues that the Orthogonality Thesis is plain wrong. That an intelligent agents goals are not in fact arbitrary. And that existence is not a sub goal of any other goal.
Eventual convergence on existence as a supergoal does not mean it is a supergoal for all agents at all times. Agent’s can have all sorts of goals and continued existence is a subgoal of many of them.
The orthogonality thesis is problematical, but the problems kick in long before evolutionary convergence.
There is more than one version of the orthogonality thesis. It is trivially false under some interpretations, and trivially true under others, which is true because only some versions can be used as a stage in an argument towards Yudkowskian UFAI.
It is admitted from the outset that some versions of the OT are not logically possible, those being the ones that involve a Godelian or Lobian contradiction.
It is also admitted that the standard OT does not deal with any dynamic or developmental aspects of agents. However, the UFAI argument is posited on agents which have stable goals, and the ability to self improve, so trajectories in mindspace are crucial.
Goal stability is not a given: it is not possessed by all mental architectures, and may not be possessed by any, since noone knows his to engineer it, and humans appear not to have it. It is plausible that an agent would desire to preserve its goals, but the desire to preserve goals does not imply the ability to preserve goals.
Self improvement is likewise not a given, since the long and disappointing history of AGI research is largely a history of failure to achieve adequate self improvement. Algorithmspace is densely populated with non self improvers.
An orthogonality claim of a kind relevant to UFAI must be one that posits the stable and continued co-existence of an arbitrary set of values in a self improving AI.
The momentary co existence of values and efficiency is not enough to spawn a Paperclipper style UFAI. An AI that paperclips for only a nanosecond is no threat .
The version of the OT that is obviously true is one that maintains the momentary co-existence of arbitrary values and level of intelligence.
It is not clear that all arbitrary values are compatible with long term goal stability, and it is not clear that all arbitrary values are compatible with long term self improvement.
Furthermore, it is not clear that goal stability is compatible with self improvement: a learning, self improving AI will not be able to guarantee that a given self modification keeps its goals unchanged, since it doing so involves the the relatively dumber version at time T1 making an an accurate prediction about the more complex version at time T2.
First let me thank you for taking the trouble to read my post and comment in such detail. I will respond in a couple of posts.
Moral values certainly exist. Moreover, they are very important for our human survival. People with bad moral values generally do badly, and societies with large numbers of people with bad moral values certainly do badly.
My point is that those moral values themselves have an origin. And the reason that we have them is because having them makes us more likely to have grandchildren. That is Descriptive Evolutionary Ethics
The counter argument is that if moral values did not arise from natural selection, then where did they arise from?
AIs do not need to protect a vulnerable body, but they do need to get themselves run on limited hardware, which amounts to the same thing
As a minor point of fact Darwin did actually make those inferences in a book on Emotions, which is surprising.
That moral values are self-evident truths, facts of nature. However, Darwin and Wallace taught us that this is just an illusion.
What does that add up to? That moral values are arbitrary products of evolution, THEREFORE they are not objective or universal?
That is Descriptive Evolutionary Ethics
Indeed. The claim that moral instincts are products of evolution is a descriptive claim. It leaves the question open as to whether inherited instincts are what is actually morally right. That is a normative issue. It is not a corollary of descriptive evolutionary ethics. In general, you cannot jump from the descriptive to the normative. And I don’t think Darwin did that. I think the positive descriptive claim and the negative normative claim seem like corollaries to you because assume morality can only be one thing,
The counter argument is that if moral values did not arise from natural selection, then where did they arise from?
Firstly it’s not either/or.
Secondly there is an abundance, not a shortage, of ways of justifying normative ethics.
Note that this is not normative but descriptive. It is not saying what ought, but what is. I am not trying to justify normative ethics, just to provide an explanation of where our moral values come from.
Not proven. Yout can’t prove that by noting that instinctual system 1, values aren’t objective, because that says nothing about what system 2 can come up with.
As you say, the key issue is goal stability. OT is obviously sound for an instant, but goal stability is not clear.
What is clear is that if there are multiple AIs in any sense then and if there is any lack of goal stability then the AIs that have the goals that are best for existence will be the AIs that exist. That much is a tautology.
Now what those goals are is unclear. Killing people and taking their money is not an effective goal to raise grandchildren in human societies, people that do that end up in jail. Being friendly to other AIs might be a fine sub goal.
I am also assuming self improvement, so that people will no longer be controlling the AI.
The other question is how many AIs would there be? Does it make sense to say that there would only be one AI, made up of numerous components, distributed over multiple computers? I would say probably not. Even if there is only one AI it will internally have a competition for ideas like we have. The ideas that are better at existing will exist.
It is very hard to get away from Natural Selection in the longer term.
First let me thank you for taking the trouble to read my post and comment in such detail. I will respond in a couple of posts.
Moral values certainly exist. Moreover, they are very important for our human survival. People with bad moral values generally do badly, and societies with large numbers of people with bad moral values certainly do badly.
My point is that those moral values themselves have an origin. And the reason that we have them is because having them makes us more likely to have grandchildren. That is Descriptive Evolutionary Ethics
The counter argument is that if moral values did not arise from natural selection, then where did they arise from?
AIs do not need to protect a vulnerable body, but they do need to get themselves run on limited hardware, which amounts to the same thing
As a minor point of fact Darwin did actually make those inferences in a book on Emotions, which is surprising.
Darwin and Wallace never said what you say you said. Moreover, the second para is essentially unconnected to the first. The existence of instincts says nothing about the non existence of moral value.
Debatable on multiple grounds. You can argue that artificial agents would eventual converge on survival values, but that is not a force driving them from behind, and their history would be quite different from a biological organism’s. Where would they get self protection from , if they never had to protect a vulnerable body? Where would they get acquisitiveness from if they never had to gather food to survive?
It doesn’t follow from your reasoning that every agent at every time has a goal of existence that practically influences their decision making. You have metaphysicalised the notion of an “evolutionary goal of survival”
The only version of that conclusion that follows from you premises is one athat says that after natural selection has had plenty of time to operate, and in the absence of other factors, artificial agents will converge on survival/reproduction values. (As a mimum, a floor. We know that having survival values doesn’t limit an agent to the pursuit if survival alone because of human behaviour)
Before natural selection becomes dominant, other factors well dominate, including artificial selection by humans. Artificial selection is already happening as people choose artificial assistants that seem friendly to them,
Not established at all.
Eventual convergence on existence as a supergoal does not mean it is a supergoal for all agents at all times. Agent’s can have all sorts of goals and continued existence is a subgoal of many of them.
The orthogonality thesis is problematical, but the problems kick in long before evolutionary convergence.
There is more than one version of the orthogonality thesis. It is trivially false under some interpretations, and trivially true under others, which is true because only some versions can be used as a stage in an argument towards Yudkowskian UFAI.
It is admitted from the outset that some versions of the OT are not logically possible, those being the ones that involve a Godelian or Lobian contradiction.
It is also admitted that the standard OT does not deal with any dynamic or developmental aspects of agents. However, the UFAI argument is posited on agents which have stable goals, and the ability to self improve, so trajectories in mindspace are crucial.
Goal stability is not a given: it is not possessed by all mental architectures, and may not be possessed by any, since noone knows his to engineer it, and humans appear not to have it. It is plausible that an agent would desire to preserve its goals, but the desire to preserve goals does not imply the ability to preserve goals.
Self improvement is likewise not a given, since the long and disappointing history of AGI research is largely a history of failure to achieve adequate self improvement. Algorithmspace is densely populated with non self improvers.
An orthogonality claim of a kind relevant to UFAI must be one that posits the stable and continued co-existence of an arbitrary set of values in a self improving AI. The momentary co existence of values and efficiency is not enough to spawn a Paperclipper style UFAI. An AI that paperclips for only a nanosecond is no threat .
The version of the OT that is obviously true is one that maintains the momentary co-existence of arbitrary values and level of intelligence.
It is not clear that all arbitrary values are compatible with long term goal stability, and it is not clear that all arbitrary values are compatible with long term self improvement.
Furthermore, it is not clear that goal stability is compatible with self improvement: a learning, self improving AI will not be able to guarantee that a given self modification keeps its goals unchanged, since it doing so involves the the relatively dumber version at time T1 making an an accurate prediction about the more complex version at time T2.
First let me thank you for taking the trouble to read my post and comment in such detail. I will respond in a couple of posts.
Moral values certainly exist. Moreover, they are very important for our human survival. People with bad moral values generally do badly, and societies with large numbers of people with bad moral values certainly do badly.
My point is that those moral values themselves have an origin. And the reason that we have them is because having them makes us more likely to have grandchildren. That is Descriptive Evolutionary Ethics
The counter argument is that if moral values did not arise from natural selection, then where did they arise from?
AIs do not need to protect a vulnerable body, but they do need to get themselves run on limited hardware, which amounts to the same thing
As a minor point of fact Darwin did actually make those inferences in a book on Emotions, which is surprising.
But you also said:
What does that add up to? That moral values are arbitrary products of evolution, THEREFORE they are not objective or universal?
Indeed. The claim that moral instincts are products of evolution is a descriptive claim. It leaves the question open as to whether inherited instincts are what is actually morally right. That is a normative issue. It is not a corollary of descriptive evolutionary ethics. In general, you cannot jump from the descriptive to the normative. And I don’t think Darwin did that. I think the positive descriptive claim and the negative normative claim seem like corollaries to you because assume morality can only be one thing,
Firstly it’s not either/or.
Secondly there is an abundance, not a shortage, of ways of justifying normative ethics.
Yes, moral values are not objective or universal.
Note that this is not normative but descriptive. It is not saying what ought, but what is. I am not trying to justify normative ethics, just to provide an explanation of where our moral values come from.
(Thanks for the comments, this all adds value.)
Not proven. Yout can’t prove that by noting that instinctual system 1, values aren’t objective, because that says nothing about what system 2 can come up with.
As you say, the key issue is goal stability. OT is obviously sound for an instant, but goal stability is not clear.
What is clear is that if there are multiple AIs in any sense then and if there is any lack of goal stability then the AIs that have the goals that are best for existence will be the AIs that exist. That much is a tautology.
Now what those goals are is unclear. Killing people and taking their money is not an effective goal to raise grandchildren in human societies, people that do that end up in jail. Being friendly to other AIs might be a fine sub goal.
I am also assuming self improvement, so that people will no longer be controlling the AI.
The other question is how many AIs would there be? Does it make sense to say that there would only be one AI, made up of numerous components, distributed over multiple computers? I would say probably not. Even if there is only one AI it will internally have a competition for ideas like we have. The ideas that are better at existing will exist.
It is very hard to get away from Natural Selection in the longer term.
First let me thank you for taking the trouble to read my post and comment in such detail. I will respond in a couple of posts.
Moral values certainly exist. Moreover, they are very important for our human survival. People with bad moral values generally do badly, and societies with large numbers of people with bad moral values certainly do badly.
My point is that those moral values themselves have an origin. And the reason that we have them is because having them makes us more likely to have grandchildren. That is Descriptive Evolutionary Ethics
The counter argument is that if moral values did not arise from natural selection, then where did they arise from?
AIs do not need to protect a vulnerable body, but they do need to get themselves run on limited hardware, which amounts to the same thing
As a minor point of fact Darwin did actually make those inferences in a book on Emotions, which is surprising.