“Out of Control” appears to be primarily about decentralized decision making processes like democracy and capitalism. I never said that brains controlled the artifacts of civilization in a centralized fashion, I just said that they control them. Obviously human beings use all sorts of decentralized methods to help coordinate with each other.
That being said, while systems are not controlled in a centralized manner, they are restricted in a centralized manner. For instance, capitalism only works properly if people are prevented from killing and stealing. Even if there is no need to centrally control everything to get positive results, there is a need to centrally control some things.
There seems to be a later section in “Out of Control” where Kelly suggests giving up control to our machines is good in the same way that dictators giving up central control to democracy and capitalism is good. This seems short-sighted, especially in light of things like Bostrom’s orthogonality thesis. The reasons democracy and capitalism do so much good is that:
Human minds are an important component of them, and (most) humans care about morality, so the systems tend to be steered towards morally good results.
There are some centralized restrictions on what these decentralized systems are able to do.
Unless you are somehow able to program the machines with moral values (i.e. make an FAI), turning control over to them seems like a bad idea. Creating moral machines isn’t impossible, but the main point of Eliezer’s writing is that it is much, much harder than it seems. I think he’s quite correct.
As for memetics, the idea impressed me when I first came across it, but there doesn’t seem to have been much development in the field since then. I am no longer impressed. In any case, the main reason memes “propagate” is that they help a brain fulfill its desires in some way, so really ever-evolving memes are just another one of the human mind’s tools in its continuing quest for universal domination.
As for memetics, the idea impressed me when I first came across it, but there doesn’t seem to have been much development in the field since then. I am no longer impressed. In any case, the main reason memes “propagate” is that they help a brain fulfill its desires in some way, so really ever-evolving memes are just another one of the human mind’s tools in its continuing quest for universal domination.
From a biological perspective, brains are seen as being a way for genes and memes to make more copies of themselves.
That this is a valuable point of view is illustrated by some sea squirts—which digest their own brains to further their own reproductive ends.
In nature, genes are fundamental, while brains are optional and expendable.
From a biological perspective, brains are seen as being a way for genes and memes to make more copies of themselves....
...In nature, genes are fundamental, while brains are optional and expendable.
Genes are biologically fundamental, certainly. You will get no argument from me there (although the fact that brains are biologically expendable does not imply that it is moral to expend them). The evidence that memes are more fundamental than brains, however, is not nearly as strong.
It is quite possible to model memes as “reproducing,” by being passed from one brain to another. But most of the time the reason the meme is passed from one brain to another is because they aid the brain in fulfilling its desires in some way. The memes associated with farming, for instance, spread because they helped the brain fulfill its desire to not starve. In instances where brains stopped needing the farming memes to obtain food (such as when the Plains Indians acquired horses and were suddenly able to hunt bison more efficiently) those memes promptly died out.
There are parasitic memes, cult ideologies for instance, that reproduce by exploiting flaws in the brain’s cognitive architecture. But the majority of memes “reproduce” by demonstrating their usefulness to the brain carrying them. You could say that a meme’s “fitness” is measured by its usefulness to its host.
You could say that a meme’s “fitness” is measured by its usefulness to its host.
That wouldn’t be terribly accurate, though. Smoking memes, obesity memes, patriotism memes, and lots of advertising and marketing memes are not good for their hosts, but rather benefit those attempting to manipulate them. However, there’s usually a human somewhere at the end of the chain today.
That probably won’t remain the case, though. After the coming memetic takeover we are likely to have an engineered future—and then it will be memes all the way down.
That probably won’t remain the case, though. After the coming memetic takeover we are likely to have an engineered future—and then it will be memes all the way down.
The memetic takeover you describe would just consist of intelligences running on computer-like substrates instead of organic substrates. That isn’t morally relevant to me, I don’t care if the creatures of the future are made of carbon or silicon. I care about what sort of minds they have, what they value and believe in.
I’m not sure referring to an intelligent creature that is made of computing code instead of carbon as a “meme” is true to the common definitions of the term. I always thought of memes as contagious ideas and concepts, not as a term to describe an entire intellect.
After the memetic takeover there would still be intelligent creatures, they’d just run on a different substrate. Many of them could possibly be brain-like in design or have human-like values. They would continue to exchange memes with each other just as they did before, and those memes would spread or die depending on their usefulness to the intelligent creatures. Just like they do now.
I’m not sure referring to an intelligent creature that is made of computing code instead of carbon as a “meme” is true to the common definitions of the term.
People don’t call the works of Shakespeare a “meme” either. Conventionally, such things are made of memes—and meme products.
“Out of Control” appears to be primarily about decentralized decision making processes like democracy and capitalism. I never said that brains controlled the artifacts of civilization in a centralized fashion, I just said that they control them. Obviously human beings use all sorts of decentralized methods to help coordinate with each other.
That being said, while systems are not controlled in a centralized manner, they are restricted in a centralized manner. For instance, capitalism only works properly if people are prevented from killing and stealing. Even if there is no need to centrally control everything to get positive results, there is a need to centrally control some things.
There seems to be a later section in “Out of Control” where Kelly suggests giving up control to our machines is good in the same way that dictators giving up central control to democracy and capitalism is good. This seems short-sighted, especially in light of things like Bostrom’s orthogonality thesis. The reasons democracy and capitalism do so much good is that:
Human minds are an important component of them, and (most) humans care about morality, so the systems tend to be steered towards morally good results.
There are some centralized restrictions on what these decentralized systems are able to do.
Unless you are somehow able to program the machines with moral values (i.e. make an FAI), turning control over to them seems like a bad idea. Creating moral machines isn’t impossible, but the main point of Eliezer’s writing is that it is much, much harder than it seems. I think he’s quite correct.
As for memetics, the idea impressed me when I first came across it, but there doesn’t seem to have been much development in the field since then. I am no longer impressed. In any case, the main reason memes “propagate” is that they help a brain fulfill its desires in some way, so really ever-evolving memes are just another one of the human mind’s tools in its continuing quest for universal domination.
From a biological perspective, brains are seen as being a way for genes and memes to make more copies of themselves.
That this is a valuable point of view is illustrated by some sea squirts—which digest their own brains to further their own reproductive ends.
In nature, genes are fundamental, while brains are optional and expendable.
Genes are biologically fundamental, certainly. You will get no argument from me there (although the fact that brains are biologically expendable does not imply that it is moral to expend them). The evidence that memes are more fundamental than brains, however, is not nearly as strong.
It is quite possible to model memes as “reproducing,” by being passed from one brain to another. But most of the time the reason the meme is passed from one brain to another is because they aid the brain in fulfilling its desires in some way. The memes associated with farming, for instance, spread because they helped the brain fulfill its desire to not starve. In instances where brains stopped needing the farming memes to obtain food (such as when the Plains Indians acquired horses and were suddenly able to hunt bison more efficiently) those memes promptly died out.
There are parasitic memes, cult ideologies for instance, that reproduce by exploiting flaws in the brain’s cognitive architecture. But the majority of memes “reproduce” by demonstrating their usefulness to the brain carrying them. You could say that a meme’s “fitness” is measured by its usefulness to its host.
That wouldn’t be terribly accurate, though. Smoking memes, obesity memes, patriotism memes, and lots of advertising and marketing memes are not good for their hosts, but rather benefit those attempting to manipulate them. However, there’s usually a human somewhere at the end of the chain today.
That probably won’t remain the case, though. After the coming memetic takeover we are likely to have an engineered future—and then it will be memes all the way down.
The memetic takeover you describe would just consist of intelligences running on computer-like substrates instead of organic substrates. That isn’t morally relevant to me, I don’t care if the creatures of the future are made of carbon or silicon. I care about what sort of minds they have, what they value and believe in.
I’m not sure referring to an intelligent creature that is made of computing code instead of carbon as a “meme” is true to the common definitions of the term. I always thought of memes as contagious ideas and concepts, not as a term to describe an entire intellect.
After the memetic takeover there would still be intelligent creatures, they’d just run on a different substrate. Many of them could possibly be brain-like in design or have human-like values. They would continue to exchange memes with each other just as they did before, and those memes would spread or die depending on their usefulness to the intelligent creatures. Just like they do now.
People don’t call the works of Shakespeare a “meme” either. Conventionally, such things are made of memes—and meme products.