Design-space traps: mapping the utility-design trajectory space
This is a small section on a paper I’m writing on moral enhancement. I’m trying to briefly summarize some of the points which were already made concerning local optima in evolutionary process and safety regarding taking humanity out of those local optima. You might find the text helpful in that it summarizes a very important concept. I don’t think there’s nothing new here, but I hope the way I tried to more properly phrase the utility-design trajectory space topology at the end can be fruitful. I would appreciate any insights you might have about that formulation in the end, how to better develop it more rigorously and some consequences. I do have some ideas, but I would want to hear what you have to say first. Any other kind of general feedback on the text is also welcomed. But keep in mind this is just a section of a larger paper and I’m mainly interested in how to develop and what are the consequences of the framework at the end, rather than in properly developing any points in the middle.
Local optima are points where every nearby reachable positions are worse off, but there is at least one far away position which is vastly better. A strong case has been made that evolution often gets stuck on such local optima. In evolutionary processes, fitness is a monotonic function, i.e., it will necessarily increase or be maintained, any decrease in fitness will always be selected against. If there are vastly better solutions (for, e.g., solving cooperation problems) but in order to achieve those solutions organisms would have to pass through a lesser fit step, evolution will never reach that vastly better configuration. Evolutionary processes are limited by the topology of the fitness-design trajectory space, it can only go from design x to design y if there is at least one trajectory from x to y which is flat or ascendant, any trajectory momentarily descendent cannot be taken by the evolutionary processes. Say one is on the cyan ring ridge of the colored graphic.
Nevertheless, it should be noted getting out of evolutionary local optima might not always be easy or even possible. Fitness does have a relatively strong correlation with overall human utility. And although human intelligence is not so dull as evolutionary process and does accept a decrease in utility in order to achieve a better design in the end, if the downward moat is deep enough, the risk of catastrophe—or much worse, extinction -, might not be worth taking. At least by being monotonic on a dimension correlated with utility, evolution was able to rightly avoid extreme losses. Perform widespread willy-nilly human enhancement, and we might fall on the moat guarding utility-design space garden’s delicious low-hanging fruits and not come back up. Particularly so in the case of moral enhancement, there is a self-reinforcing aspect of changing morality, motivations, values and desires. It might be the case tampering with deep and fundamental human morality is irreversible, because once we fundamentally value something else, we would not have any compelling reason for wanting to come back to our old values, desires or aspirations. Thus, it seems there are indeed cases where a small step past the edge of the moat will lead us to an irreversible path. To correctly map how each technology shapes utility-design trajectory space topology is a task deeply needed in order to carefully avoid falling on moats while attempting to reach local optima low-hanging fruits, or on even more dangerous existential holes. We ought to better get stuck at local optima than absolute minima.
Utility-design trajectory space could be more properly defined as a space on Rn+u , a point there would use n-coordinates to locate all physically possible designs in all relevant dimensions n, it is defined by the laws of physics and by an utility function on u. A point will correspond to a design a iff all its neighbouring points x correspond to designs one physical step away from design a. Emergent designer processes such as evolution, human enhancement and AIs draw shapes on Rn+u by connecting points that are linked by one possible step under that process. Evolution’s hand is monotonic on dimension f, fitness, which makes for a pretty clumsy drawing. Biochemical human enhancement can more freely vary on f, but might contain other constraints elsewhere, that, e.g., uploaded minds would not. Extinctions correspond to singularities on u, once reached no other point is reachable, it designates lack of design. These points that can be reached but cannot reach need to be correctly mapped. It would also be relevant to investigate how each technology draws its specific shape on design space. Using u as some height analogue, some technologies might be inherently prone to shape moats with peaks on the middle, extinctions holes, effortless utility maximizing curves and so on. I believe moral enhancement draws a particularly bumpy hole-prone shape. FAI an ever utility-ascending shape, with all mishaps being existential holes.
Not true; chance plays a huge role in natural selection. This isn’t just a nitpick; chance serves the function that it serves to escape from local minima. Enzyme configurations, for instance, have often gone through less-fit intermediaries to arrive at more-fit solutions.
It’s not quite monotonic, but the actual mechanism is much more difficult to visualize.
Instead of one point in design space, think of a cloud of points. This cloud makes a successor cloud. Each point in the cloud has weight in the successor cloud proportional to it’s height, and makes points close to it in a random fashion.
There’s a certain ability to hop over downward slopes, but it takes many generations to expect it (and the more severe and deep the downward slope, the more time it will take).
So, strictly monotonic local optima aren’t necessary, but evolution tends to travel faster up steeper slopes.
Yes, both lines of thought did cross my mind once. But they are wrong. Mind that I’m talking about design-shaping processes, I’m not talking about all the design which are ultimately produced. That is, I’m not talking about which organisms are produced in each generation during the evolutionary process. Rather, I’m talking about which organisms are selected for and against by the process, I’m referring to the process in itself and what it does with designs over generations. And no matter how odd and random mutations a generation of organisms might have, no matter which less-fit intermediaries enzymes or whatnot, if they have less fitness they will be selected against. There’s no mechanism which could, in a predictive manner, account for the fact those less-fit enzymes will almost certainly lead to more-fit enzymes. The less-fit will be selected against, and if they survive long enough to arrive at a more-fit design, then—and only then—they will be selected for. One thing is what evolution does to the design landscape from one generation to the next, another thing—very much related—is how that design landscape will look like.
Of course, a less-fit mutation might survive for a while, but it will always be continuously being selected against. Eventually, it might suddenly achieve a better fit design than all the previous, but it must do so against the odds of evolution. I think the lesson to bring home here is that we are talking about one specific design-shaping process, and there are many other—not predominant—processes at play when shaping the design of life on earth. Those are not evolutionary process.
I’m not disagreeing with you there. Being selected against is different than not being born in the first place.
Less fit directions have exactly as many direct children as more fit directions. The difference is that there are more grandchildren in more fit directions, and fewer in less fit directions. Depending on the parameters for how much variance there is from generation to generation, evolution can cross small (relative to mutations per generation) downward slopes in design space.
Let’s have a ridiculously over-simplified example. There are four possible designs, AA, AB, BA, BB. AB and BA have value of .5, AA has value of 1, and BB has value of 2. Everyone starts at AA and has 1 child. 1% of children get moved from AA to AB or BA, from AB to either AA, or BB, from BA to either AA or BB. With 20k intial members, after one generation there’s 200 members each of AB and BA. Those 200 members of AB have 100 children, one of which is BB. That lone member of BB has 2 children each generation until essentially everything is BB.
Ta-da, evolution managed to make it’s way past a negative slope in design space and out of a local maxima.
I’m not saying that every local maxima can have some down-slope traversal, but that shallow enough moats can be crossed by random action. At this point, the now higher local maxima takes over.
If that’s not enough, imagine that there are literally infinite starting members of a certain organism. After each generation, there will be an infinite number of each of every possible descendant. Even the less fit ones. It’s trivial for evolution to get out of a local maxima with an infinite number of tries.
Agree. My point is that those designs with less fitness were not shaped by evolutionary process on that generation in which they decreased in fitness. It was a random process or whatnot. I’m not talking about all the designs that came to be, or to all beings who were or could have been born. I’m referring to what evolution does to design, and evolution does not make a design decrease in fitness, even though it may allow other process to do so. I think I’m not being clear here. Once we concentrate on things the evolutionary process actively does, in opposition to what may happen or what have happened, it should be straightforward that there are no decreases in fitness.
This is not consistent with what you said in the article (emphasis mine);
Thanks, I will make sure to make it clear in the post.
Minor nitpick
Strictly speaking there only has to be one far away point that is even slightly better for a point to only be a local, rather than global, optimum.
There doesn’t have to be any point that is better. A global optimum is also a local optimum.
That would be yet even useless, but I will specify better what I meant in the text.
Thanks. Corrected. The problem is that ultimately a very rigorous definition is useless. If there’s only one point, very, very far away, which is a little better, this isn’t really relevant and it would be somewhat odd to call your position a local optimum.
I really don’t think that’s true, if you mean ‘fitness’ in the evolutionary sense. One massive counterexample is the popularity of birth control—which seems to rise as people feel better off. Evolutionary fitness is not what we, as humans, value. And a good job too, I say: evolution produces horrors and monstrosities, favouring only those things that tend to reproduce.
That’s why I said relatively. Obviously, what we value is better correlated with utility since this is almost a tautological statement. However, so far we weren’t able to come up with any other function better correlated with utility than fitness, although we can see many clear cases where it fails miserably in doing that.
Really? Are people with eight children much better off than people with one child? Do people who choose to have no children lead terrible lives? Even crude metrics like “income” and “seconds humans spend smiling” seem like much better proxies for utility than genetic fitness.
One thing are good proxies for utility at the present time, another is coming up with a design-shaping process which generates utility out of a primitive earth across billions of years. Even looking forward, would you claim that a smiling or income maximizer would do better than evolution in the next 10,000 years? I highly doubt. Perhaps a better comparison, but still at fault, would be between our attempts of envisioning a FAI and evolution.
Optimizations have costs as well as benefits. Elimination of a genetic disease? Great! Now the cost is more humans to feed, clothe, educate etc. with no assurance the saved lives not will include those who are burdened in some other way. Thus a move up to the red is certainly a slip down as well, in some way. This leads to the question of ‘optimization for who?’ Then you can start deciding what percentage gets the red up and what percentage gets the blue down. There’s no avoiding unintended consequences, only mitigating them.
This is not true in a relevant manner. If we define the u ‘height’ dimension as total utility, than movements upwards will always imply increases in utility. Of course, this is only a ideal situation, since we can’t really assess total utility, but where I’m getting on is that the move up is suppose to mean an increase in utility, and if it is not, then you have to fix it.