I think eliezer would agree with what you’re saying here, in the same post mentioned;
The phrase “emerges from” is acceptable, just like “arises from” or “is caused by” are acceptable, if the phrase precedes some specific model to be judged on its own merits.
However, this is not the way “emergence” is commonly used. “Emergence” is commonly used as an explanation in its own right.
So he would agree that as long as you’re not using the word “emergence” as an explanation—since the sequence is on words which are used in common language which don’t predict anything by themselves— and are actually acknowledging the various mechanisms beneath, which you’re understanding using higher level non fundamental abstractions.
(I.e: There’s no way you can model a 747 quark-by-quark, so you’ve got to use a multi-level map with explicit cognitive representations of wings, airflow, and so on. This doesn’t mean there’s a multi-level territory. The true laws of physics, to the best of our knowledge, are only over elementary particle fields.)
I think that when physicists say “There are no fundamental rainbows,” the anti-reductionists hear, “There are no rainbows.”
If you don’t distinguish between the multi-level map and the mono-level territory, then when someone tries to explain to you that the rainbow is not a fundamental thing in physics, acceptance of this will feel like erasing rainbows from your multi-level map, which feels like erasing rainbows from the world.
So it’s quite clear that he’s actually fine with higher level abstractions like you’re using here, as long as they predict things. The phrase “intelligence is emergent” as what intelligence is doesn’t predict anything and is a blank phrase, this is what he was opposed to.
I wished he was a lot more clear with these things back then, it took me quite a bit of time to understand his position (it’s fairly neat imo)
Thanks for your comment, I appreciate your points, and see that Yudkowsky appreciates some use of higher-level abstractions as a pragmatic tool that is not erased by reductionism. But I still feel like you’re being a bit too charitable. I re-read the ‘it’s okay to use ‘emerge”’ parts several times, and as I understand it, he’s not meaning to refer to a higher-level abstraction, he’s using it in the general sense “whatever byproduct comes from this” in which case it would be just as meaningful to say “heat emerges from the body” which does not reflect any definition of emergence as a higher-level abstraction. I think the issue comes into focus with your final point:
The phrase “intelligence is emergent” as what intelligence is doesn’t predict anything and is a blank phrase, this is what he was opposed to.
But it is not correct to say that acknowledging intelligence as emergent doesn’t help us predict anything. If emergence can be described as a pattern that happens across different realms then it can help to predict things, through the use of analogy. If for instance we can see that neurones are selected and strengthened based on use, we can transfer some of our knowledge about natural selection in biological evolution to provide fruitful questions to ask, and research to do, on neural evolution. If we understand that an emergent system has reached equilibrium, it can help us to ask useful questions about what new systems might emerge on top of that system, questions we might not otherwise ask if we were not to recognise the shared pattern.
A question I often ask myself is “If the world itself is to become increasingly organised, at some point do we cease to be autonomous entities an on a floating rock, and become instead like automatic cells within a new vector of autonomy (the planet as super-organism)”. This question only comes about if we acknowledge that the world itself is subject to the same sorts of emergent processes that humans and other animals are (although not exactly, a planet doesn’t have much of a social life, and that could be essential to autonomy). I find these predictions based on principles of emergence interesting and potentially consequential.
But I still feel like you’re being a bit too charitable. I re-read the ‘it’s okay to use ‘emerge”’ parts several times, and as I understand it, he’s not meaning to refer to a higher-level abstraction, he’s using it in the general sense “whatever byproduct comes from this” in which case it would be just as meaningful to say “heat emerges from the body” which does not reflect any definition of emergence as a higher-level abstraction.
[,,,]
But it is not correct to say that acknowledging intelligence as emergent doesn’t help us predict anything. If emergence can be described as a pattern that happens across different realms then it can help to predict things, through the use of analogy.
I don’t think Eliezer uses emergence that way. He is using it the way, if a person is asked “why do hands have X muscular movement?” , one may reply “it’s an emergent phenomena” , that is the explanation which doesn’t predict anything he’s criticizing, unless a person clarifies what they mean by emergent phenomena.
A proper explanation could be: (Depending on what is meant by the word “Why”) [1]
Evolutionary: The reason why X muscular movement got selected for.
Biological/Shape/Adaptation; How it works or got implemented.
The common use of the word emergent is such that when a person is perplexed by the idea of free will, and finds a lack of contra-causal free will troubling to their preliminary intuitions , their encounter with the idea “free will is emergent” resolves the cognitive dissonance mistaking it for an explanation[2] when it holds no predictive power, and doesn’t actually work to resolve the initial confusion alongside physics.
What examples do I have in back of my mind that I think he’s criticizing this particular usage?
Eg-1: He uses the example of “Intelligence is emergent”.
In online spaces when asked, “Where is the ‘you’ in the brain if it’s all just soft flesh?” , people often say , “I am emergent” . Which doesn’t quite predict anything, like I learn nothing about when do I cease to be “I” , or why do I feel like “I” etc.
Eg-2: He uses the example of “Free will is emergent” , where he mentions the phrasing “one level emerges from the other”.
To dissolve the puzzle of free will, you have to simultaneously imagine two levels of organization while keeping them conceptually distinct. To get it on a gut level, you have to seethe level transition—the way in which free will is how the human decision algorithm feels from inside. (Being told flatly “one level emerges from the other” just relates them by a magical transition rule, “emergence”.)
Eg-3; He uses “behavior of ant colony is emergent” in original post.
Eg-4; He also emphasizes that he’s fine with the term that chemistry “arises from” interaction between atoms as per QED. Since chemistry—or parts of it, can be predicted in terms of QED. [2]
Chemistry arises from interactions between atoms, according to the specific model of quantum electrodynamics.
Which he clarifies is fairly equivalent to “Chemistry emerges from interaction between atoms as per QED”.
None of these examples seem to argue against “emergence as a pattern that happens across different realms” . That seems like a different thing altogether and can be assigned a different word.
This particular usage he is criticizing, and this is the trap majority of people including my past self & a lot of people I know in real life fall into. Which is why, I think the disagreement here is mostly semantic, as highlighted by TAG. This can also be categorized as the trap of strong emergentism, or intuitions behind it such that it satisfies the human interrogation without adding anything to understanding. Moreover, the sequence in question is named “Mysterious answers” where he goes over certain concepts which are in common zeitgeist used as an explanations even when they’re not.[2]
From what I understand the way you’re using it, in emergent cycle to import over to other places Eliezer would agree with your use case. He uses this as an argument as to why maths is useful :-
The apples are behaving like numbers? What do you mean? I thought numbers were this ethereal mathematical model that got pinpointed by axioms, not by looking at the real world.
“Whenever a part of reality behaves in a way that conforms to the number-axioms—for example, if putting apples into a bowl obeys rules, like no apple spontaneously appearing or vanishing, which yields the high-level behavior of numbers—then all the mathematical theorems we proved valid in the universe of numbers can be imported back into reality. The conclusion isn’t absolutely certain, because it’s not absolutely certain that nobody will sneak in and steal an apple and change the physical bowl’s behavior so that it doesn’t match the axioms any more. But so long as the premises are true, the conclusions are true; the conclusion can’t fail unless a premise also failed. You get four apples in reality, because those apples behaving numerically isn’t something you assume, it’s something that’s physically true. When two clouds collide and form a bigger cloud, on the other hand, they aren’t behaving like integers, whether you assume they are or not.”
But if the awesome hidden power of mathematical reasoning is to be imported into parts of reality that behave like math, why not reason about apples in the first place instead of these ethereal ‘numbers’?
It seems like your emergent cycle is closer to this. He similarly to your emergent cycle for systems also asserts probability theory as laws underlying rational belief and similarly decision theory for rational actions of all agents:-
Probability theory is the set of laws underlying rational belief. The mathematics of probability applies equally to “figuring out where your bookcase is” and “estimating how many hairs were on Julius Caesars head,” even though our evidence for the claim “Julius Caesar was bald” is likely to be more complicated and indirect than our evidence for the claim “theres a bookcase in my room.” It’s all the same problem of how to process the evidence and observations to update one’s beliefs. Similarly, decision theory is the set of laws underlying rational action, and is equally applicable regardless of what one’s goals and available options are .
Decision theory works because it is sufficiently similar to the goal oriented systems in the universe. He also thinks intelligence is lawful, as in it’s orderly such as following decision theory. Which seems similar to your defense in the sense of multiple leveled maps.
To further flesh the point, he would agree with you on the eyes part :-
The notion of a “configuration space” is a way of translating object descriptions into object positions. It may seem like blue is “closer” to blue-green than to red, but how much closer? It’s hard to answer that question by just staring at the colors. But it helps to know that the (proportional) color coordinates in RGB are 0:0:5, 0:3:2 and 5:0:0. It would be even clearer if plotted on a 3D graph.
In the same way, you can see a robin as a robin—brown tail, red breast, standard robin shape, maximum flying speed when unladen, its species-typical DNA and individual alleles. Or you could see a robin as a single point in a configuration space whose dimensions described everything we knew, or could know, about the robin.
A robin is bigger than a virus, and smaller than an aircraft carrier—that might be the “volume” dimension. Likewise a robin weighs more than a hydrogen atom, and less than a galaxy; that might be the “mass” dimension. Different robins will have strong correlations between “volume” and “mass”, so the robin-points will be lined up in a fairly linear string, in those two dimensions—but the correlation won’t be exact, so we do need two separate dimensions.
[,,,]
We can even imagine a configuration space with one or more dimensions for every distinct characteristic of an object, so that the position of an object’s point in this space corresponds to all the information in the real object itself. Rather redundantly represented, too—dimensions would include the mass, the volume, and the density.
[,,,]
Suppose we mapped all the birds in the world into thingspace, using a distance metric that corresponds as well as possible to perceived similarity in humans: A robin is more similar to another robin, than either is similar to a pigeon, but robins and pigeons are all more similar to each other than either is to a penguin, etcetera.
Then the center of all birdness would be densely populated by many neighboring tight clusters, robins and sparrows and canaries and pigeons and many other species. Eagles and falcons and other large predatory birds would occupy a nearby cluster. Penguins would be in a more distant cluster, and likewise chickens and ostriches.
He probably would assign the intensional/word “eye” to an extensional similarity cluster but I think you and him might still disagree on nuances:
The atoms of a screwdriver don’t have tiny little XML tags inside describing their “objective” purpose. The designer had something in mind, yes, but that’s not the same as what happens in the real world. If you forgot that the designer is a separate entity from the designed thing, you might think, “The purpose of the screwdriver is to drive screws”—as though this were an explicit property of the screwdriver itself, rather than a property of the designer’s state of mind. You might be surprised that the screwdriver didn’t reconfigure itself to the flat-head screw, since, after all, the screwdriver’s purpose is to turn screws.
[,,,]
So the screwdriver’s cause, and its shape, and its consequence, and its various meanings, are all different things; and only one of these things is found within the screwdriver itself.
Where do taste buds come from? Not from an intelligent designer visualizing their consequences, but from a frozen history of ancestry: Adam liked sugar and ate an apple and reproduced, Barbara liked sugar and ate an apple and reproduced, Charlie liked sugar and ate an apple and reproduced, and 2763 generations later, the allele became fixed in the population. For convenience of thought, we sometimes compress this giant history and say: “Evolution did it.” But it’s not a quick, local event like a human designer visualizing a screwdriver. This is the objective cause of a taste bud.
What is the objective shape of a taste bud? Technically, it’s a molecular sensor connected to reinforcement circuitry. This adds another level of indirection, because the taste bud isn’t directly acquiring food. It’s influencing the organism’s mind, making the organism want to eat foods that are similar to the food just eaten.
What is the objective consequence of a taste bud? In a modern First World human, it plays out in multiple chains of causality: from the desire to eat more chocolate, to the plan to eat more chocolate, to eating chocolate, to getting fat, to getting fewer dates, to reproducing less successfully. This consequence is directly opposite the key regularity in the long chain of ancestral successes which caused the taste bud’s shape. But, since overeating has only recently become a problem, no significant evolution (compressed regularity of ancestry) has further influenced the taste bud’s shape.
What is the meaning of eating chocolate? That’s between you and your moral philosophy. Personally, I think chocolate tastes good, but I wish it were less harmful; acceptable solutions would include redesigning the chocolate or redesigning my biochemistry.
Which is to say he would disagree with the blanket categorization of “purpose”, which can be a problematic term and leaves space for misunderstanding regarding normativity and would likely advocate for more clearer thinking and wordings along the lines highlighted above.
Although I think he would be fine with concepts like ‘agency’ in decision theories to degree to which axioms happen to coincide with that of reality—just like apples behaving like numbers listed above— since it can be bounded to physical systems amongst other considerations such as well biological agent sustaining itself etc.
Which brings us to his second potential source of disagreement, regarding analogies :[3]
A medieval alchemist puts lemon glazing onto a lump of lead. The lemon glazing is yellow, and gold is yellow. It seems like it ought to work… but the lead obstinately refuses to turn into gold. Reality just comes back and says, “So what? Things can be similar in some aspects without being similar in other aspects.”
[,,,]
The general form of failing-by-analogy runs something like this:
You want property P.
X has property P.
You build Y, which has one or two surface similarities S to X.
You argue that Y resembles X and should also P.
Yet there is no reasoning which you can do on Y as a thing-in-itself to show that it will have property P, regardless of whether or not X had ever existed.
[,,,]
If two processes have forms that are nearly identical, including internal structure that is similar to as many decimal places as you care to reason about, then you may be able to almost-prove results from one to the other. But if there is even one difference in the internal structure, then any number of other similarities may be rendered void. Two deterministic computations with identical data and identical rules will yield identical outputs. But if a single input bit is flipped from zero to one, the outputs are no longer required to have anything in common. The strength of analogical reasoning can be destroyed by a single perturbation.
Yes, sometimes analogy works. But the more complex and dissimilar the objects are, the less likely it is to work. The narrower the conditions required for success, the less likely it is to work. The more complex the machinery doing the job, the less likely it is to work. The more shallow your understanding of the object of the analogy, the more you are looking at its surface characteristics rather than its deep mechanisms, the less likely analogy is to work.
[,,,]
Admittedly, analogy often works in mathematics—much better than it does in science, in fact. In mathematics you can go back and prove the idea which analogy originally suggested. In mathematics, you get quick feedback about which analogies worked and which analogies didn’t, and soon you pick up the pattern. And in mathematics you can always see the entire insides of things; you are not stuck examining the surface of an opaque mystery. Mathematical proposition A may be analogous to mathematical proposition B, which suggests the method; but afterward you can go back and prove A in its own right, regardless of whether or not B is true. In some cases you may need proposition B as a lemma, but certainly not all cases.
Which is to say: despite the misleading surface similarity, the “analogies” which mathematicians use are not analogous to the “analogies” of alchemists, and you cannot reason from the success of one to the success of the other.
Which basically goes over the necessity for having a reasoning as to why, someone suspects this analogy to work aside from being merely similar in certain aspects, because most of them don’t. Take the example of more sophisticated analogy of biological evolution extended to corporations:
Do corporations evolve? They certainly compete. They occasionally spin off children. Their resources are limited. They sometimes die.
But how much does the child of a corporation resemble its parents? Much of the personality of a corporation derives from key officers, and CEOs cannot divide themselves by fission. Price’s Equation only operates to the extent that characteristics are heritable across generations. If great-great-grandchildren don’t much resemble their great-great-grandparents, you won’t get more than four generations’ worth of cumulative selection pressure—anything that happened more than four generations ago will blur itself out. Yes, the personality of a corporation can influence its spinoff—but that’s nothing like the heritability of DNA, which is digital rather than analog, and can transmit itself with 10^-8 errors per base per generation.
With DNA you have heritability lasting for millions of generations. That’s how complex adaptations can arise by pure evolution—the digital DNA lasts long enough for a gene conveying 3% advantage to spread itself over 768 generations, and then another gene dependent on it can arise. Even if corporations replicated with digital fidelity, they would currently be at most ten generations into the RNA World.
Now, corporations are certainly selected, in the sense that incompetent corporations go bust. This should logically make you more likely to observe corporations with features contributing to competence. And in the same sense, any star that goes nova shortly after it forms, is less likely to be visible when you look up at the night sky. But if an accident of stellar dynamics makes one star burn longer than another star, that doesn’t make it more likely that future stars will also burn longer—the feature will not be copied onto other stars. We should not expect future astrophysicists to discover complex internal features of stars which seem designed to help them burn longer. That kind of mechanical adaptation requires much larger cumulative selection pressures than a once-off winnowing.
Think of the principle introduced in Einstein’s Arrogance—that the vast majority of the evidence required to think of General Relativity had to go into raising that one particular equation to the level of Einstein’s personal attention; the amount of evidence required to raise it from a deliberately considered possibility to 99.9% certainty was trivial by comparison. In the same sense, complex features of corporations which require hundreds of bits to specify, are produced primarily by human intelligence, not a handful of generations of low-fidelity evolution. In biology, the mutations are purely random and evolution supplies thousands of bits of cumulative selection pressure. In corporations, humans offer up thousand-bit intelligently designed complex “mutations”, and then the further selection pressure of “Did it go bankrupt or not?” accounts for a handful of additional bits in explaining what you see.
Since this analogy can mislead one to think corporations can evolve on their own and someone may conclude that hence anyone can become CEO of a corporation, but it’s the case that majority of changes are due to human intelligence so that would be an example of failure by analogy.[4]
As per my amateur analysis you seem to have taken caution in your emergent cycle analogy, as to only apply it when entropy has inverse relationship, which is still quite a broad application and but on surface seems to isolate the generalization to certain systems which have sufficiently enough internal structure for the analogy to carry over under those constraints.
A thing to note here is Eliezer is a predictivist, both of these type of explanation would narrow down experience to hand’s muscle moving a certain way.
In this post he criticized neural networks, and as we know that particular prediction of his aged poorly, but those are for different reasons, but the general point regarding analogy still stands.
I think eliezer would agree with what you’re saying here, in the same post mentioned;
So he would agree that as long as you’re not using the word “emergence” as an explanation—since the sequence is on words which are used in common language which don’t predict anything by themselves— and are actually acknowledging the various mechanisms beneath, which you’re understanding using higher level non fundamental abstractions.
To reiterate In a reductionism post he mentions
So it’s quite clear that he’s actually fine with higher level abstractions like you’re using here, as long as they predict things. The phrase “intelligence is emergent” as what intelligence is doesn’t predict anything and is a blank phrase, this is what he was opposed to.
I wished he was a lot more clear with these things back then, it took me quite a bit of time to understand his position (it’s fairly neat imo)
Thanks for your comment, I appreciate your points, and see that Yudkowsky appreciates some use of higher-level abstractions as a pragmatic tool that is not erased by reductionism. But I still feel like you’re being a bit too charitable. I re-read the ‘it’s okay to use ‘emerge”’ parts several times, and as I understand it, he’s not meaning to refer to a higher-level abstraction, he’s using it in the general sense “whatever byproduct comes from this” in which case it would be just as meaningful to say “heat emerges from the body” which does not reflect any definition of emergence as a higher-level abstraction. I think the issue comes into focus with your final point:
But it is not correct to say that acknowledging intelligence as emergent doesn’t help us predict anything. If emergence can be described as a pattern that happens across different realms then it can help to predict things, through the use of analogy. If for instance we can see that neurones are selected and strengthened based on use, we can transfer some of our knowledge about natural selection in biological evolution to provide fruitful questions to ask, and research to do, on neural evolution. If we understand that an emergent system has reached equilibrium, it can help us to ask useful questions about what new systems might emerge on top of that system, questions we might not otherwise ask if we were not to recognise the shared pattern.
A question I often ask myself is “If the world itself is to become increasingly organised, at some point do we cease to be autonomous entities an on a floating rock, and become instead like automatic cells within a new vector of autonomy (the planet as super-organism)”. This question only comes about if we acknowledge that the world itself is subject to the same sorts of emergent processes that humans and other animals are (although not exactly, a planet doesn’t have much of a social life, and that could be essential to autonomy). I find these predictions based on principles of emergence interesting and potentially consequential.
I don’t think Eliezer uses emergence that way. He is using it the way, if a person is asked “why do hands have X muscular movement?” , one may reply “it’s an emergent phenomena” , that is the explanation which doesn’t predict anything he’s criticizing, unless a person clarifies what they mean by emergent phenomena.
A proper explanation could be: (Depending on what is meant by the word “Why”) [1]
Evolutionary: The reason why X muscular movement got selected for.
Biological/Shape/Adaptation; How it works or got implemented.
The common use of the word emergent is such that when a person is perplexed by the idea of free will, and finds a lack of contra-causal free will troubling to their preliminary intuitions , their encounter with the idea “free will is emergent” resolves the cognitive dissonance mistaking it for an explanation[2] when it holds no predictive power, and doesn’t actually work to resolve the initial confusion alongside physics.
What examples do I have in back of my mind that I think he’s criticizing this particular usage?
Eg-1: He uses the example of “Intelligence is emergent”.
In online spaces when asked, “Where is the ‘you’ in the brain if it’s all just soft flesh?” , people often say , “I am emergent” . Which doesn’t quite predict anything, like I learn nothing about when do I cease to be “I” , or why do I feel like “I” etc.
Eg-2: He uses the example of “Free will is emergent” , where he mentions the phrasing “one level emerges from the other”.
Eg-3; He uses “behavior of ant colony is emergent” in original post.
Eg-4; He also emphasizes that he’s fine with the term that chemistry “arises from” interaction between atoms as per QED. Since chemistry—or parts of it, can be predicted in terms of QED. [2]
Which he clarifies is fairly equivalent to “Chemistry emerges from interaction between atoms as per QED”.
None of these examples seem to argue against “emergence as a pattern that happens across different realms” . That seems like a different thing altogether and can be assigned a different word.
This particular usage he is criticizing, and this is the trap majority of people including my past self & a lot of people I know in real life fall into. Which is why, I think the disagreement here is mostly semantic, as highlighted by TAG. This can also be categorized as the trap of strong emergentism, or intuitions behind it such that it satisfies the human interrogation without adding anything to understanding. Moreover, the sequence in question is named “Mysterious answers” where he goes over certain concepts which are in common zeitgeist used as an explanations even when they’re not.[2]
From what I understand the way you’re using it, in emergent cycle to import over to other places Eliezer would agree with your use case. He uses this as an argument as to why maths is useful :-
It seems like your emergent cycle is closer to this. He similarly to your emergent cycle for systems also asserts probability theory as laws underlying rational belief and similarly decision theory for rational actions of all agents:-
Decision theory works because it is sufficiently similar to the goal oriented systems in the universe. He also thinks intelligence is lawful, as in it’s orderly such as following decision theory. Which seems similar to your defense in the sense of multiple leveled maps.
To further flesh the point, he would agree with you on the eyes part :-
He probably would assign the intensional/word “eye” to an extensional similarity cluster but I think you and him might still disagree on nuances:
Which is to say he would disagree with the blanket categorization of “purpose”, which can be a problematic term and leaves space for misunderstanding regarding normativity and would likely advocate for more clearer thinking and wordings along the lines highlighted above.
Although I think he would be fine with concepts like ‘agency’ in decision theories to degree to which axioms happen to coincide with that of reality—just like apples behaving like numbers listed above— since it can be bounded to physical systems amongst other considerations such as well biological agent sustaining itself etc.
Which brings us to his second potential source of disagreement, regarding analogies :[3]
Which basically goes over the necessity for having a reasoning as to why, someone suspects this analogy to work aside from being merely similar in certain aspects, because most of them don’t. Take the example of more sophisticated analogy of biological evolution extended to corporations:
Since this analogy can mislead one to think corporations can evolve on their own and someone may conclude that hence anyone can become CEO of a corporation, but it’s the case that majority of changes are due to human intelligence so that would be an example of failure by analogy.[4]
As per my amateur analysis you seem to have taken caution in your emergent cycle analogy, as to only apply it when entropy has inverse relationship, which is still quite a broad application and but on surface seems to isolate the generalization to certain systems which have sufficiently enough internal structure for the analogy to carry over under those constraints.
A thing to note here is Eliezer is a predictivist, both of these type of explanation would narrow down experience to hand’s muscle moving a certain way.
For Eliezer— a predictivist—an explanation narrows down anticipated experience.
In this post he criticized neural networks, and as we know that particular prediction of his aged poorly, but those are for different reasons, but the general point regarding analogy still stands.
Although I can see someone making the case of memetic evolution.