Yudkowsky extends this (somewhat uncharitable) reading of emergence into full straw-man when he equates emergence with magic.
Being more interested in discovering the sort of sources Eliezer was writing against but did not give examples of, than inventing a way of using the word “emergence” that would avoid that scorn, I googled “the power of emergence”. That seemed the sort of woo phrase that would likely come up in that sort of thing. Rich pickings, among which I found this:
Emergence is something magical that happens in certain systems, systems that involve lots of relatively low-level, simple agents interacting with each other and following simple rules. Somehow through those interactions a higher level order is created. The best way to think about it is to think about something like an ant colony, which is a wonderful example of emergence at work. Ant colonies involve thousands and thousands of ants, none of which have any real global information about what the colony needs to do or where the food is or where the nest is beyond its bear-bones [sic] location.
There’s more, but that’s representative. Here’s the headline paragraph of the article:
What is the relationship between the disparate elements of consciousness, communities and computer games? The answer can be found in emergence, change that occurs from the bottom up. In an exclusive interview, we asked Stephen to explain more.
All of these are attributing powers to “emergence” itself, rather than the word simply being, as some commenters would have it, a name for the very ordinary phenomenon whereby some properties of a thing are not properties of any of its parts. If that were the way that the word was being understood, we would not find a computer being described as non-emergent but a termite cathedral as emergent. The mystery is being worshipped.
Being more interested in discovering the sort of sources Eliezer was writing against but did not give examples of
I don’t believe this is a useful endeavor. To put it simply, there is a ton of stupid stuff out there (most stuff that gets written is stupid), so picking a particular batch of bad statements to analyze gives very little evidence overall because they are unlikely to be representative of the entirety of writing or thinking on a matter. Particularly when your selection process is biased (through correlations between rhetoric and substance) towards finding precisely the stupidest of stuff among it.
Here are a few more hits I found notable on “the power of emergence”:
This is entirely unsurprising in light of the above. A lot of stuff is written, most of it is bad, so no wonder many of the leading hits are bad. It would be truly surprising if this didn’t happen.
Yes, you absolutely would! In a sufficiently large dataset of words written by humans, you are almost always able to stumble upon some examples of any other trait you’re interested in… regardless of the overall distribution or the value of the highest-quality examples.
Rich pickings, among which I found this
When Eliezer was rallying against the mysterianism of the past, he was using the example of the world’s leading authorities, thinkers, and scientists engaging in (supposedly) bad epistemology. As an illustrative example, he wasn’t picking out a random article entry or book, but rather the words of Lord Kelvin as he argued for “elan vital.”
But then, when commenters rightly accused him of hindsight bias and making light of the genuine epistemological conundrums humanity was facing in the past, Eliezer-2007 engaged with a weakman. He argued the futility of emergence as supposed microcosm for how the irrationality of today’s world results in grossly inadequate philosophy. But he didn’t engage with the philosophy itself, as presented by its leading proponents, but instead took shots at much shakier versions argued by confused and (frankly) stupider people.[1]
As I said above, the following exchange remains illustrative of this:
HI: Aren’t superconductivity and ferromagnetism perfect examples of emergent phenomena?
Eliezer Yudkowsky: Yes. So are non-superconductivity and non-ferromagnetism. That’s the problem.
Perplexed: Uh. No. Non-superconductivity is not usually considered as an example of emergence. Because the non-superconductive system is composed of smaller subsystems which are themselves non-superconductive. Same goes for non-ferromagnetism. Not “emergent” because nothing new is emerging from the collective that was not already present in the components.
As an aside, it’s reflective of what Eliezer does these days as well: almost exclusively arguing with know-nothing e/acc’s on Twitter where he can comfortably defeat their half-baked arguments, instead of engaging with more serious anti-doom arguments by more accomplished and reasonable proponents.
Particularly when your selection process is biased (through correlations between rhetoric and substance) towards finding precisely the stupidest of stuff among it.
I did not have to scrape any barrels to find those examples. They were all from the first page of Google hits. Steven Johnson’s book is not new-age woo. Of my First/Second/Third examples, the first is a respectable academic text, and only the third is outright woo. So I do not think that “the power of emergence” is optimised for finding nonsense (it was the only phrase I tried), nor am I depending on the vastness of the internet to find these. Certainly, my search was biased — or less tendentiously, intended — to find the sort of thing that might have been Eliezer’s target: not new-age woo, but the things respectable enough to be worth his noticing at all. Lacking the actual sources he had in mind, what I found does look like the right sort of thing. I notice that the Steven Johnson book predates his posting.
As I said above, the following exchange remains illustrative of this:
It’s worth quoting a little further:
anonymous: The apparent disagreement here, comes from different understandings of the word “non-superconductivity”.
By “non-superconductivity”, Yudkowsky means (non-super)conductivity, i.e. any sort of conductivity that is not superconductivity. This is indeed emergent, since conductivity does not exist at the level of quantum field.
By “non-superconductivity”, Perplexed means non-(superconductivity), i.e. anything that is not superconductivity. This is not emergent as Perplexed explained.
In your footnote you say:
As an aside, it’s reflective of what Eliezer does these days as well: almost exclusively arguing with know-nothing e/acc’s on Twitter where he can comfortably defeat their half-baked arguments, instead of engaging with more serious anti-doom arguments by more accomplished and reasonable proponents.
Selection effect again? I don’t look at Twitter, but I did notice that Eliezer recently gave a three-hour interview and wrote a book on the subject.
Consider the following two possible interpretations of the dialogue between Eliezer and Perplexed. First:
Alice: Aren’t superconductivity and ferromagnetism perfect examples of emergent phenomena?
Bob: Yes. So are “regular” conductivity and “regular” ferromagnetism. That’s the problem.
Alice: … what? Yeah, of course they are also examples of emergent phenomena; X being an emergent phenomenon obviously doesn’t prevent an unrelated Y from being an emergent phenomenon. What does that have to do with anything? There’s no problem here.
And second:
Alice: Aren’t superconductivity and ferromagnetism perfect examples of emergent phenomena?
Bob: Yes. So are “the lack of superconductivity” and “the lack of ferromagnetism”. That’s the problem.
Alice: Ah, I understand how those could be problematic: if both X and “the lack of X” are described the same way, then the descriptor becomes a meaningless semantic stopsign that muddies our thinking. However, on the object level in this case, the lack of superconductivity is clearly not an emergent phenomenon (and same for the other example).
I rejected the first interpretation because I trust Eliezer’s intelligence enough that I don’t think he would go for a random non sequitur that reflects nothing about the topic at hand. anonymous’s interpretation doesn’t seem to be likely to be what Eliezer intended (and even if it is the correct interpretation, it just means Eliezer was engaging in a different, yet simpler error).
Selection effect again? I don’t look at Twitter, but I did notice that Eliezer recently gave a three-hour interview and wrote a book on the subject.
Yeah, it’s probably a selection effect; the vast majority of Eliezer’s public communication comes on Twitter (unfortunate, but better than Facebook, I suppose...). Eliezer’s interview and podcast appearances, as well as (AFAIK) the book, also seem entirely geared towards a smart-but-not-technically-proficient-in-alignment audience as its primary target, in line with MIRI’s focus on public outreach to more mainstream audiences and institutions.
Nevertheless, having not read the book itself, I should suspend judgement on it for now.
Thanks for your comment, but I think it misses the mark somewhat.
While googling to find someone who expresses a straw-man position in the real-world is a form of straw-manning itself, this comment goes further to misrepresent a colloquial use of the word “magical” to mean literal (supernatural) “magic”.
While I haven’t read the book referenced, the quotes provided do not give enough context to claim that the author doesn’t mean what he obviously means (to me at least) that the development of an emergent phenomena seems magical… does it not seem magical? Seeming magical is not a claim that something is not reducible to its component parts, it just means it’s not immediately reducible without some thorough investigation into the mechanisms at work. Part and parcel of the definition of emergence is that it is a non-magical (bottom-up) way of understanding phenomena that seem remarkable (magical), which is why he uses a clearly non-supernatural system like an anthill to illustrate it.
Despite all this, the purpose of the post was to give a clear definition of emergence that doesn’t fall into Yudkowsky’s strawman—not a claim that no one has ever used the word loosely in the past. As conceded in the preamble (paraphrasing) I don’t expect something written 18 years ago to perfectly reflect the conceptual landscape of today.
Despite all this, the purpose of the post was to give a clear definition of emergence that doesn’t fall into Yudkowsky’s strawman
I am taking Eliezer’s word for it that he has encountered people seriously using the word “emergence” in the way that he criticises, that it is not a straw man. The sources I found bolster that view. (In the Steven Johnson interview I took him to be using “magical” figuratively, but as various people have pointed out, if someone thinks the world is weird, they’re weird for thinking that the world should carry on behaving just like the tiny fragment of it they know about so far.) To respond by inventing a different concept and calling it by the same name does not bear on the matter.
This is an error I see people making over and over. They disagree with some criticism of an idea, but instead of arguing against that criticism, they come up with a different idea and use the same name for it. This leaves the criticism still standing. Witness the contortions people go through to defend a named theory of causal reasoning (CDT, EDT, etc.) by changing the theory and keeping the same name. That is not a defence of the theory, but a different theory. That different theory may be a useful new development! But that is what it is, not a defence of the original theory.
This is an error I see people making over and over… That different theory may be a useful new development! But that is what it is, not a defence of the original theory.
I think this is the crux of our disagreement. Yudkowsky was denying the usefulness of a term entirely because some people use it vaguely. I am trying to provide a less vague and more useful definition of the term—not to say Yudkowsky is unjustified in criticising the use of the term, but that he is unjustified in writing it off completely because of some superficial flaws in presentation, or some unrefined aspects of the concept.
An error that I see happening often is throwing out the baby with the bathwater, and I’ve read people on Less Wrong (even Yudkowsky I think, though I can’t remember where, sorry) write in support of ideas like “Error Correction” as a virtue and Bayesian updating whereby we take criticisms as an opportunity to refine a concept rather than writing it off completely.
I am trying to take part in that process, and I think Yudkowsky would have been better served had he done the same—suggested a better definition that is useful.
Being more interested in discovering the sort of sources Eliezer was writing against but did not give examples of, than inventing a way of using the word “emergence” that would avoid that scorn, I googled “the power of emergence”. That seemed the sort of woo phrase that would likely come up in that sort of thing. Rich pickings, among which I found this:
So, no straw man at all. Am I unfairly chopping that off? The context is an interview with Steven Johnson, the author of “Emergence: The Connected Lives of Ants, Brains, Cities and Software” (2002). Here’s a longer extract:
There’s more, but that’s representative. Here’s the headline paragraph of the article:
And he gives us “magical”.
Here’s another, more serious review of the same book.
Here are a few more hits I found notable on “the power of emergence”:
First.
Second.
Third.
All of these are attributing powers to “emergence” itself, rather than the word simply being, as some commenters would have it, a name for the very ordinary phenomenon whereby some properties of a thing are not properties of any of its parts. If that were the way that the word was being understood, we would not find a computer being described as non-emergent but a termite cathedral as emergent. The mystery is being worshipped.
I don’t believe this is a useful endeavor. To put it simply, there is a ton of stupid stuff out there (most stuff that gets written is stupid), so picking a particular batch of bad statements to analyze gives very little evidence overall because they are unlikely to be representative of the entirety of writing or thinking on a matter. Particularly when your selection process is biased (through correlations between rhetoric and substance) towards finding precisely the stupidest of stuff among it.
This is entirely unsurprising in light of the above. A lot of stuff is written, most of it is bad, so no wonder many of the leading hits are bad. It would be truly surprising if this didn’t happen.
Yes, you absolutely would! In a sufficiently large dataset of words written by humans, you are almost always able to stumble upon some examples of any other trait you’re interested in… regardless of the overall distribution or the value of the highest-quality examples.
When Eliezer was rallying against the mysterianism of the past, he was using the example of the world’s leading authorities, thinkers, and scientists engaging in (supposedly) bad epistemology. As an illustrative example, he wasn’t picking out a random article entry or book, but rather the words of Lord Kelvin as he argued for “elan vital.”
But then, when commenters rightly accused him of hindsight bias and making light of the genuine epistemological conundrums humanity was facing in the past, Eliezer-2007 engaged with a weakman. He argued the futility of emergence as supposed microcosm for how the irrationality of today’s world results in grossly inadequate philosophy. But he didn’t engage with the philosophy itself, as presented by its leading proponents, but instead took shots at much shakier versions argued by confused and (frankly) stupider people.[1]
As I said above, the following exchange remains illustrative of this:
As an aside, it’s reflective of what Eliezer does these days as well: almost exclusively arguing with know-nothing e/acc’s on Twitter where he can comfortably defeat their half-baked arguments, instead of engaging with more serious anti-doom arguments by more accomplished and reasonable proponents.
I did not have to scrape any barrels to find those examples. They were all from the first page of Google hits. Steven Johnson’s book is not new-age woo. Of my First/Second/Third examples, the first is a respectable academic text, and only the third is outright woo. So I do not think that “the power of emergence” is optimised for finding nonsense (it was the only phrase I tried), nor am I depending on the vastness of the internet to find these. Certainly, my search was biased — or less tendentiously, intended — to find the sort of thing that might have been Eliezer’s target: not new-age woo, but the things respectable enough to be worth his noticing at all. Lacking the actual sources he had in mind, what I found does look like the right sort of thing. I notice that the Steven Johnson book predates his posting.
It’s worth quoting a little further:
In your footnote you say:
Selection effect again? I don’t look at Twitter, but I did notice that Eliezer recently gave a three-hour interview and wrote a book on the subject.
Consider the following two possible interpretations of the dialogue between Eliezer and Perplexed. First:
And second:
I rejected the first interpretation because I trust Eliezer’s intelligence enough that I don’t think he would go for a random non sequitur that reflects nothing about the topic at hand. anonymous’s interpretation doesn’t seem to be likely to be what Eliezer intended (and even if it is the correct interpretation, it just means Eliezer was engaging in a different, yet simpler error).
They still do not represent an “example of the world’s leading authorities, thinkers, and scientists engaging in (supposedly) bad epistemology,” as in the example Eliezer picked out of Lord Kelvin committing a supposedly basic epistemological error. And as a result of this, the argument Eliezer was implicitly making also fails.
Yeah, it’s probably a selection effect; the vast majority of Eliezer’s public communication comes on Twitter (unfortunate, but better than Facebook, I suppose...). Eliezer’s interview and podcast appearances, as well as (AFAIK) the book, also seem entirely geared towards a smart-but-not-technically-proficient-in-alignment audience as its primary target, in line with MIRI’s focus on public outreach to more mainstream audiences and institutions.
Nevertheless, having not read the book itself, I should suspend judgement on it for now.
A lot of trouble could have been saved by distinguishing strong and weak emergence.
Thanks for your comment, but I think it misses the mark somewhat.
While googling to find someone who expresses a straw-man position in the real-world is a form of straw-manning itself, this comment goes further to misrepresent a colloquial use of the word “magical” to mean literal (supernatural) “magic”.
While I haven’t read the book referenced, the quotes provided do not give enough context to claim that the author doesn’t mean what he obviously means (to me at least) that the development of an emergent phenomena seems magical… does it not seem magical? Seeming magical is not a claim that something is not reducible to its component parts, it just means it’s not immediately reducible without some thorough investigation into the mechanisms at work. Part and parcel of the definition of emergence is that it is a non-magical (bottom-up) way of understanding phenomena that seem remarkable (magical), which is why he uses a clearly non-supernatural system like an anthill to illustrate it.
Despite all this, the purpose of the post was to give a clear definition of emergence that doesn’t fall into Yudkowsky’s strawman—not a claim that no one has ever used the word loosely in the past. As conceded in the preamble (paraphrasing) I don’t expect something written 18 years ago to perfectly reflect the conceptual landscape of today.
I am taking Eliezer’s word for it that he has encountered people seriously using the word “emergence” in the way that he criticises, that it is not a straw man. The sources I found bolster that view. (In the Steven Johnson interview I took him to be using “magical” figuratively, but as various people have pointed out, if someone thinks the world is weird, they’re weird for thinking that the world should carry on behaving just like the tiny fragment of it they know about so far.) To respond by inventing a different concept and calling it by the same name does not bear on the matter.
This is an error I see people making over and over. They disagree with some criticism of an idea, but instead of arguing against that criticism, they come up with a different idea and use the same name for it. This leaves the criticism still standing. Witness the contortions people go through to defend a named theory of causal reasoning (CDT, EDT, etc.) by changing the theory and keeping the same name. That is not a defence of the theory, but a different theory. That different theory may be a useful new development! But that is what it is, not a defence of the original theory.
I think this is the crux of our disagreement. Yudkowsky was denying the usefulness of a term entirely because some people use it vaguely. I am trying to provide a less vague and more useful definition of the term—not to say Yudkowsky is unjustified in criticising the use of the term, but that he is unjustified in writing it off completely because of some superficial flaws in presentation, or some unrefined aspects of the concept.
An error that I see happening often is throwing out the baby with the bathwater, and I’ve read people on Less Wrong (even Yudkowsky I think, though I can’t remember where, sorry) write in support of ideas like “Error Correction” as a virtue and Bayesian updating whereby we take criticisms as an opportunity to refine a concept rather than writing it off completely.
I am trying to take part in that process, and I think Yudkowsky would have been better served had he done the same—suggested a better definition that is useful.