Could Democritus have predicted intelligence explosion?
Also see: History of the Friendly AI concept.
The ancient atomists reasoned their way from first principles to materialism and atomic theory before Socrates began his life’s work of making people look stupid in the marketplace of Athens. Why didn’t they discover natural selection, too? After all, natural selection follows necessarily from heritability, variation, and selection, and the Greeks had plenty of evidence for all three pieces. Natural selection is obvious once you understand it, but it took us a long time to discover it.
I get the same vibe from intelligence explosion. The hypothesis wasn’t stated clearly until 1965, but in hindsight it seems obvious. (Michael Vassar once told me that once he became a physicalist he said “Oh! Intelligence explosion!” Except of course he didn’t know the term “intelligence explosion.” And he was probably exaggerating.)
Intelligence explosion follows from physicalism and scientific progress and not much else. Since materialists had to believe that human intelligence resulted from the operation of mechanical systems located in the human body, they could have realized that scientists would eventually come to understand these systems so long as scientific progress continued. (Herophilos and Erasistratus were already mapping which nerves and veins did what back in the 4th century B.C.)
And once human intelligence is understood, it can be improved upon, and this improvement in intelligence can be used to improve intelligence even further. And the ancient Greeks certainly had good evidence that there was plenty of room above us when it came to intelligence.
The major hang-up for predicting intelligence explosion may have been the the inability to imagine that this intelligence-engineering could leave the limitations of the human skull and move to a speedier, more dependable and scalable substrate. And that’s why Good’s paper had to wait until the age of the computer.
</ speculation>
An alternative hypothesis that might explain the evidence: Ideas like natural selection, atomism and intelligence explosion are independently invented a great many times, but the apparent scarcity comes from the factors that lead to them being recorded or spread not being in place. For example Darwin was nowhere near the first to talk about natural selection, but the combination of a critical mass of scientific evidence and a receptive cultural climate made it possible to spread it. Conversely, Democritus may have had the idea of an intelligence explosion but there was no reason for it to be recorded and passed down.
This is a disappointing post.
On atomism: the atomic theory of the pre-Socratics was just another in a series of essentially navel-gazing theories. Today we privilege and single out that theory for the obvious reasons, but it doesn’t deserve much more praise than the alternatives. The philosophers involved had absolutely no way to test it and in fact weren’t interested in doing so. And the alternatives were as convincing or more convincing, given the available evidence, as their speculation.
Suppose some SF writer in 1940s wrote a hack piece about an alien invasion where we turn out to live in “ten dimensions” and the aliens are coming out of the extra ones. And suppose string theory actually comes through and proves itself in another 20-30 years. Giving credit to the pre-Socratic atomists for figuring out atoms would be as silly as giving credit to that SF author for figuring out before everyone else that we really live in 10 dimensions.
On natural selection: the hard part with natural selection is not figuring out that it happens. It’s figuring out that it happens and it’s responsible for a large portion of evolutionary diversity. That’s not obvious at all, to put it very mildly. In retrospect or not. And to even formulate that hypothesis—which, given only everyday knowledge, seems way too bizarre to even consider—requires a huge body of biological knowledge, taxonomy, anatomy, the Galapagos islands, etc. Darwin’s achievement was not to say “hey, natural selection happens”. It was to say “hey, I know that natural selection on the face of it looks like it could drive at most some small change within a species, but actually, THIS is the mechanism mostly responsible for EVERYTHING”.
But more than that, you can’t even talk about natural selection before you’re convinced there’s evolution. Biologists were by and large convinced there was evolution by the time Darwin showed up. How the hell would ancient Greeks be able to get there just by sitting down and thinking “from first principles”? You need to dissect hundreds of species and see how similar they are inside. You need fossil records and something to compare them to. You need geology to tell you that Earth is really old. You need Lamarck.
On intelligence explosion: not sure where to begin, maybe with “they could have realized that scientists would eventually come to understand these systems so long as scientific progress continued”. They didn’t have any conception of scientific progress. They didn’t know anything about “scientists”, either. And they had absolutely no basis for thinking that just because people are trying to understand e.g. intelligence, that they’re actually going to succeed.
I don’t agree, but let’s talk about something else.
Agreed. There were good reasons from the “physics” of the day to reject atomism. If I remember correctly, Aristotle’s argument went something like this:
The existence of atoms requires the existence of a void.
How could atoms move around if there wasn’t any space for them to move to?
The speed of an object is determined by the “thickness” (density) of the medium it is travelling through.
If you drop a ball through the air (a very thin medium), it will move much faster than a ball dropped through water (a thicker medium) and faster still than a ball dropped through a jar of honey (a very thick medium).
A void doesn’t have any “thickness”.
If you dropped a ball through a void it would move infinitely fast.
Actual infinites are impossible.
Therefore, the void (and atoms which require a void) does not exist.
Sounds pretty reasonable, doesn’t it?
I think so. And there were other reasons, too, for Aristotle’s theory of the four elements to look more appealing than the atomism he was rejecting. For example, it attempted to explain hot and cold by incorporating them as basic qualities of the elements and giving some rules about how one can turn into the other. Taking hot vs cold and dry vs wet as the basic qualities, we have four possibilities:
hot and dry → fire
hot and wet → air
cold and dry → earth
cold and wet → water
Transitions between these four that change only one quality are easy and more common (like water->air by evaporation, or air->water by rain), while those that change both qualities (air<-->earth, fire<-->water) are harder, next to impossible. This actually corresponds to observed phenomena to some degree. The atomic theory had nothing of the kind and didn’t even attempt to account for things like temperature.
Why not? After all, you had said “the hard part with natural selection is not figuring out that it happens. It’s figuring out that it happens and it’s responsible for a large portion of evolutionary diversity.”
It’s a fair actual question, even if the answer is “they couldn’t have,” so “the hell” doesn’t belong.
You don’t. You could assume it was infinitely old, and get other things right.
The question is how something would have been possible, if it was possible. Your dismissal is too quick and seems based on showing how the Ancient Greeks couldn’t have readily used the same evidence and thought that actually worked in recent history.
What I meant to say (and thought it was clear from the context, but was possibly wrong) was that you can’t talk about the important thing about natural selection—the “hard part” I’d mentioned earlier—without knowing about evolution. When people talk about natural selection as Darwin’s great achievement that could or could not have been achieved earlier, it is this hard part they are referring to (unless they’re confused and don’t understand this, in which case it this hard part they ought to be referring to).
I think the question is rather on the rhetorical side (and I proceeded to give a sample of reasons for thinking so), so “the hell” is there to hint at the exasperation at a post that seems blithely naive.
Let me augment that: really old and slowly changing. By the end of the 18th century, geologists knew that shark teeth found on mountain tops are likely explained by the fact that a long time ago, these rocks were under water. The idea that geological processes happen very slowly, in “deep time”, and accumulate to produce huge changes was a direct inspiration to biologists in coming up with evolution.
Lukeprog’s post didn’t ask “how could they have discovered this through means other than with what modern science discovered it”. Instead, it said “these discoveries follow from a few basic first principles and they could have just thought about them, but didn’t”. And my dismissal works by pointing out that this view is incredibly naive and ignorant of the massive amount of evidence that modern science needed to accumulate before these discoveries could be made.
We were both wrong; here’s the relevant part:
So, the Greeks had evidence for some intermediary conclusions, and it is asserted that they could have worked their way from there to a good understanding of natural selection. Not necessarily using first principles to help discover natural selection.
You haven’t said why natural selection wouldn’t follow from those things listed (although saying why it would is the OP’s responsibility), or that the Greeks didn’t have enough evidence for those things. Instead, you addressed the possibility of going from those to a good understanding of natural selection, arguing that ”...to even formulate that hypothesis...requires...” and you listed things that the historical human discoverers actually required to get it, when the issue is the minimum it should have required.
But your response is an appeal to incredulity that puts far too much weight on what led people to discover evolution to convincingly address whether or not a different way was possible. Showing that a set of things was sufficient doesn’t show they were necessary.
As the OP thought the argument from “heritability, variation, and selection” to natural selection strong enough to be implicit, you should argue for a reason to believe he would have falsely believe that before dismissing the idea. Without that, all we have is a clash of intuitions, and on your part it doesn’t look like you’ve updated much on lukeprog’s apparent extreme confidence that it could have been so derived (which I infer from his unfortunate failure to argue for the point).
As pointed out by others, the atomism of Democritus is not a solid theory, but a blue vs. green position. One fellow thinks that if you divide something repeatedly, you’ll never stop being able to divide it; another fellow disagrees with him, they come up with elaborate verbal justifications for their positions.
So why don’t we consider some Greek theories that were actually solid theories? Aristarchos argued that, since the sun was so much bigger than the Earth, it made more sense for the Earth to orbit the sun than the sun to orbit the Earth.
Other mathematicians thought about this theory, and eventually responded with, “well, if that were true, we should expect to see parallax effects on other stars. We don’t, so either everything orbits around the Earth or the stars are unimaginably far away.”
Well, it turns out that the stars are unimaginably far away, and so the parallax effects are too weak to see with the naked eye. But the Greeks had their priors in the right place for the evidence they had access to.
Likewise, things like natural selection (as we understand it) and intelligence explosion don’t seem like they’re obvious without evidence to back them up, and could be reasonably dismissed without that evidence.
People here routinely make statements regarding the universe about which the available evidence is still inconclusive. If we care about the accuracy of such claims it seems worthwhile to pay attention to the how such statements worked out in the past.
The right question, of course, is not “Could Democritus have predicted an intelligence explosion”, but rather “How could Democritus have predicted an intelligence explosion?”
(Similarly, a question that I am particularly interested in: “How could Isaac Newton have invented general relativity?”)
What would be the minimum of knowledge you would need to tell him about so that the concept would be only a single inferential step away, and thus so that he would have thought of it spontaneously? (And thus what corresponding concept(s) are we missing?)
My guess at the answer to the Democritus question is similar to yours: the computer, and industrial technology more generally, really helps to implant physicalism into one’s “gut” anticipations.
You say it like it’s a mathematical theorem or an experimentally tested physical model. In fact, there are plenty of caveats in the former approach, and there is a falsifying evidence in the latter (no recognizable super-intelligence on Earth or in space, billions of years after the Big Bang). Imagine that the Moore’s law fizzles in the next few years, where will your intelligence explosion be? Or maybe there is a law that states that stupidity grows faster than intelligence, resulting in the world overrun by idiots, killing further progress.
Yes, that’s quite irritating. If it was just me, I would acknowledge my ignorance. But even people like Shane Legg are less certain about the possibility of an intelligence explosion than people associated with SI. Which makes me wonder what it is that they know and he doesn’t. Shane Legg writes:
An intelligence explosion is a possibility. But some people here seem to think it is almost a certainty. That’s just weird.
Peter Corning might be the person to ask about this question. He’s studied synergy in evolution.
The idea of a Tower of optimisation seems relevant here. Daniel Dennett, Alan Winfield and myself have worked on this.
I think this illustrates the main perspective which is needed to be able to make predictions on this topic.
“Tower of optimisation” may just be the coolest-sounding theory I’ve heard this month.
The wording of that sentence confused me enough to make me think about the subject for these past few hours. Good job, I guess. ^^
I’m not sure it’s quite accurate to say they didn’t discover natural selection. They tended to believe a lot of other nonsense, though, which made it difficult to fully realize the idea.
Aristotle considered something like natural selection but rejected it because it didn’t otherwise fit his worldview (which was opposed to atomism). Here he considers selection and variation, although he doesn’t mention heritability explicitly.
Source is http://classics.mit.edu/Aristotle/physics.2.ii.html, part 8. The reference to Empedocles is talking about http://en.wikipedia.org/wiki/Empedocles#Cosmogony, who arguably came closer, but it sounds like he had some pretty strange ideas about the origins of things. Anyway, I’d suggest that incorrect (scientific and metaphysical) beliefs were at least as problematic as lack of imagination in this case.
I don’t know too much about Greek philosophy, though, so I’d be interested to hear someone explore this further.
A later atomist, Lucretius wrote about this in more detail. A translation is here you’ll want to ctrl-f for his section on the origins of vegetable and animal life. His position is much like that of Empedocles.
I don’t see why these wouldn’t count as coming up with the idea of natural selection. They obviously lacked any understanding of genetics or how heredity functions. They didn’t have the particular example of the Darwin’s finches to illustrate fine-grained evolution—but Lucretius is talking quite explicitly about about fitness and survival as an explanation for why animals and plants are the way they are.
Successful scientific theories are often prefigured before the experimental capacity and conceptual framework exists for the hypothesis to seem sensible. The interesting question is—how can we identify the equivalent hypotheses of today?
I came here to post this. De rerum natura is clearly a copy of a copy of a reconstruction of a translation of Gary Drescher’s Good and Real, sent back in time 2300 years.
Lots of specific pieces of science fiction come to mind.
Yes: I don’t see any evidence there that Aristotle realised that traits were inherited, or that radiation explains the diversity of life. It seems safe to say that he didn’t really “get” it.
William Jones got quite a bit closer—in 1786 - noting that many languages had evolved from a common ancestor.
Of course Aristotle realized traits were inherited—in particular he name’s semen and menstrual blood as the directors of the child’s development. Heredity is a really obvious thing to civilizations with that practice animal husbandry, and you know—sexual reproduction. And of course no one knew anything about mutation by radiation.
I was actually referring to radiating organsims—i.e. adaptive radiation—though what I wrote could be seen as being somewhat ambiguous.
http://intelligenceexplosion.com/ paints a pretty naive picture.
Designing artificial intelligences is a skill composed of many sub-skills of varying levels of difficulty.
Machines are better at many of those today.
To say “one day” creates the bizarre and totally incorrect idea that machine skill will overtake human skill on the specified task on one day—and after that things will go much faster. What is actually happening is that machine skill has been gradually overtaking human skill in a range of domains, one domain at a time. The process has been going on for many decades now. Composite tasks that involve many skills—like designing complex machines by mastering hardware and software engineering—will thus be accelerated gradually by the increase in performance of machines.
This appears to be a confusion propagated by those wanting to exaggerate the risks of machine intelligence. I have a web page about this and must have told people about this publicly a dozen times now, but few seem to listen, and consequently the same nonsense gets repeatedly pumped out, deluding wave after wave of newcomers about this point. Perhaps brevity excuses this instance, but it is beginning to look like a deliberate deception—to try and make things seem worse than they appear, by making the transition to intelligent machines seem more sudden than is likely.
This video shows the position that I am arguing against (at 18:43).
It might make good propaganda, but it is based on an inacurate picture of what it likely to happen. We know enough to see that already.
I agree with almost all of this article. However, the conclusion that the transition won’t happen quite suddenly seems to me to be wrong.
Many things seem to go through a technological stage of progress. Music, for example. It became possible to record it tunefully in the 1900′s, and by 1960, recording had essentially been mastered to the fidelity the human ear could hear.
It became possible to create electronic sounds to some extent in the 1950′s, and this led to the electric guitar, the Hammond organ, and various analogue synthesizers. Then came digital synthesizers, which over a relatively short time displaced the analogues, and led to a point in the 1990′s when it became possible to create any sound. Now your phone is powerful enough to do this.
In the 1970′s simple digital light detectors existed. These became consumer digital cameras in the late 1990′s. Now we have basically reached a level where pixels are no longer an issue, and prices have dropped immensely.
Digital flat screens were science fiction for decades, and comparatively suddenly became possible, then expensively affordable, then cheaper than CRT’s.
In each case there’s a longish incubation period where nothing much apparently changes for some years. Then there’s a rush of progress over little more than a couple of decades, leading to a new status quo where the old technology is completely displaced.
AI is starting to stir. It’s had a long period where initial success was replaced by apparent stasis for some time. But now we are seeing real progress again, and I suspect a period of disruptive change caused by AI technologies is not that far off.
To make a prediction here—we will go from having essentially useless AI to human level AI in around a decade or two—just as we have seen with digital cameras, displays, synths etc. The biggest uncertainty in this is which decade it will be. And the machines won’t stop at human level—they will drive straight through and keep going over about a 5 year period. And it’s only after that has happened that progress may start speeding up because of it.
Uh, to me that looks like 4 examples of gradual progress, and 0 examples of explosions (that is, none that are more like fooms than gradual curves).
How are you defining “explosion” though? A plot of the number of splitting nuclei per unit time in a recently-detonated nuclear bomb looks like a gradual curve—if viewed on an appropriate timescale...
It doesn’t sound as though we disagree too much. I expect progress on billion year timescales, though it won’t be so dramatic after a while. I’m not arguing for low levels of disruption—but I don’t think that systematically exaggerating the expected level of disruption is particularly helpful.
Once a transparently constructed AGI becomes a good programmer it can improve itself directly. A tight feedback loop like this is rather different from the rest of the progress in AI so far.
Sure—though before machine programmers can automatically program other machine programmers they will be able to automatically program sort routines, test routines, search routines, perform refactoring, compile code, check code, find bugs, fix bugs—and so on. Those things speed up development too. The autocatalytic aspect of this is not going to start at some point in the future. It started decades ago—centuries ago if you trace the phenomenon to its roots a bit more enthusiastically.
Did the ancient Greeks even have a concept of humans as varying over time? Certainly they didn’t have a single machine which did anything even remotely like thinking.
I quite clearly remember growing up in a universe which was static. People even talked about mechanisms to create hyrdrogen in deep space in order to keep the density constant in light of the Hubble expansion. I REMEMBER seeing the big bang theory (not the TV show!) and just marveling at the audacity. How could the universe have started? What was here before?
And quite similarly I remember thinking there is “history” and there is “now.” History had things like WWII and the bubonic plague in it, now had civil rights, the Soviet Union, and the Vietnam War. I was probably in my 30s when I finally realized that 1) we ARE in hisotry and 2) it ain’t slowing down, if anything its speeding up.
When I put myself in the place of the greeks, humans would seem static, human culture would seem largely static, machines would seem darn near irrelevant to anything.
I can’t imagine I could look forward to an intelligence explosion until I had some sense that looking backward there had been an intelligence explosion. Even now you can talk to Rabbis who will argue with you about evolution by asking you if you really believe you are descended from a monkey. Did ANYONE in ancient greece have any idea that we had any other relation to monkeys or horses or dogs other than sharing the planet with them?
Yes, the atomists did believe that the world changed, that the time of humans was a transient epoch in the history of the universe. Lucretius talked specifically about this. We don’t have any direct access to Greek atomists, but Lucretius seems to be copying Greek sources.
The concept seems to me to involve higher levels of abstraction than natural selection. And note that even in natural selection, two concepts—“variation” and “differential survival” are indeed obvious, but the third—“heritability”—relies on lots of complex machinery, and we still have no good explanation how it came about in the first place.
To get the “intelligence explosion” idea, you need to assemble “goal-seeking behaviour” and “cross-domain generalization”, add “universality of computation”, “substrate independence” and “recursive self-improvement” into the mix, and these are complex ideas to start with.
The non-obvious part to me is “recursive self-improvement”. If I imagine myself looking at my own source code, and trying to figure out which part of it is responsible for how many dual N-back steps I can cope with, generalizing (admittedly from the limited perspective of having used a merely human brain) suggests that the search might prove fruitless - there may not be a single “line of code” where this limitation is encoded as a constant.
Could democritus have predicted an Intelligence Explosion?
Only if he had been superhumanly able to follow arguments through to the maximum extent of their consequences.
The number and complexity of the intervening steps, listed by morendil, put “FOOM from Atoms alone” firmly beyond the predictive power from limited evidence exhibited by any historic person who in the end turned out to be right. Which is to say, beyond Democritus, as he perhaps marks the upper end of this scale. There is good reason to believe that there is not sufficient working memory available to members of species Homo Sapiens to discover “FOOM from Atoms alone.” I mean, none of the greatest scientists were near that good.
I agree that physicalism affords the possibility of an Intelligence Explosion from the observed existence of minds, and the physical limits to the size and power of possible minds, compared to the size and power of observed minds.
The laws of physics allow minds.
The laws of physics put some upper limit on the size and power of minds, the limits are many orders of magnitude greater than those of observed minds.
Therefore, there is plenty of room above us.
From this we can conclude that superintelligent minds are permitted by physics, but not necessarily that one can jump from intelligence to superintelligence through an intelligence explosion.
We know something about physical limits of Minds. Less is known about the limits to the progress of science, and it’s more relevant proxy, engineering. We do not know if the maximally powerful engineering available to evolved minds is above the level required for initiating the chain-reaction of recursive self-improvement, initiating an intelligence explosion towards the physical limits for minds.
This begs the question: Is the ceiling of engineering prowess available to evolved minds lower than the floor of engineering prowess required for recursive self improvement?
To the best of my knowledge, this question is undecided.
Is there good evidence in one direction or the other?
Superintelligence by way of a controlled Intelligence Explosion seems harder than General Artificial Intelligence by Uploading by a large amount. Like intergalactic travel is harder than interstellar travel by a large amount.
An Intelligence Explosion may permitted under physicalism, yet be prohibited by cost and complexity, like intergalactic travel is permitted under physicalism, yet prohibited by cost and engineering complexity.
Thereby saving minds such as we, around the minimum level of intelligence required for general intelligence, from being displaced by minds that are powerful to the maximum degree that the limits-of-physics allow.
Safety wise initiating an Intelligence Explosion seems around as safe as going to the moon by riding on the heat and pressure of a nuclear chain reaction instead of on the tip of a large chemical rocket.
I still don’t perceive it to be anywhere close to being “obvious” that an intelligence explosion, as opposed to learning and intelligence amplification, is more than a possibility (in the sense that is far from being certain).
It seems like a false dichotomy to me. “Explosion” in this context surely just refers to rapid exponential growth—as seen in an a nuclear explosion. We are seeing that already in functional intelligence, and it is happening through education and augmentation.
Right! I started assuming Intelligence Explosion followed from physicalism, since Luke is a cool guy, and he said so. I think I changed my mind somewhere in the middle of writing the post above. Maybe stop at “Oh! Superintelligence!” instead of also “Oh! Intelligence explosion!”, but Michael Vassar is smarter than me, so I’m probably missing something.
Oh. Crap. I think I reversed my position again.
Why? Well, a sufficiently powerful mind that is given enough time should explode or extinguish.
When I was a kid, my parents, especially my dad, used to talk about difficulties he had in raising me, and how they were related to difficulties that he had as a kid with his parents.
And I thought “that means I’ll get to grow up better than my dad. And when I grow up and have kids, I’ll have an advantage, and my kids will be able to grow up better off than me. And eventually my children’s children’s children will be rad as all get-out.”
And while I hadn’t really noticed until now, that was my point of reference for “intelligence explosion.” It now seems clear to me that (a) most people haven’t even begun to advance along this line, and/or see it as having some bounded destination, and (b) other types of changes will blow these out of the water, in the way that memetic evolution in humans blew genetic evolution out of the water. And honestly, that’s scary. Because the improved-parenting intelligence improvement comes with a responsibility and morality improvement as well, built right in. Because it’s a people-improvement, rather than mind-improvement.
But this type of intelligence increase, not really an explosion, seems pretty Greek-accessible to me, in the same way that “Science must pass Judaism” was accessible to Eliezer as a kid.
You correctly claim this inference to be Greek-accessible, while not an explosion, because it is explicitly mentioned in Plato’s “The Republic”—although the measurement of good bringing up is defined as morality, not intelligence.
What little I know about ancient Greek thought makes me doubt that they would have been equipped to follow up on the full implications of “materialism” in the modern sense. Following Newton, we’re now all completely accustomed to the idea that everything on Earth is governed by the same basic principles as everything else in the universe. We’re used to the idea of matter and energy and forces being quantified into precise algebraic formulas, and even random events can be described with some precision. But the Greeks, for all their knowledge of geometry, did not know algebra or probability.
As I understand it, the prevailing view in those days was that the things above the moon—the stars and planets—followed mathematical patterns (moving in circles, of course, because circles are so obviously terrific). However, on Earth, in our imperfect sublunary sphere, things had tendencies and impulses. Fire more or less wanted to go up, and water down, more or less in the same way that a squirrel wants to run away from a dog.
According to Wikipedia :
Why did he believe sensing could never be false? Surely he must have known about say mirages or camouflage.
I would have thought so too, and maybe he had a deeper meaning lost in translation, but for a brilliant thinker, Aristotle could occasionally espouse some ideas that seem just plain stupid, at least to an ordinary modern person. I’m an ordinary modern person myself, so I’m not going to claim Aristotle was correct about this or that. I only meant to say, I don’t think we can assume that even the most sophisticated* Greeks were “materialists” at all in the modern sense. With regard to intelligence in particular, I don’t see that they “had to believe that human intelligence resulted from the operation of mechanical systems located in the human body.”
*using that term loosely, not to imply the formal school of Sophists.
Nitpick about the image at facingthesingularity.com, just above the title : Great image if you don’t know the story, but can convey quite the wrong ideas to those who played the game it refers to (Mass Effect). (Spoiler ahead.) The space city this image depicts is a the centre of a galactic trap given by malevolent gods. This trap presents itself as a galactic transport network, which helps the development of a galactic civilization. (By the way, nobody understand how it works.) Then it “activates” and help the bad guys wipe out this civilization. Then they wait for the next, rinse and repeat. (Don’t ask what is the need of such a complicated plot for such powerful beings.)
Hopefully, those who notice this will also notice that you totally didn’t mean that.
What’s wrong with a metaphor for world-destroying tech? That’s the default we should be wary about.
Hmm, if the idea of a doomsday machine doesn’t make you flinch from the whole Singularity concept, then OK, it may not be that bad. However, given the 19th century explorer on the foreground, looking at the city, looking forward to the future it represents, I doubt we could suppose one purpose of this image is to warn us about existential risks.
Plus, there’s still the part where we use magic that (i) were given from above, and (ii) is way beyond our strength. It makes us pawns, while we’d probably like to be the Chess Masters. It is something to be wary about, but less obviously so. One could get the idea that Luke supports the idea of creating a god which, a bit like Yahweh, “kindly” dominates us all. Luke probably do support a form of Singleton, but I’d bet on something like “laws of physics 2.0″ instead of something more… personified.
But were they justified? I don’t see how it is obvious from first principles that matter is discrete. It seemed to me more along the lines of the ancient Greeks just making stuff up.
This surprises me. If it’s even close to true, I’m curious about what Michael Vassar was thinking about that convinced him of physicalism.
Atomism had been “predicted” by Democritus, and still, it wasn’t accepted until Dalton. This is a big piece of evidence that atomism wasn’t obvious for people unfamiliar with weighing chemical compounds. The same would apply to natural selection. Saying that it is obvious once you understand it smells of hindsight bias; ideas often look obvious when known to be true. The ancient Greeks didn’t know that learned traits are not inherited, nor they had much evidence for this fact. They didn’t know that the boundaries between species aren’t unsurmountable, nor they had much evidence: almost no knowledge of fossils, not particularly good understanding of breeding and its history. It was perfectly reasonable for an ancient Greek philosopher to assume that a population of goats will forever remain a population of goats. The individual goats may vary, but the variability could be always constrained by some Platonic prototype of goat category whose boundaries are defined and maintained by gods or other eternal forces of nature. Not to mention that the important part of natural selection idea—that it is the driving force behind evolution—doesn’t automatically follow from only heritability, variation and selection. In a stable environment the populations may well occupy local minima in the fitness landscape and remain stable for a long time.
But what I am most bothered about is that this post appears as an attempt to declare natural selection and intelligence explosion as equally obvious, from today’s perspective.
This is anachronistic. Ancients believed that the boundaries between some species were weaker than we do today, for example, that oats were degenerate barley.
I don’t know to what extent they thought species were transmutable. But they didn’t think it impossible.
Just as their false belief that the Earth was eternal may have enabled them to figure out evolution had they thought more clearly, so too with some of their other false beliefs, such as that some species could easily turn into another.
Or maybe not. It’s worth thinking about.
This is precisely worded to not address the issue of whether or not they could have ruled out some amount of Lamarkian inheritance, yes?
Lamarckian inheritance is widely regarded to be a factual phenomenon these days.
This is a very important premise, I’m not sure it’s obvious. It took me a long time to realize the generalized form “if you understand it, you know how to do better”. I seem to remember the greeks thought experiments were abominations, so it seems plausible that the fundamental statement of engineering was also unknown to them.
Okay. Now we know, intelligence explosion will most likely happen. Also we know some other important things in this context.
What is still unheard of, not published widely—and can be now predicted from this high point of view?
That this implies you might live in a simulation? That’s the best example I can think of.
Another one
Except that all of us are only instances. Not just those copied in that machine, this step isn’t necessary.
I other words, you are just an instance of me. And vice versa.
Even if they had a materialist account of mind, why would their experience of mechanism lead them to believe mechanism could be sped up or greatly enhanced as in intelligence explosion scenarios? I’m not familiar with Greek atomist approaches to the mind, but later materialist ideas viewed the nervous system as analogous to a pneumatic system, with “animal spirits” or corpuscles being pumped into muscles from the brain. Given the mechanist analogy with a pneumatic system, why would they think this system could be greatly improved by operating on itself? Can any sense be made of a self-modifying pneumatic system? It’s not until you get computation that self-improvement seems “obvious.” It was Turing who first proposed that a self-modifying computer program could learn and improve and I’m not sure that you could come to that conclusion with anything less than what Turing knew, since you need both the generality of recursion theory and the notion of a mechanical implementation of it (the latter of which Turing supplied).
Could Democritus have predicted extragalactic travel?
An Intelligence Explosion may permitted under physicalism, yet be prohibited by cost and complexity, like intergalactic travel is permitted under physicalism, yet prohibited by cost and engineering complexity.
Thereby saving minds such as we, around the minimum level of intelligence required for general intelligence, from being displaced by minds that are powerful to the maximum degree that the limits-of-physics allow.
Intelligence explosion || big stray meteorite.