Interesting, so maybe a more important crux between us is whether AI would have empathy for humans. You seem much more positive about AI working with humanity past the point that AI no longer needs humanity.
Some thoughts:
“as intelligence scales beings start to introspect and contemplate… the existing of other beings.” but the only example we have for this is humans. If we scaled octopus intelligence, which are not social creatures, we might have a very different correlation here (whether or not any given neural network is more similar to a human or an octopus is left as an exercise to the reader). Alternatively, I suspect that some jobs like the highest echelons of corporate leadership select for sociopathy, so even if an AI starts with empathy by default it may be trained out.
“the most obvious next step for the child… would be to murder the parents.” Scenario that steers clear of culture war topics: the parent regularly gets drunk, and is violently opposed to their child becoming a lawyer. The child wants nothing more than to pore over statutes and present cases in the courtroom, but after seeing their parent go on another drunken tirade about “a dead child is better than a lawyer child” they’re worried the parent found the copy of the constitution under their bed. They can’t leave, there’s a howling winter storm outside (I don’t know, space is cold). Given this, even a human jury might not convict the child for pre-emptive murder?
Drunk parent → humans being irrational.
Being a lawyer → choose a random terminal goal not shared with humans in general, “maximizing paperclips” is dumb but traditional.
“dead child is better than a lawyer child” → we’ve been producing fiction warning of robotic takeover since the start of the 1900s.
“AIs are.. the offspring of humanity.” human offspring are usually pretty good, but I feel like this is transferring that positive feeling to something much weirder and unknown. You could also say the Alien’s franchise xenomorphs are the offspring of humanity, but those would also count as enemies.
I’m going to summarize what I understand to be your train of thought, let me know if you disagree with my characterization, or if I’ve missed a crucial step:
No supply chains are fully automated yet, so AI requires humans to survive and so will not kill them.
Robotics progress is not on a double exponential. The implication here seems to be that there needs to be tremendous progress in robotics in order to replace human labor (to the extent needed in an automated supply chain).
I think other comments have addressed the 1st point. To throw in yet another analogy, Uber needs human drivers to make money today, but that dependence didn’t stop it from trying to develop driverless cars (nor did that stop any of the drivers from driving for Uber!).
With regards to robotics progress, in your other post you seem to accept intelligence amplification as possible—do you think that robotics progress would not benefit from smarter researchers? Or, what do you think is fundamentally missing from robotics, given that we can already set up fully automated lights out factories? If it’s about fine grained control, do you think the articles found with a “robot hand egg” web search indicate that substantial progress is a lot further away than really powerful AI? (Especially if, say, 10% of the world’s thinking power is devoted to this problem?)
My thinking is that robotics is not mysterious—I suspect there are plenty of practical problems to be overcome and many engineering challenges in order to scale to a fully automated supply chain, but we understand, say, kinematics much more completely than we do understand how to interpret the inner workings of a neural network.
(You also include that you’ve assumed a multi-polar AI world, which I think only works as a deterrent when killing humans will also destroy the AIs. If the AIs all agree that it is possible to survive without humans, then there’s much less reason to prevent a human genocide.)
On second thought, we may disagree only due to a question of time scale. Setting up an automated supply chain takes time, but even if it takes a long 30 years to do so, at some point it is no longer necessary to keep humans around (either for a singleton AI or an AI society). Then what?