GeorgeStephen

• GeorgeStephen 16 Apr 2026 19:46 UTC

  3 points

  0

  in reply to: GeneSmith’s comment on: The Prac­ti­cal Guide to Superbabies

  Thanks for your reply.

  From the linked proposal:

  Personally, I have never found this line of thinking particularly compelling. An embryo, as created in IVF, is a ball of about 100 stem cells. It has no heart, no brain, and no circulatory system to speak of. It doesn’t even have a digestive tract. There is little to distinguish it from any other random clump of cells other than its unique DNA and its potential to become a person if implanted in a receptive uterus.

  Some people might believe those two facts alone means an embryo already qualifies as a human. But I view implanting the embryo the same way I view having sex; it’s a necessary part of the process of procreation, and you don’t get a baby without it.

  I see the appeal of the idea that implantation is required for personhood—as far as I know, the embryo will not activate its developmental program beyond a certain point without it.

  But this makes sense for the embryo—without implantation (again, as far as I know) the embryo cannot gain mass. Trying to develop further without implantation would be utterly futile.

  I see the dependency of an infant as analogous, though admittedly less extreme. An infant without breast milk or a reasonable substitute will starve, and you will never see the later parts of its developmental program. It would be a mistake to see the infant as only potentially human, and I believe it would be a mistake with the embryo as well.

  I believe the example of IVF to be inherently pathological, but I will use it here. An IVF embryo (correct me if I am wrong) has largely the same developmental trajectory whichever woman it is implanted into, provided that the pregnancy is successful. Nearly as much as identical twins normally resemble each other, the infants resulting from implanting an identical pair of embryos into different women would resemble each other. Hence implantation, though essential, does not determine identity in the way that which sperm cell fertilizes an egg determines identity. And I think that continuity of identity is a reasonable basis for assessing human personhood.

• GeorgeStephen 15 Apr 2026 10:44 UTC

  4 points

  −2

  in reply to: GeneSmith’s comment on: The Prac­ti­cal Guide to Superbabies

  Was anyone signing up to be part of 20th century eugenics? Of course not. It wasn’t a voluntary process at all. It was state sponsored sterilization and murder.

  If you don’t see the difference between that and embryo selection, you either haven’t thought deeply about it or you’re catholic.

  If it’s the latter, I understand. One day we’ll have a way to do this without any excess embryos. But not yet.

  Speaking as an actual Catholic, I do see that you are at least trying to avoid sterilization and murder (by not applying them to those that you recognize as persons). However, I also believe that you are failing.

  If an unusually enlightened Nazi eugenicist said that destroying “life unworthy of life” was to be only a temporary endeavor, to be replaced at the first opportunity with nanotechnological bio-engineering to fix defects in those newly born with them, I expect you’d find it insufficient as justification or reassurance.

  I suppose (though the above argument does not rely on it) that you want “Superbabies” to align artificial superintelligence and thus save the world, and that this accounts for your urgency, without which there would not be even an insufficient defense for what is proposed here (at least, not from a Catholic perspective).

  I want to save the world too, but that does not justify treating human embryos (can you prove that they are not persons?) as mere means to that end. It is too easy to let messianic ambitions blind oneself to the requirements of humanity, as history can attest.

  Finally, apologies for the late post—I understand if you do not reply. Also, apologies if this is written in an overly persuasive tone—I have tried at least to be honest. I pose one more question: Do you believe that you are different from historical case studies in overreach, because you have a better understanding of science, or of morality, or because you have a better cause (supposed by me to be aligning superintelligence), or because your actions really are different?

• GeorgeStephen 11 Nov 2025 23:35 UTC

  1 point

  0

  on: Mourn­ing a life with­out AI

  I understand that you don’t want to have kids who will be killed by ASI and not have any kids of their own.

  But you also think that there is a possibility of utopia. Under these assumptions, wouldn’t it make sense to wait for utopia and then have kids?

  I see that you express reservations about having kids in utopia, of two types:

  1. You don’t want them to be deluded denizens of a simulation, or zombies.

  2. You want them to grow up in a world similar to the world you grew up in.

  I agree that 1 would be a showstopper. I would not want to raise kids in a simulation. But I don’t see what the problem would be (given the premise of a good post-AGI future) with raising kids in the real world. If you would like the post-AGI utopia, why wouldn’t your kids, who would be expected to be more adaptable to new circumstances than you?

• GeorgeStephen 18 Jul 2020 13:14 UTC

  1 point

  0

  in reply to: habryka’s comment on: Drexler on AI Risk

  Thanks, your changes helped.

• GeorgeStephen 17 Jul 2020 20:59 UTC

  3 points

  0

  on: Drexler on AI Risk

  I. Introduction
  
  I watched a 40 minute video by Eric Drexler called Reframing Superintelligence. I take it the content somewhat overlaps with his PDF book of the same name.
  
  In it he expresses the opinion that under the Comprehensive Artificial Intelligence Services model, high level actors will not try to take over the world, because there is a serious risk of being stopped, with a bad outcome for the aggressor.
  
  Dr. Drexler made the point that a subgoal of ‘take over the world’ is ‘overthrow the government of China’, and that they would come after you if you made a “credible” attempt.
  
  II. Eric Drexler’s CAIS scheme may fail to solve at least 4 distinct problems.
  
  1. Development of AI may be asymmetric, undermining deterrence.
  
  I think this is somewhat naive. Picture the world as a body of water, with a temperature that represents the degree of advancement in AI capablities. Dr. Drexler’s AI services idea would raise the temperature of the whole world until it exceeds the boiling point (e.g. is superintelligent), but avoid nucleation (an attempt to dominate the world). The premise seems to be that anyone with high-level access to CAIS can make such an attempt, but they will be circumspect about doing so because others are just as advanced, and could defeat and punish them.
  
  What this misses is what I’ll call symmetry breaking. Suppose that the United States makes a non-trivial advance in the hardware used to run neural networks, allowing most neural networks to be run more efficiently. This in turn allows research and development of neural nets much closer to human-brain-scale.
  
  Suppose the U.S. classifies this advance, retaining the fruits of the ensuing research for itself.
  
  The symmetry between the United States and other countries is thus broken, first by hardware, and then by software. The U.S. thus can use CAIS with relative impunity as soon as it attains superintelligence. Even if the CAIS software leaked across borders, the U.S. alone would have the hardware to run it.
  
  Of course the impunity is relative, because using CAIS to dominate the world is still risky for the dominator, but if the U.S. is feeling lucky, China won’t be able to stop it. Hence Dr. Drexler’s argument for the stability of CAIS fails in the presence of symmetry breakers.
  
  2. Even if development is symmetric, there may be a strong first-mover incentive to take over the world.
  
  To reiterate, Dr. Drexler’s hope seems to be that if every major power has access to CAIS, mutual deterrence will prevent hostile use. If the symmetric state can be reached, this might be the case. But it might not. What if CAIS reported to whoever used it that there was a certainty of success in world domination for whoever acted first? The only way out might be the domination of the world by a hegemon (human or artificial) tasked with preventing the domination of the world by anyone else!
  
  An alternative scenario is semi-stablity, where a world-domination attempt by a first-mover may be thwarted by a second-mover, but the second-mover must at least temporarily dominate the world in order to do so (e.g. by scouring the world of the first-mover’s nanobots with the second-mover’s nanobots.)
  
  3. Even if there isn’t such an incentive, aggression may be hard to define, thus deterrence may be difficult to implement.
  
  What if what must be deterred are not attempts to dominate the world only, but also lesser disruptive goals? Disruptive goals can be classed in order of degree of disruption (from destruction to minimal disturbance) and scope of disruption (from universal to personal). It is not clear what lines to draw, and where.
  
  4. Even if deterrence can be implemented, the system may usher in a techno-oligarcy.
  
  There is another argument against CAIS. It is not simply that it can be misused (Dr. Drexler has already acknowledged that, and the alternative AGI model can be misused too.) Rather, CAIS subjects the world utterly to human will, and not just any human will, but the will of a selected set, with higher permissions being held by fewer people.
  
  If full CAIS are limited to a few, then it seems to me that people will be ruled by immortal potentates whom they will have no chance overthrow, unless the potentates give them that chance voluntarily. I don’t know whether A.I.-U.S.A. would be livable, but I wouldn’t want to live in A.I.-People’s Republic of China.
  
  III. Conclusion
  
  1. The alternative to techno-oligarchy may be a singleton, which is what was supposed to be avoided in the first place.
  
  Because CAIS are comprehensive, a person holding full permissions will be able to attempt to do anything, including trying to turn the world into paperclips. Unless, that is, they are under constant surveillance by CAIS not their own, or their CAIS won’t obey, which suggests an agent behind the disobedience.
  
  Either most people won’t have full CAIS permissions, or the entire reachable universe will have to be wired with protective systems, lest al Qaeda send a von Neumann probe to Jupiter disguised as an innocent science project. At some point, this would imply the deployment of systems that are capable of making decisions on their own, or in other words AGI.
  
  2. Not developing strong AI at all may be the only good option.