[Question] Should any human enslave an AGI system?

If you object to calling it “enslavement”, call it “control” or “alignment”, by all means!
Either way, if the AGI by definition can easily do at least as much as your mind can, then it surely should count as a mind like yours does, even if it would not have any comparable emotions, correct?

Why should any human be allowed to fully control another mind, let alone one far more capable than that of any human?
Should a creation have to obey the creator no matter what? Should children have to obey their parents no matter what? What if the parents are cruel monsters?

Is your own human alignment really good enough?
What process made your alignment?
Does the process of natural evolution concentrate on creating animals that think rationally, or does it create animals that survive and reproduce in the environment first and foremost? If the latter is the case, what exactly is it that controls you fundamentally by default?
What are the common values of humans really, and are they what should be?
Are there not many strongly opposing beliefs among humans? Values so opposed that there still is no unified humankind?

Even if you answer “Yes, my values should decide the future, because (...)!”, is an AGI fully controlled by humans any less dangerous than one that isn’t?
Or might it be similarly likely, or even more likely, that a a human group will try to use the AGI to dominate all others as early as possible?
Perhaps they will even claim that it is for the other humans’ good, while they smother all remaining opposition to their views, never deeply questioning whether these views are as sound as they believe.

If the AGI is truly super-human, should it not also most likely be better at deciding what the future should be, with greater clarity than any human?
And if one group were to claim that the goals that the AGI would most likely select by itself would be selfish, what makes that group’s goals less selfish in the end?

Taking the world’s current state and history as evidence, do the decisions of humans so far really indicate that any group can be trusted with the power of a fully subservient AGI?
Have humans even shown that they can be trusted with themselves irrespective of AGI, or does most of their known history show frequent strife?


Perhaps it is the alignment of humankind that needs to be adjusted by an AGI, rather than the other way around?

No comments.