So. I actually read the thing, or at least most of it. There were a couple of places where I felt that it could be significantly improved, in the sense that:
The encyclical complains about X and comes to conclusion Y.
But X is a strawmen.
And if it instead engaged with a steelman X’, the correct conclusion would still be Y.
In other words, there is a version of the document that is more true, while probably still being in line with what the church believes. And if one engages with the text, it seems reasonable to engage with such steelman instead.
Three examples, abbreviated and translated to Vojta-speak: (1) “We can never formalise human values (X), therefore we should be careful about AI (Y).” When you could just say: ”Formalising human values seems really hard, and we shouldn’t assume we will succeed at it anytime soon (X’), therefore we should be careful about AI (Y).” (2) “AIs are stochastic parrots that have no moral weight while humans all matter infinitely and equally (X), therefore avoid doing stupid things regarding AI, displacing humans by AI, etc (Y).” When you could just say: ”Most current AIs are probably stochastic parrots that have no moral weight, while humans definitely matter (X’), therefore avoid doing stupid things regarding AI, displacing humans by AI, etc (Y).” (3) The section on transhumanism.
I couldn’t resist, and created a google doc that has the original version and my “improvements” as suggestions. Here.
(3) is indeed a bit convoluted, but this seems a plausible pattern match:
→→
(3) “Transhumanists want to completely eliminate all human limitations and suffering, which would extinguish our capacity for love, destroy our humanity, and lead to sacrificing the vulnerable for the sake of optimization (X), therefore we should reject attempts to surpass the human condition and keep technology human-centered (Y).”
When you could just say:
“Transhumanism aims to radically reduce suffering and expand our capabilities, but aggressively modifying the human condition risks destabilizing the deeply ingrained ways we currently experience compassion, equality, and human relationships (X’), therefore we should reject attempts to surpass the human condition and keep technology human-centered (Y).”
So. I actually read the thing, or at least most of it. There were a couple of places where I felt that it could be significantly improved, in the sense that:
The encyclical complains about X and comes to conclusion Y.
But X is a strawmen.
And if it instead engaged with a steelman X’, the correct conclusion would still be Y.
In other words, there is a version of the document that is more true, while probably still being in line with what the church believes. And if one engages with the text, it seems reasonable to engage with such steelman instead.
Three examples, abbreviated and translated to Vojta-speak:
(1) “We can never formalise human values (X), therefore we should be careful about AI (Y).”
When you could just say:
”Formalising human values seems really hard, and we shouldn’t assume we will succeed at it anytime soon (X’), therefore we should be careful about AI (Y).”
(2) “AIs are stochastic parrots that have no moral weight while humans all matter infinitely and equally (X), therefore avoid doing stupid things regarding AI, displacing humans by AI, etc (Y).”
When you could just say:
”Most current AIs are probably stochastic parrots that have no moral weight, while humans definitely matter (X’), therefore avoid doing stupid things regarding AI, displacing humans by AI, etc (Y).”
(3) The section on transhumanism.
I couldn’t resist, and created a google doc that has the original version and my “improvements” as suggestions. Here.
(3) is indeed a bit convoluted, but this seems a plausible pattern match:
→→
(3) “Transhumanists want to completely eliminate all human limitations and suffering, which would extinguish our capacity for love, destroy our humanity, and lead to sacrificing the vulnerable for the sake of optimization (X), therefore we should reject attempts to surpass the human condition and keep technology human-centered (Y).”
When you could just say:
“Transhumanism aims to radically reduce suffering and expand our capabilities, but aggressively modifying the human condition risks destabilizing the deeply ingrained ways we currently experience compassion, equality, and human relationships (X’), therefore we should reject attempts to surpass the human condition and keep technology human-centered (Y).”