The way I see it, yes, Natural Selection defeats the Orthogonality Thesis. But Meta-Trolling—e.g. simulators choosing the fitness function—defeats Natural Selection; and then the Orthogonality Thesis defeats Meta-Trolling, because you can’t know the value systems of the simulators, so sufficiently intelligent agents have to steelman phenomena (treat appearances as real). #trollosophy
This post is easy to criticize, but I wonder if someone could steelman it.
I guess reading Robin Hanson’s via of AGI that compete with each other would be a steelmanned version.
The way I see it, yes, Natural Selection defeats the Orthogonality Thesis. But Meta-Trolling—e.g. simulators choosing the fitness function—defeats Natural Selection; and then the Orthogonality Thesis defeats Meta-Trolling, because you can’t know the value systems of the simulators, so sufficiently intelligent agents have to steelman phenomena (treat appearances as real). #trollosophy