As written there, the strong form of the orthogonality thesis states ‘there’s no extra difficulty or complication in creating an intelligent agent to pursue a goal, above and beyond the computational tractability of that goal.’
I don’t know whether that’s intended to mean the same as ‘there are no types of goals that are more ‘natural’ or that are easier to build agents that pursue, or that you’re more likely to get if you have some noisy process for creating agents’.
I feel like I haven’t seen a good argument for the latter statement, and it seems intuitively wrong to me.
https://arbital.com/p/orthogonality/
As written there, the strong form of the orthogonality thesis states ‘there’s no extra difficulty or complication in creating an intelligent agent to pursue a goal, above and beyond the computational tractability of that goal.’
I don’t know whether that’s intended to mean the same as ‘there are no types of goals that are more ‘natural’ or that are easier to build agents that pursue, or that you’re more likely to get if you have some noisy process for creating agents’.
I feel like I haven’t seen a good argument for the latter statement, and it seems intuitively wrong to me.
Thanks, that page is much more informative than anything else I’ve read on the orthogonality thesis.
1 From Arbital:
Also my claim is an existential claim, and I find it valuable because it could be an opportunity to design aligned AI.
2 Arbital claims that orthogonality doesn’t require moral relativism, so it doesn’t seem incompatible with what I am calling naturalism in the post.
3 I am ok with rejecting positions similar to what Arbital calls universalist moral internalism. Statements like “All agents do X” cannot be exact.