Eliezer ---
Could you please expand on your statement that “p(cults|aliens) isn’t less than p(cults|~aliens)” even if greater evidence for aliens were to emerge.
My intuition is that the clearer the evidence for something, the more agreement there will be on the pertinent details. While an individual may be more likely to belong to an alien cult if aliens are present, won’t the number and divergence of cults change depending on the strength of evidence?
This strikes me as parallel to one of your earlier posts, where I think you argued that a multiplicity of weak arguments offers no evidence as the truth of an new argument reaching the same conclusion. Intuitively, I disagree with this. I still think the landscape of arguments currently in use must have some correlation to the truth of a new argument that ‘happens’ to come to the same conclusion. Could you talk more about this someday?
Thanks!
Eliezer ---
I’m confused by your desire for an ‘automatic controlled shutdown’ and your fear that further meta-reasoning will override ethical inhibitions. In previous writings you’ve expressed a desire to have a provably correct solution before proceeding. But aren’t you consciously leaving in a race-condition here?
What’s to prohibit the meta-reasoning from taking place before the shutdown triggers? It would seem that either you can hard-code an ethical inhibition or you can’t. Along those lines, is it fair to presume that the inhibitions are always negative, so that non-action is the safe alternative? Why not just revert to a known state?