She needs an alien theory of mind to understand what the alien wants
Absolutely, I would think that the first order of business would be to learn that alien theory of mind (and be very conservative until that’s done).
Maybe you’re saying that this alien theory of mind is unlearnable, even for a very intelligent Alice? That seems pretty surprising, and I don’t feel the force of that intuition (despite the Occam’s razor impossibility result).
Absolutely, I would think that the first order of business would be to learn that alien theory of mind (and be very conservative until that’s done).
Maybe you’re saying that this alien theory of mind is unlearnable, even for a very intelligent Alice? That seems pretty surprising, and I don’t feel the force of that intuition (despite the Occam’s razor impossibility result).
Developing this idea a bit: https://www.lesswrong.com/posts/kMJxwCZ4mc9w4ezbs/how-an-alien-theory-of-mind-might-be-unlearnable