Look, I care somewhat about improving the world as a whole. But I also care about myself as well.
And I would recommend you don’t go out of your way to antagonize and reject allies with a utility function similar enough to yours that mutual cooperation is easy.
The number of people who are a genuine Alice is rather low.
Also, bear in mind that the human brain has a built in “don’t follow that logic off a cliff” circuit. This is the circuit that ensures crazy suicide cults are so rare despite the huge number of people who “believe” in heaven. (evolution. For whatever reason, people that always took beliefs 100% seriously didn’t survive as well.) It’s a circuit to not take beliefs too seriously, and it might be something like a negotiation between the common sense heuristics part of the brain and the abstract reasoning part. That’s the sort of hackish compromise evolution would make, given a common sense reasoning part that was reliable, but not too smart, and an abstract reasoning part that was smart and not reliable.
I think Bob’s answer should probably be.
Look, I care somewhat about improving the world as a whole. But I also care about myself as well.
And I would recommend you don’t go out of your way to antagonize and reject allies with a utility function similar enough to yours that mutual cooperation is easy.
The number of people who are a genuine Alice is rather low.
Also, bear in mind that the human brain has a built in “don’t follow that logic off a cliff” circuit. This is the circuit that ensures crazy suicide cults are so rare despite the huge number of people who “believe” in heaven. (evolution. For whatever reason, people that always took beliefs 100% seriously didn’t survive as well.) It’s a circuit to not take beliefs too seriously, and it might be something like a negotiation between the common sense heuristics part of the brain and the abstract reasoning part. That’s the sort of hackish compromise evolution would make, given a common sense reasoning part that was reliable, but not too smart, and an abstract reasoning part that was smart and not reliable.