First let me say that with respect to the world of alignment research, or the AI world in general, I am nothing. I don’t have a job in those areas, I am physically remote from where the action is. My contribution consists of posts and comments here.
This assertion deserves a lot of attention IMO, worthy of a post on its own, something along the lines of Why Rationalists Aren’t Winners (meant not to mock, but to put it in terms of what rationalism is supposed to do). The gist is that morality is useful for mass coordination to solve collective action problems. When you participate in deliberation about what is good for the group, help arrive at shared answers to the question “how ought we to behave?” and then commit to following those answers, that is power and effectiveness. Overcoming biases that help with coordination so you can, what, win at poker, is not winning. Nassim Nicholas Taleb covers this quite well.
Thanks for working through your thinking. And thanks for bringing my/our attention to “The Peace War,” I was not aware of it until now. My only caveat is that one must discount the verisimilitude of science fiction because it demands conflict to be interesting to read. It creates oppressive conditions for the protagonists to overcome, when rational antagonists would eschew those oppressive conditions so that there’s no need to protect themselves from plucky protagonists.
The same kind of reasoning applies to the bringing about of AI overlords if you don’t have to. @Mars_Will_Be_Ours covers this well in their comment.
The egoist/nihilist categories aren’t mutually exclusive. “For the environment” is not nihilistic nor non-egoist when the environment is the provider of everything you need to live a good, free, peaceful albeit finite life.
This assertion deserves a lot of attention IMO, worthy of a post on its own, something along the lines of Why Rationalists Aren’t Winners (meant not to mock, but to put it in terms of what rationalism is supposed to do). The gist is that morality is useful for mass coordination to solve collective action problems. When you participate in deliberation about what is good for the group, help arrive at shared answers to the question “how ought we to behave?” and then commit to following those answers, that is power and effectiveness. Overcoming biases that help with coordination so you can, what, win at poker, is not winning. Nassim Nicholas Taleb covers this quite well.
Thanks for working through your thinking. And thanks for bringing my/our attention to “The Peace War,” I was not aware of it until now. My only caveat is that one must discount the verisimilitude of science fiction because it demands conflict to be interesting to read. It creates oppressive conditions for the protagonists to overcome, when rational antagonists would eschew those oppressive conditions so that there’s no need to protect themselves from plucky protagonists.
The same kind of reasoning applies to the bringing about of AI overlords if you don’t have to. @Mars_Will_Be_Ours covers this well in their comment.
The egoist/nihilist categories aren’t mutually exclusive. “For the environment” is not nihilistic nor non-egoist when the environment is the provider of everything you need to live a good, free, peaceful albeit finite life.