Thank you! Carl Shulman’s post still seems written from the some-values-are-just-better-than-others perspective that’s troubling me, but your 2009 comment is very relevant. (Despite future-you having issues with it.)
The question “which values, if any, we should try to preserve” you wrote in that comment is, I think, the crux of my issue. I’m having trouble thinking about it, and questions like it, given my “you can’t talk about what’s right, only what wins” assumption. I can (try to) think about whether a paperclip maximizer or The Culture is more likely to overrun the galaxy, but I don’t know how say that one scenario is “better”, except from the perspective of my own values.
What could it possibly mean, to say that something is “better”, except from some perspective, within some context? What could it possibly mean to say that something is “right” (in principle), other than from some larger perspective, within a larger context?
It’s always perspectival—the illusion of objectivity arises because you share your values, fine-grained and deeply hierarchical, due to your place as a twig on a branch on a tree rooted in the mists of a common physics and with a common evolutionary trajectory. Of course you share values with your neighboring twigs, and you can find moral agreement by traversing the tree of evolutionarily instilled values back toward the trunk to find a branch that supports you and your neighboring agents, but from what god-like point of view could they ever be “objective”?
Carl Shulman’s Spreading happiness to the stars seems little harder than just spreading addresses this topic. I also wrote a comment in 2009 that talks about this, although I’m not sure I fully endorse that comment now.
Thank you! Carl Shulman’s post still seems written from the some-values-are-just-better-than-others perspective that’s troubling me, but your 2009 comment is very relevant. (Despite future-you having issues with it.)
The question “which values, if any, we should try to preserve” you wrote in that comment is, I think, the crux of my issue. I’m having trouble thinking about it, and questions like it, given my “you can’t talk about what’s right, only what wins” assumption. I can (try to) think about whether a paperclip maximizer or The Culture is more likely to overrun the galaxy, but I don’t know how say that one scenario is “better”, except from the perspective of my own values.
What could it possibly mean, to say that something is “better”, except from some perspective, within some context? What could it possibly mean to say that something is “right” (in principle), other than from some larger perspective, within a larger context?
It’s always perspectival—the illusion of objectivity arises because you share your values, fine-grained and deeply hierarchical, due to your place as a twig on a branch on a tree rooted in the mists of a common physics and with a common evolutionary trajectory. Of course you share values with your neighboring twigs, and you can find moral agreement by traversing the tree of evolutionarily instilled values back toward the trunk to find a branch that supports you and your neighboring agents, but from what god-like point of view could they ever be “objective”?