Quite probably true. Which of course suggests the question: How (or how much) should “typical humans” be consulted about our plans for their future?
Yeah, I know that is an unfair way to ask the question. And I admit that Eliezer, at least, is actually doing something to raise the waterline. But it is a serious ethical question for utilitarians and a serious political question for egoists. And the closest thing I have seen to an answer for that question around here is something like “Well, we will scan their brains, or observe their behavior, or something. And then try to get something coherent out of that data. But God forbid we should ask them about it. That would just confuse things.”
Not to mention that typical humans behave like hyperbolic discounters, and many can not even be made to understand the concept of a “discount rate”.
Quite probably true. Which of course suggests the question: How (or how much) should “typical humans” be consulted about our plans for their future?
Yeah, I know that is an unfair way to ask the question. And I admit that Eliezer, at least, is actually doing something to raise the waterline. But it is a serious ethical question for utilitarians and a serious political question for egoists. And the closest thing I have seen to an answer for that question around here is something like “Well, we will scan their brains, or observe their behavior, or something. And then try to get something coherent out of that data. But God forbid we should ask them about it. That would just confuse things.”