I made a weak statement “humans do not always act like power-seeking ruthless consequentialists”. If you want to disagree with that, it’s not enough to demonstrate that humans sometimes act like power-seeking ruthless consequentialists; rather, you would need to argue that all humans, always, with no exceptions, act like power-seeking ruthless consequentialists. That’s a very strong statement which seems totally crazy to me. You really believe that?
If so… umm, I’m not really sure where to start. Like, some humans sometimes have sacrificed their lives for others. Some humans sometimes have committed suicide. Some humans sometimes have felt strongly that their dying family member should feel comfortable in their last minutes of life, even when that person’s comfort level could not possibly have any lasting consequences. Some humans sometimes have just been going with the flow, not particularly thinking about the long-term consequences of their actions at all. Some humans sometimes have had no particular idea what long-term consequence of their action they even want to happen—and forget about actually choosing actions by back-chaining from those desired consequences. Etc. Right?
I made a weak statement “humans do not always act like power-seeking ruthless consequentialists”. If you want to disagree with that, it’s not enough to demonstrate that humans sometimes act like power-seeking ruthless consequentialists; rather, you would need to argue that all humans, always, with no exceptions, act like power-seeking ruthless consequentialists. That’s a very strong statement which seems totally crazy to me. You really believe that?
I may have misunderstood what you were claiming in the intro. I thought you were saying something like: “most people don’t act like psychos most of the time, which is surprising”. But it seems here you are saying that actually what you meant was: “most people act like psychos most of the time, but rarely may act in other ways, and it’s surprising that ever happens”.
Hmm, OK, well I do also believe the stronger claim “most people don’t act like psychos most of the time, which is surprising” :)
Like, people watch TV. Power-seeking ruthless consequentialists would not watch TV.
I’m not sure how to operationalize this disagreement. Also, it doesn’t seem like there’s much at stake that makes it worth arguing about.
I do think that human long-term consequentialism makes the world go round (see my other comment). I just don’t think human long-term consequentialism is how the median human is spending most of their waking hours.
Like, people watch TV. Power-seeking ruthless consequentialists would not watch TV.
This is a point where I strongly disagree. I’m not going to claim that the exact amount or type humans watch is optimal, but the general category of “consuming fictional content” seems more likely adaptive than not. I would expect that any AI system with human-comparable intelligence would also find it beneficial to engage in some activity analogous to consuming fictional content.
Also, it doesn’t seem like there’s much at stake that makes it worth arguing about.
That’s fair, but one of the stated goals of the post is “pushing back against optimists”, and it’s using a framing that an optimist of my ilk would not accept. As Richard Sutton has put it, much pessimist discourse takes as an unstated assumption that “evil is optimal”. With that as a foundational assumption, it’s very natural to end up with pessimistic conclusions, but the assumption is doing most of the work, not the arguments built on it.
I made a weak statement “humans do not always act like power-seeking ruthless consequentialists”. If you want to disagree with that, it’s not enough to demonstrate that humans sometimes act like power-seeking ruthless consequentialists; rather, you would need to argue that all humans, always, with no exceptions, act like power-seeking ruthless consequentialists. That’s a very strong statement which seems totally crazy to me. You really believe that?
If so… umm, I’m not really sure where to start. Like, some humans sometimes have sacrificed their lives for others. Some humans sometimes have committed suicide. Some humans sometimes have felt strongly that their dying family member should feel comfortable in their last minutes of life, even when that person’s comfort level could not possibly have any lasting consequences. Some humans sometimes have just been going with the flow, not particularly thinking about the long-term consequences of their actions at all. Some humans sometimes have had no particular idea what long-term consequence of their action they even want to happen—and forget about actually choosing actions by back-chaining from those desired consequences. Etc. Right?
I may have misunderstood what you were claiming in the intro. I thought you were saying something like: “most people don’t act like psychos most of the time, which is surprising”. But it seems here you are saying that actually what you meant was: “most people act like psychos most of the time, but rarely may act in other ways, and it’s surprising that ever happens”.
Hmm, OK, well I do also believe the stronger claim “most people don’t act like psychos most of the time, which is surprising” :)
Like, people watch TV. Power-seeking ruthless consequentialists would not watch TV.
I’m not sure how to operationalize this disagreement. Also, it doesn’t seem like there’s much at stake that makes it worth arguing about.
I do think that human long-term consequentialism makes the world go round (see my other comment). I just don’t think human long-term consequentialism is how the median human is spending most of their waking hours.
This is a point where I strongly disagree. I’m not going to claim that the exact amount or type humans watch is optimal, but the general category of “consuming fictional content” seems more likely adaptive than not. I would expect that any AI system with human-comparable intelligence would also find it beneficial to engage in some activity analogous to consuming fictional content.
That’s fair, but one of the stated goals of the post is “pushing back against optimists”, and it’s using a framing that an optimist of my ilk would not accept. As Richard Sutton has put it, much pessimist discourse takes as an unstated assumption that “evil is optimal”. With that as a foundational assumption, it’s very natural to end up with pessimistic conclusions, but the assumption is doing most of the work, not the arguments built on it.