Like, people watch TV. Power-seeking ruthless consequentialists would not watch TV.
This is a point where I strongly disagree. I’m not going to claim that the exact amount or type humans watch is optimal, but the general category of “consuming fictional content” seems more likely adaptive than not. I would expect that any AI system with human-comparable intelligence would also find it beneficial to engage in some activity analogous to consuming fictional content.
Also, it doesn’t seem like there’s much at stake that makes it worth arguing about.
That’s fair, but one of the stated goals of the post is “pushing back against optimists”, and it’s using a framing that an optimist of my ilk would not accept. As Richard Sutton has put it, much pessimist discourse takes as an unstated assumption that “evil is optimal”. With that as a foundational assumption, it’s very natural to end up with pessimistic conclusions, but the assumption is doing most of the work, not the arguments built on it.
This is a point where I strongly disagree. I’m not going to claim that the exact amount or type humans watch is optimal, but the general category of “consuming fictional content” seems more likely adaptive than not. I would expect that any AI system with human-comparable intelligence would also find it beneficial to engage in some activity analogous to consuming fictional content.
That’s fair, but one of the stated goals of the post is “pushing back against optimists”, and it’s using a framing that an optimist of my ilk would not accept. As Richard Sutton has put it, much pessimist discourse takes as an unstated assumption that “evil is optimal”. With that as a foundational assumption, it’s very natural to end up with pessimistic conclusions, but the assumption is doing most of the work, not the arguments built on it.