I think the crux is whether Daniel Kokotajlo is asking for a far more minor effort, where I actually disagree that he is asking for a far more minor effort, and the other part is I think trivial inconveniences matter here a lot, given the stakes of being right or wrong on AI.
I do agree details are unfortunately sparse, but I don’t think we need to ask people to create entire scenarios, because most of the details necessary won’t come in the form of a story.
You can provide detailed, useful criticism without having to predict almost everything about AI and society.
I don’t see any evidence that these requests have led to substantially less criticism than otherwise; and I still think on the margin more concrete stories would be substantially beneficial (especially quite different stories to AI 2027), so it’s good that people are being challenged to write them.
I still think on the margin more concrete stories would be substantially beneficial (especially quite different stories to AI 2027), so it’s good that people are being challenged to write them.
Not clear if challenging people (with obligation-valence) leads to more stories or less general engangement. The consequentialist justification isn’t obviously valid.
Sure, but again, I think the person who has done the most pro-social work on this front is the person who is most allowed to have obligation-valence in requests.
This is analogous to a communal space that for years has been in dire need of redecorating (painting etc), and one person finally takes it upon themselves to do a bunch of work on it. Others who use the space then criticize the work done, people who never did the work for years. The person who did the work is in the best place to say “Rather than spending a lot of time criticizing, I’d ask you to do a bit of work to paint the rest of the room the way you would like it painted.”
Sometimes the work done makes the communal space worse for some particular user. For example, if the redecorator puts in some pretty potted plants that one other user is allergic to, then calling the redecorator’s work “pro-social” implies that it’s perfectly fine for society to exile the allergy-sufferer. The criticism “hey, I’m allergic to those plants you put in; your redecoration effort has exiled me from the communal space” is valid, and the response “well, if you didn’t want to be exiled, you should have done the redecoration yourself” would be quite bad!
Sure, if someone’s critique is “these detailed stories do not help” then that’s quite different, but I think most people agree that how the immediate future goes is very important, most people are speaking in vagueries and generalities, attempting to write detailed stories showing your best guess for how the future could go is heavily undersupplied, and that this work is helpful for having concrete things to debate (even if all the details will be wrong).
You are discussing who gets to create the obligations, the shape of motivations for those who might set them up for others. But my point is about how people would react to (being at risk of) having an obligation directed at them, even if it’s done by the most deserving and socially approved person to do so. It’s not obvious that they respond by becoming more likely to carry it out, or by becoming more avoidant of the situations where they could be singled out as a target of such obligations, however well-justified and prosocial.
I mean we can debate whether it’s useful to bring up social obligation in a situation where one person is doing something pro-social for everyone else. Insofar as someone has done something pro-social and everyone is just criticizing them and not doing any of the work themselves, then I think they are also behaving in a way as to disincentivize the work being done!
There’s a question of where the responsibility lies. Should we behave as if it solely lies in the one person who did the work, or should it lie in everyone who benefits from it? Seems to me like the answer is the latter. If there is responsibility then it is a true thing to talk about, its existence is not optional, and it’s accurate to point out that others are not bearing any share of the work that they benefit from. Your point is that it could be unproductive, but I cannot tell whether you are also saying “and they also didn’t have an obligation until you brought it up”, which seems to me like a subtle but important mistake.
Insofar as someone has done something pro-social and everyone is just criticizing them and not doing any of the work themselves, then I think they are also behaving in a way as to disincentivize the work being done!
The people standing around can be valuable without doing any of the kind of work they are criticising. Not always, but it happens. (Of course this shouldn’t be a reason to get less vigilant about the other kind of people standing around, who are making things worse.) When it does happen and the bystanders are doing something valuable, putting an obligation on them can make them leave (or proactively never show up in the first place), which is damaging. Or it can motivate them to step up, which could be even more valuable than what they were doing originally.
My point is that it’s not obvious which of these is the case, as a matter of consequences of putting social pressure on such people. And therefore it’s wrong to insist that the pressure is good, unless it becomes clearer whether it actually is good. Also, it’s in any case misleading to equivocate between the people standing around making things worse, and the people who would only be standing around if there are no obligations, but who occasionally do something valuable in other ways.
One reason for the equivocation might be applicability of enforcement. If you’ve already decided that refusers of prosocial obligations must be punished (or exiled from a group), then you might be mentally putting them in the same bucket as the directly disruptive bystanders. But these are different categories, their equivalence begs the question on whether the refusers of prosocial obligations should really be punished.
I think the crux is whether Daniel Kokotajlo is asking for a far more minor effort, where I actually disagree that he is asking for a far more minor effort, and the other part is I think trivial inconveniences matter here a lot, given the stakes of being right or wrong on AI.
I do agree details are unfortunately sparse, but I don’t think we need to ask people to create entire scenarios, because most of the details necessary won’t come in the form of a story.
You can provide detailed, useful criticism without having to predict almost everything about AI and society.
I don’t see any evidence that these requests have led to substantially less criticism than otherwise; and I still think on the margin more concrete stories would be substantially beneficial (especially quite different stories to AI 2027), so it’s good that people are being challenged to write them.
Not clear if challenging people (with obligation-valence) leads to more stories or less general engangement. The consequentialist justification isn’t obviously valid.
Sure, but again, I think the person who has done the most pro-social work on this front is the person who is most allowed to have obligation-valence in requests.
This is analogous to a communal space that for years has been in dire need of redecorating (painting etc), and one person finally takes it upon themselves to do a bunch of work on it. Others who use the space then criticize the work done, people who never did the work for years. The person who did the work is in the best place to say “Rather than spending a lot of time criticizing, I’d ask you to do a bit of work to paint the rest of the room the way you would like it painted.”
Sometimes the work done makes the communal space worse for some particular user. For example, if the redecorator puts in some pretty potted plants that one other user is allergic to, then calling the redecorator’s work “pro-social” implies that it’s perfectly fine for society to exile the allergy-sufferer. The criticism “hey, I’m allergic to those plants you put in; your redecoration effort has exiled me from the communal space” is valid, and the response “well, if you didn’t want to be exiled, you should have done the redecoration yourself” would be quite bad!
Sure, if someone’s critique is “these detailed stories do not help” then that’s quite different, but I think most people agree that how the immediate future goes is very important, most people are speaking in vagueries and generalities, attempting to write detailed stories showing your best guess for how the future could go is heavily undersupplied, and that this work is helpful for having concrete things to debate (even if all the details will be wrong).
You are discussing who gets to create the obligations, the shape of motivations for those who might set them up for others. But my point is about how people would react to (being at risk of) having an obligation directed at them, even if it’s done by the most deserving and socially approved person to do so. It’s not obvious that they respond by becoming more likely to carry it out, or by becoming more avoidant of the situations where they could be singled out as a target of such obligations, however well-justified and prosocial.
I mean we can debate whether it’s useful to bring up social obligation in a situation where one person is doing something pro-social for everyone else. Insofar as someone has done something pro-social and everyone is just criticizing them and not doing any of the work themselves, then I think they are also behaving in a way as to disincentivize the work being done!
There’s a question of where the responsibility lies. Should we behave as if it solely lies in the one person who did the work, or should it lie in everyone who benefits from it? Seems to me like the answer is the latter. If there is responsibility then it is a true thing to talk about, its existence is not optional, and it’s accurate to point out that others are not bearing any share of the work that they benefit from. Your point is that it could be unproductive, but I cannot tell whether you are also saying “and they also didn’t have an obligation until you brought it up”, which seems to me like a subtle but important mistake.
The people standing around can be valuable without doing any of the kind of work they are criticising. Not always, but it happens. (Of course this shouldn’t be a reason to get less vigilant about the other kind of people standing around, who are making things worse.) When it does happen and the bystanders are doing something valuable, putting an obligation on them can make them leave (or proactively never show up in the first place), which is damaging. Or it can motivate them to step up, which could be even more valuable than what they were doing originally.
My point is that it’s not obvious which of these is the case, as a matter of consequences of putting social pressure on such people. And therefore it’s wrong to insist that the pressure is good, unless it becomes clearer whether it actually is good. Also, it’s in any case misleading to equivocate between the people standing around making things worse, and the people who would only be standing around if there are no obligations, but who occasionally do something valuable in other ways.
One reason for the equivocation might be applicability of enforcement. If you’ve already decided that refusers of prosocial obligations must be punished (or exiled from a group), then you might be mentally putting them in the same bucket as the directly disruptive bystanders. But these are different categories, their equivalence begs the question on whether the refusers of prosocial obligations should really be punished.