A critique in a high-ambiguity context is almost always (in all but the most technical domains where individual claims can be cheaply verified) a request to engage in a prolonged exchange, whose quality will be highly dependent on the quality and intelligence and reasonableness of the interlocutor.
Critique is often disproportionally easy, and additionally beyond that, because it tends to have a low correct-positive rate, difficult to evaluate. Generation is often disproportionally hard and, as long as its in a domain in which overall performance can be measured, relatively easy to verify[1].
This means that if you want to create an environment in which people have lots of evidence that the other people they are interfacing are high-quality contributors and worth engaging with, it helps a lot if you have a general expectation that the people who make those bids are generally expected to have done generative work themselves. Or more broadly, seeing that someone has done some good generative work is a lot of evidence they will be worth engaging with. Seeing that someone has done a medium amount of maybe-good-maybe-bad critique is not that much evidence they are worth engaging with.
People leaving bad comments really is actually a major reason why most of the internet is bad. It appears your theory doesn’t really admit for a comment to be bad, which I think makes it inapplicable to most of the internet.
This is true independently of whether people are biased against people who critique them.
(I don’t particularly expect I will want to engage much further on this, though I generally enjoy moderation discussion, as I don’t really expect to hit your real cruxes here, and also broadly feel like I’ve spent my fair share of moderation discussion time with you)
This is not universally true in all domains, and figuring out where it’s true is I think also a non-terrible guide to when you can build high-trust groups with critique-dominated work. Computer security is one such example domain. Stock trading is maybe another, though my sense is in order to actually make good trades you need to both understand what everyone else is right about, and then usually need to also understand what they are wrong about, though I feel like naively just knowing that people are wrong about one thing should be enough to make money off of that, but the few times I’ve tried it has really been quite difficult to do that.
I’m surprised by this, as I was expecting moderation discussions to generally be a pain point/something that is avoided because it reliably brings up lots of drama that blows up.
On the rest of the comment, I do agree that Zack doesn’t really realize that criticism can both be wrong in an evidential/Bayesian sense, and very importantly costly to evaulate as being wrong, because criticism is disproportinately cheap to generation, and I’d go further than habryka and say that the expectation in a domain you are trying to get in and it’s even somewhat tractable, verifying something as correct or not is way easier than generating the thing yourself, such that you need a lot of evidence to prevent both interlocutors from wasting their time (yes, this is related to P vs NP) because they are not perfect Bayesians, and good generative comments (in an evidential sense) are far more costly signals/far more evidence than being a good critic.
That said, I do think there is an issue when people are expected to write up whole scenarios/posts just so they can criticize one important aspect of something, and that’s because it disincentivizes way too much criticism, as now you are asking them to predict far more stuff than is necessary to correctly criticize most arguments.
For example, you don’t need to create a different AI 2027 scenario in order to criticize the fact that there’s too many degrees of freedom in choosing a fit to a curve, which Titotal did:
Daniel Kokotajlo did this here, and while I think his comment was understandable, I don’t exactly like this trend of asking people to write their own scenarios/posts just to criticize one aspect of the story, as there are cheaper ways to resolve the issue:
In relation to moderation discussions, while I do think rate limiting is useful to have for moderators, especially to slow down heated discussions or to let other people do the talking, and thus it shouldn’t be removed as a moderator power, I don’t like the use of rate limiting as a way to force replies to go to post level, as there are many examples of good comments that don’t work as a post, and there are real advantages for the audience around comments, especially around threading/response times that don’t really work with posts as replies.
I’m surprised by this, as I was expecting moderation discussions to generally be a pain point/something that is avoided because it reliably brings up lots of drama that blows up.
If I didn’t learn to enjoy moderation discussions and community governance discussions, I would have long lost motivation to do this job. These things are deep passions of mine. They are not fun to have with everyone, and especially if you are talking about specific norm enforcement can be taxing and adversarial, but at a high level, I really care about this stuff and spend a lot of my time thinking and talking about it.
Daniel Kokotajlo did this here, and while I think his comment was understandable, I don’t exactly like this trend of asking people to write their own scenarios/posts just to criticize one aspect of the story, as there are cheaper ways to resolve the issue:
I disagree; I think there is a dearth of substantive detail from people in discussing the future of AI, that doing it well is pro-social, and I think the person who has done the most work in pushing ahead our discourse on that is the person with the most right to challenge others to make a similar (while far more minor) effort.
I think the crux is whether Daniel Kokotajlo is asking for a far more minor effort, where I actually disagree that he is asking for a far more minor effort, and the other part is I think trivial inconveniences matter here a lot, given the stakes of being right or wrong on AI.
I do agree details are unfortunately sparse, but I don’t think we need to ask people to create entire scenarios, because most of the details necessary won’t come in the form of a story.
You can provide detailed, useful criticism without having to predict almost everything about AI and society.
I don’t see any evidence that these requests have led to substantially less criticism than otherwise; and I still think on the margin more concrete stories would be substantially beneficial (especially quite different stories to AI 2027), so it’s good that people are being challenged to write them.
I still think on the margin more concrete stories would be substantially beneficial (especially quite different stories to AI 2027), so it’s good that people are being challenged to write them.
Not clear if challenging people (with obligation-valence) leads to more stories or less general engangement. The consequentialist justification isn’t obviously valid.
Sure, but again, I think the person who has done the most pro-social work on this front is the person who is most allowed to have obligation-valence in requests.
This is analogous to a communal space that for years has been in dire need of redecorating (painting etc), and one person finally takes it upon themselves to do a bunch of work on it. Others who use the space then criticize the work done, people who never did the work for years. The person who did the work is in the best place to say “Rather than spending a lot of time criticizing, I’d ask you to do a bit of work to paint the rest of the room the way you would like it painted.”
Sometimes the work done makes the communal space worse for some particular user. For example, if the redecorator puts in some pretty potted plants that one other user is allergic to, then calling the redecorator’s work “pro-social” implies that it’s perfectly fine for society to exile the allergy-sufferer. The criticism “hey, I’m allergic to those plants you put in; your redecoration effort has exiled me from the communal space” is valid, and the response “well, if you didn’t want to be exiled, you should have done the redecoration yourself” would be quite bad!
Sure, if someone’s critique is “these detailed stories do not help” then that’s quite different, but I think most people agree that how the immediate future goes is very important, most people are speaking in vagueries and generalities, attempting to write detailed stories showing your best guess for how the future could go is heavily undersupplied, and that this work is helpful for having concrete things to debate (even if all the details will be wrong).
You are discussing who gets to create the obligations, the shape of motivations for those who might set them up for others. But my point is about how people would react to (being at risk of) having an obligation directed at them, even if it’s done by the most deserving and socially approved person to do so. It’s not obvious that they respond by becoming more likely to carry it out, or by becoming more avoidant of the situations where they could be singled out as a target of such obligations, however well-justified and prosocial.
I mean we can debate whether it’s useful to bring up social obligation in a situation where one person is doing something pro-social for everyone else. Insofar as someone has done something pro-social and everyone is just criticizing them and not doing any of the work themselves, then I think they are also behaving in a way as to disincentivize the work being done!
There’s a question of where the responsibility lies. Should we behave as if it solely lies in the one person who did the work, or should it lie in everyone who benefits from it? Seems to me like the answer is the latter. If there is responsibility then it is a true thing to talk about, its existence is not optional, and it’s accurate to point out that others are not bearing any share of the work that they benefit from. Your point is that it could be unproductive, but I cannot tell whether you are also saying “and they also didn’t have an obligation until you brought it up”, which seems to me like a subtle but important mistake.
Insofar as someone has done something pro-social and everyone is just criticizing them and not doing any of the work themselves, then I think they are also behaving in a way as to disincentivize the work being done!
The people standing around can be valuable without doing any of the kind of work they are criticising. Not always, but it happens. (Of course this shouldn’t be a reason to get less vigilant about the other kind of people standing around, who are making things worse.) When it does happen and the bystanders are doing something valuable, putting an obligation on them can make them leave (or proactively never show up in the first place), which is damaging. Or it can motivate them to step up, which could be even more valuable than what they were doing originally.
My point is that it’s not obvious which of these is the case, as a matter of consequences of putting social pressure on such people. And therefore it’s wrong to insist that the pressure is good, unless it becomes clearer whether it actually is good. Also, it’s in any case misleading to equivocate between the people standing around making things worse, and the people who would only be standing around if there are no obligations, but who occasionally do something valuable in other ways.
One reason for the equivocation might be applicability of enforcement. If you’ve already decided that refusers of prosocial obligations must be punished (or exiled from a group), then you might be mentally putting them in the same bucket as the directly disruptive bystanders. But these are different categories, their equivalence begs the question on whether the refusers of prosocial obligations should really be punished.
A critique in a high-ambiguity context is almost always (in all but the most technical domains where individual claims can be cheaply verified) a request to engage in a prolonged exchange
I don’t think so, I think critics usually see themselves as pointing out errors in the original and would be quite happy with an “oops, good pickup”. Extended exchanges happen because authors usually don’t agree. Often at least one of the parties is confused/motivated/doing a poor job for some reason, which means long exchanges are often frustrating and people are wary about getting into them. But the critique isn’t a request for an exchange.
I’m also a bit confused by this discussion. Is your position actually: you can only judge a critic by their papers or LessWrong posts? This seems odd, can’t you judge them by their other critiques? A critique seems much easier to evaluate than a paper.
This depends a lot on the style of the critique. One of the most common forms of critique is “ask a leading question”. Clearly that is an invitation for a prolonged exchange. Think of socratic exchanges.
There definitely exist critiques that can be self-contained, and stand on their own as a counterveiling perspective. I tried to point to that in some of the examples in my comments. It’s particularly common in technical domains.
In most other domains, there is enough adversarialness, combined with enough ambiguity that usually a critique requires multiple back and forths to come into focus (and also, it’s usually socially advantageous for critics to intentionally not make their critique maximally explicit or easy to judge in an attempt of making the reader fill in the gaps with whatever they think is weakest about the object of critique).
Even though all of that is true, it just seems like such a strange way to frame the problem. Yes, the commenter’s catalogue of top-level posts does provide some information about how productive a back and fourth would be. But if Said had two curated posts but commented in exactly the same style, the number of complaints about him would be no different; the only thing that would change is that people wouldn’t use the ‘he doesn’t write top level posts’ as a justification. (Do you actually doubt this?) Conversely if everyone thought his comments were insightful and friendly, no one would care about his top-level posts. The whole point about top level posts is at least 80% a rationalization/red herring.
I’m not sure that’s even true of leading questions. You can ask a leading question for the benefit of other readers who will see the question, understand the objection the question is implicitly raising, and then reflect on whether it’s reasonable.
Critique is often disproportionally easy, and additionally beyond that, because it tends to have a low correct-positive rate, difficult to evaluate. Generation is often disproportionally hard and, as long as its in a domain in which overall performance can be measured, relatively easy to verify[1].
Of course. It’s just difficult to make a correct positive claim, because there are too many theories for more than a few of them to be right, and because there are multiple objections to most of them. But if you recognise that,you can at least adopt the right meta level approach , which is epistemic modesty—the opposite of the approach officially recommended here [*] The fact that philosophy is difficult doesn’t have to keep coming as a nasty surprise.
The most generic and easy criticism is “it ain’t necessarily so...”..but it’s true! On a practical level, if you acknowledge the standard problems with a claim in advance—write modestly—don’t claim that it is necessarily so—there is no need for anyone to annoy you by pointing them out. There are rational and irrational ways of avoiding criticism. Getting better at rationality is the rational way.
Whether or not it’s relevant the person making a criticism has the right credentials depends on the kind of criticism… ranging from “you’re wrong, trust me I’m an expert” to “here’s an explicit disproof, who cares where it came from”. (Even @private_messaging)
[*]
The versions of modest epistemology I hear about usually involve deference to the majority view, to the academic mainstream, or to publicly recognized elite opinion
The version I am talking about is “it’s, hard to be right , even for experts”. It can still make sense to defer to experts, because experts can be the least bad option, in a Churchillian sense. And they can still be bad in an absolute sense, because of the general difficulty of being right.
A critique in a high-ambiguity context is almost always (in all but the most technical domains where individual claims can be cheaply verified) a request to engage in a prolonged exchange, whose quality will be highly dependent on the quality and intelligence and reasonableness of the interlocutor.
Critique is often disproportionally easy, and additionally beyond that, because it tends to have a low correct-positive rate, difficult to evaluate. Generation is often disproportionally hard and, as long as its in a domain in which overall performance can be measured, relatively easy to verify[1].
This means that if you want to create an environment in which people have lots of evidence that the other people they are interfacing are high-quality contributors and worth engaging with, it helps a lot if you have a general expectation that the people who make those bids are generally expected to have done generative work themselves. Or more broadly, seeing that someone has done some good generative work is a lot of evidence they will be worth engaging with. Seeing that someone has done a medium amount of maybe-good-maybe-bad critique is not that much evidence they are worth engaging with.
People leaving bad comments really is actually a major reason why most of the internet is bad. It appears your theory doesn’t really admit for a comment to be bad, which I think makes it inapplicable to most of the internet.
This is true independently of whether people are biased against people who critique them.
(I don’t particularly expect I will want to engage much further on this, though I generally enjoy moderation discussion, as I don’t really expect to hit your real cruxes here, and also broadly feel like I’ve spent my fair share of moderation discussion time with you)
This is not universally true in all domains, and figuring out where it’s true is I think also a non-terrible guide to when you can build high-trust groups with critique-dominated work. Computer security is one such example domain. Stock trading is maybe another, though my sense is in order to actually make good trades you need to both understand what everyone else is right about, and then usually need to also understand what they are wrong about, though I feel like naively just knowing that people are wrong about one thing should be enough to make money off of that, but the few times I’ve tried it has really been quite difficult to do that.
I’m surprised by this, as I was expecting moderation discussions to generally be a pain point/something that is avoided because it reliably brings up lots of drama that blows up.
On the rest of the comment, I do agree that Zack doesn’t really realize that criticism can both be wrong in an evidential/Bayesian sense, and very importantly costly to evaulate as being wrong, because criticism is disproportinately cheap to generation, and I’d go further than habryka and say that the expectation in a domain you are trying to get in and it’s even somewhat tractable, verifying something as correct or not is way easier than generating the thing yourself, such that you need a lot of evidence to prevent both interlocutors from wasting their time (yes, this is related to P vs NP) because they are not perfect Bayesians, and good generative comments (in an evidential sense) are far more costly signals/far more evidence than being a good critic.
That said, I do think there is an issue when people are expected to write up whole scenarios/posts just so they can criticize one important aspect of something, and that’s because it disincentivizes way too much criticism, as now you are asking them to predict far more stuff than is necessary to correctly criticize most arguments.
For example, you don’t need to create a different AI 2027 scenario in order to criticize the fact that there’s too many degrees of freedom in choosing a fit to a curve, which Titotal did:
https://www.lesswrong.com/posts/PAYfmG2aRbdb74mEp/a-deep-critique-of-ai-2027-s-bad-timeline-models#Six_stories_that_fit_the_data
Daniel Kokotajlo did this here, and while I think his comment was understandable, I don’t exactly like this trend of asking people to write their own scenarios/posts just to criticize one aspect of the story, as there are cheaper ways to resolve the issue:
https://www.lesswrong.com/posts/zuuQwueBpv9ZCpNuX/vitalik-s-response-to-ai-2027#zLtuWhcyZt8QDRm3P
In relation to moderation discussions, while I do think rate limiting is useful to have for moderators, especially to slow down heated discussions or to let other people do the talking, and thus it shouldn’t be removed as a moderator power, I don’t like the use of rate limiting as a way to force replies to go to post level, as there are many examples of good comments that don’t work as a post, and there are real advantages for the audience around comments, especially around threading/response times that don’t really work with posts as replies.
If I didn’t learn to enjoy moderation discussions and community governance discussions, I would have long lost motivation to do this job. These things are deep passions of mine. They are not fun to have with everyone, and especially if you are talking about specific norm enforcement can be taxing and adversarial, but at a high level, I really care about this stuff and spend a lot of my time thinking and talking about it.
Doesn’t G/D gap suggest we should have more criticism than generation?
Yes, but not at the ratios we see in practice.
I disagree; I think there is a dearth of substantive detail from people in discussing the future of AI, that doing it well is pro-social, and I think the person who has done the most work in pushing ahead our discourse on that is the person with the most right to challenge others to make a similar (while far more minor) effort.
I think the crux is whether Daniel Kokotajlo is asking for a far more minor effort, where I actually disagree that he is asking for a far more minor effort, and the other part is I think trivial inconveniences matter here a lot, given the stakes of being right or wrong on AI.
I do agree details are unfortunately sparse, but I don’t think we need to ask people to create entire scenarios, because most of the details necessary won’t come in the form of a story.
You can provide detailed, useful criticism without having to predict almost everything about AI and society.
I don’t see any evidence that these requests have led to substantially less criticism than otherwise; and I still think on the margin more concrete stories would be substantially beneficial (especially quite different stories to AI 2027), so it’s good that people are being challenged to write them.
Not clear if challenging people (with obligation-valence) leads to more stories or less general engangement. The consequentialist justification isn’t obviously valid.
Sure, but again, I think the person who has done the most pro-social work on this front is the person who is most allowed to have obligation-valence in requests.
This is analogous to a communal space that for years has been in dire need of redecorating (painting etc), and one person finally takes it upon themselves to do a bunch of work on it. Others who use the space then criticize the work done, people who never did the work for years. The person who did the work is in the best place to say “Rather than spending a lot of time criticizing, I’d ask you to do a bit of work to paint the rest of the room the way you would like it painted.”
Sometimes the work done makes the communal space worse for some particular user. For example, if the redecorator puts in some pretty potted plants that one other user is allergic to, then calling the redecorator’s work “pro-social” implies that it’s perfectly fine for society to exile the allergy-sufferer. The criticism “hey, I’m allergic to those plants you put in; your redecoration effort has exiled me from the communal space” is valid, and the response “well, if you didn’t want to be exiled, you should have done the redecoration yourself” would be quite bad!
Sure, if someone’s critique is “these detailed stories do not help” then that’s quite different, but I think most people agree that how the immediate future goes is very important, most people are speaking in vagueries and generalities, attempting to write detailed stories showing your best guess for how the future could go is heavily undersupplied, and that this work is helpful for having concrete things to debate (even if all the details will be wrong).
You are discussing who gets to create the obligations, the shape of motivations for those who might set them up for others. But my point is about how people would react to (being at risk of) having an obligation directed at them, even if it’s done by the most deserving and socially approved person to do so. It’s not obvious that they respond by becoming more likely to carry it out, or by becoming more avoidant of the situations where they could be singled out as a target of such obligations, however well-justified and prosocial.
I mean we can debate whether it’s useful to bring up social obligation in a situation where one person is doing something pro-social for everyone else. Insofar as someone has done something pro-social and everyone is just criticizing them and not doing any of the work themselves, then I think they are also behaving in a way as to disincentivize the work being done!
There’s a question of where the responsibility lies. Should we behave as if it solely lies in the one person who did the work, or should it lie in everyone who benefits from it? Seems to me like the answer is the latter. If there is responsibility then it is a true thing to talk about, its existence is not optional, and it’s accurate to point out that others are not bearing any share of the work that they benefit from. Your point is that it could be unproductive, but I cannot tell whether you are also saying “and they also didn’t have an obligation until you brought it up”, which seems to me like a subtle but important mistake.
The people standing around can be valuable without doing any of the kind of work they are criticising. Not always, but it happens. (Of course this shouldn’t be a reason to get less vigilant about the other kind of people standing around, who are making things worse.) When it does happen and the bystanders are doing something valuable, putting an obligation on them can make them leave (or proactively never show up in the first place), which is damaging. Or it can motivate them to step up, which could be even more valuable than what they were doing originally.
My point is that it’s not obvious which of these is the case, as a matter of consequences of putting social pressure on such people. And therefore it’s wrong to insist that the pressure is good, unless it becomes clearer whether it actually is good. Also, it’s in any case misleading to equivocate between the people standing around making things worse, and the people who would only be standing around if there are no obligations, but who occasionally do something valuable in other ways.
One reason for the equivocation might be applicability of enforcement. If you’ve already decided that refusers of prosocial obligations must be punished (or exiled from a group), then you might be mentally putting them in the same bucket as the directly disruptive bystanders. But these are different categories, their equivalence begs the question on whether the refusers of prosocial obligations should really be punished.
I don’t think so, I think critics usually see themselves as pointing out errors in the original and would be quite happy with an “oops, good pickup”. Extended exchanges happen because authors usually don’t agree. Often at least one of the parties is confused/motivated/doing a poor job for some reason, which means long exchanges are often frustrating and people are wary about getting into them. But the critique isn’t a request for an exchange.
I’m also a bit confused by this discussion. Is your position actually: you can only judge a critic by their papers or LessWrong posts? This seems odd, can’t you judge them by their other critiques? A critique seems much easier to evaluate than a paper.
This depends a lot on the style of the critique. One of the most common forms of critique is “ask a leading question”. Clearly that is an invitation for a prolonged exchange. Think of socratic exchanges.
There definitely exist critiques that can be self-contained, and stand on their own as a counterveiling perspective. I tried to point to that in some of the examples in my comments. It’s particularly common in technical domains.
In most other domains, there is enough adversarialness, combined with enough ambiguity that usually a critique requires multiple back and forths to come into focus (and also, it’s usually socially advantageous for critics to intentionally not make their critique maximally explicit or easy to judge in an attempt of making the reader fill in the gaps with whatever they think is weakest about the object of critique).
Even though all of that is true, it just seems like such a strange way to frame the problem. Yes, the commenter’s catalogue of top-level posts does provide some information about how productive a back and fourth would be. But if Said had two curated posts but commented in exactly the same style, the number of complaints about him would be no different; the only thing that would change is that people wouldn’t use the ‘he doesn’t write top level posts’ as a justification. (Do you actually doubt this?) Conversely if everyone thought his comments were insightful and friendly, no one would care about his top-level posts. The whole point about top level posts is at least 80% a rationalization/red herring.
Well, I don’t have two curated posts, but I do have one…
EDIT: No, I actually do have two! I forgot about that one!
Didn’t even know that! (Which kind of makes my point.)
See edit of the grandparent. Turns out I do have two curated posts, who knew?!
I’m not sure that’s even true of leading questions. You can ask a leading question for the benefit of other readers who will see the question, understand the objection the question is implicitly raising, and then reflect on whether it’s reasonable.
Of course. It’s just difficult to make a correct positive claim, because there are too many theories for more than a few of them to be right, and because there are multiple objections to most of them. But if you recognise that,you can at least adopt the right meta level approach , which is epistemic modesty—the opposite of the approach officially recommended here [*] The fact that philosophy is difficult doesn’t have to keep coming as a nasty surprise.
The most generic and easy criticism is “it ain’t necessarily so...”..but it’s true! On a practical level, if you acknowledge the standard problems with a claim in advance—write modestly—don’t claim that it is necessarily so—there is no need for anyone to annoy you by pointing them out. There are rational and irrational ways of avoiding criticism. Getting better at rationality is the rational way.
Whether or not it’s relevant the person making a criticism has the right credentials depends on the kind of criticism… ranging from “you’re wrong, trust me I’m an expert” to “here’s an explicit disproof, who cares where it came from”. (Even @private_messaging)
[*]
The version I am talking about is “it’s, hard to be right , even for experts”. It can still make sense to defer to experts, because experts can be the least bad option, in a Churchillian sense. And they can still be bad in an absolute sense, because of the general difficulty of being right.