Here’s my best guess for overall “moderation frame”, new this week, to handle the volume of users. (Note: I’ve discussed this with other LW team members, and I think there’s rough buy-in for trying this out, but it’s still pretty early in our discussion process, other team members might end up arguing for different solutions)
I think to scale the LessWrong userbase, it’d be really helpful to shift the default assumptions of LessWrong to “users by default have a rate limit of 1-comment-per day” and “1 post per week.”
If people get somewhat upvoted, they fairly quickly increase that rate limit to either “1 comment per hour” or “~3 comments per day” (I’m not sure which is better), so they can start participating in conversations. If they get somewhat more upvoted the rate limit disappears completely.
But to preserve this, you need to be producing content that is actively upvoted. If they get downvoted (or just produce a long string of barely-upvoted comments), they go back to the 1-per-day rate limit. If they’re getting significantly downvoted, the rate limit ratchets up (to 1 per 3 days, then once per week and eventually once-per month which is essentially saying “you’re sort of banned, but you can periodically try again, and if your new comments get upvoted you’ll get your privileges restored”)
Getting the tuning here exactly right to avoid being really annoying to existing users who weren’t doing anything wrong is somewhat tricky, but a) I think there are at least some situations where I think the rules would be pretty straightforward, b) I think it’s an achievable goal to the tune the system to basically work as intended.
When users have a rate limit, they get UI elements giving them some recommendations for what to do differently. (I think it’s likely we can also build some quick-feedback buttons that moderators and some trusted users can use, so people have a bit more idea of what to do differently).
Once users have produced a multiple highly upvoted posts/comments, they get more leniency (i.e. they can have a larger string of downvotes or longer non-upvoted back-and-forths before getting rate limited).
If we were starting a forum from scratch with this sort of design at it’s foundation, I think this could feel more like a positive thing (kinda like a videogame incentivizing good discussion and idea-generation, with built in self-moderation).
Since we’re not starting from scratch, I do expect this to feel pretty jarring and unfair to people. I think this is sad, but, I think some kind of change is necessary and we just have to pay the costs somewhere.
My model of @Vladimir_Nesov pops up to warn about negative selection here (I’m not sure whether he thinks rate-limiting is as risky as banning, for negative-selection reasons. It certainly still will cause some people to bounce off. I definitely see risks with negative selection punishing variance, but even the current number of mediocre comments has IMO been pretty bad for lesswrong, the growing amount I’m expecting in the coming year seems even worse, and I’m not sure what else to do.
shift the default assumptions of LessWrong to “users by default have a rate limit of 1-comment-per day”
Natural times I expect this to be frustrating are when someone’s written a post, got 20 comments, and tries to reply to 5 of them, but is locked after the first one. 1 per day seems too strong there. I might say “unlimited daily comments on your own posts”.
I also think I’d prefer a cut-off where after which you’re trusted to comment freely. Reading the positive-selection post (which I agree with), I think some bars here could include having written a curated post or a post with 200+ karma or having 1000 karma on your account.
I’m not particularly attached to these numbers, but fyi the scale I was originally imagining was “after the very first upvote, you get something like 3 comments a day, and after like 5-10 karma you don’t have a rate limit.” (And note, initially you get one post and one comment, so you get to reply to your post’s first comment)
I think in practice, in the world where you receive 4 comments but a) your post hasn’t been upvoted much and b) none of your responses to the first three comments didn’t get upvoted, my expectation is you’re a user I’d indeed prefer to slow down, read up on site guidelines and put more effort into subsequent comments.
I think having 1000 karma isn’t actually a very high bar, but yeah I think users with 2+ posts that either have 100+ karma or are curated, should get a lot more leeway.
Here are some principles that are informing some of my thinking here, some pushing in different directions
Karma isn’t that great a metric – I think people often vote for dumb reasons, and they vote highest in drama-threads that don’t actually reflect important new intellectual principles. I think there are maybe ways we can improve on the karma system, and I want to consider those soon. But I still think karma-as-is is at least a pretty decent proxy metric to keep the site running smoothly and scaling.
Because karma is only a proxy metric, I’d still expect moderator judgment to play a significant role in making sure the system isn’t going off the rails in the immediate future
each comment comes with a bit of an attentional cost. If you make a hundred comments and get 10 karma (and no downvotes), I think you’re most likely not a net-positive contributor. (i.e. each comment maybe costs 1/5th of a karma in attention or something like that)
in addition, I think highly upvoted comments/posts tend to be dramatically more valuable than weakly upvoted comments/posts. (i.e. a 50 karma comment is more than 10 times as valuable as a 5 karma comment, most of the time [with an exception IMO for drama threads]
The current karma system kinda encourages people to write lots of comments that get slightly upvoted and gives them the impression of being an established regular. I think in most cases users with a total average karma of ~1-2 are typically commenting in ways that are persistently annoying in some way, in a way that’d be sort of fine with each individual comment but adds up to some kind of “death by a thousand cuts” thing that makes the site worse.
On the other hand, lots of people drawn to LessWrong have a lot of anxiety and scrupulosity issues and I generally don’t want people overthinking this and spending a lot of time worrying about it.
My hope is to frame the thing more around positive rewards than punishments.
I suggest not counting people’s comments on their own posts towards the rate limit or the “barely upvoted” count. This both seems philosophically correct, and avoids penalizing authors of medium-karma posts for replying to questions (which often don’t get much if any karma).
There should be fast tracks that present no practical limits to the new users. First few comments should be available immediately upon registration, possibly regenerating quickly. This should only degrade if there is downvoting or no upvoting, and the limits should go away completely according to an algorithm that passes backtesting on first comments made by users in good standing who started commenting within the last 3-4 years. That is, if hypothetically such a rate-limiting algorithm were to be applied 3 years ago to a user who started commenting then, who later became a clearly good contributor, the algorithm should succeed in (almost) never preventing that user from making any of the comments that were actually made, at the rate they were actually made.
If backtesting shows that this isn’t feasible, implementing this feature is very bad. Crowdsource moderation instead, allow high-Karma users to rate-limit-vote on new users, but put rate-limit-level of new users to “almost unlimited” by default, until rate-limit-downvoted manually.
I’m less optimistic than Ray about rate limits, but still think they’re worth exploring. I think getting the limits/rules correct will be tricky since I do care about the normal flow of good conversation not getting impeded.
I think it’s something we’ll try soon, but not sure if it’ll be the priority for this week.
“users by default have a rate limit of 1-comment-per day” and “1 post per week.”
Imagine a system that lets a user write their comments or posts in advance, and then publishes comments according to these limits automatically. Then these limits wouldn’t be enough. On the other hand, if you want to write a comment, you want to write it right away, instead of only starting to write it the next day because you are out of commenting chits. It’s very annoying if the UI doesn’t allow you to do that and instead you need to write it down in a file on your own device, make a reminder to go back to the site once the timeout is up, and post it at that time, all the while remaining within the bounds of the rules.
Also, being able to reply to responses to your comments is important, especially when the responses are requests for clarification, as long as that doesn’t turn into an infinite discussion. So I think commenting chits should accumulate to a maximum of at least 3-4, even if it takes a week to get there, possibly even more if it’s been a month. But maybe an even better option is for all but one of these to be “reply chits” that are weaker than full “comment chits” and only work for replies-to-replies to your own comments or posts. While the full “comment chits” allow commenting anywhere.
I don’t see a way around the annoyance of feasibility of personally managed manual posting schedule workarounds other than implementing the queued-posting feature on LW, together with ability to manage the queue, arranging the order/schedule in which the pending comments will be posted. Which is pretty convoluted, putting this whole development in question.
An obvious issue that could be fixed by the UI but isn’t, that can be worked around outside the UI, is deliberate degradation of user experience. The blame is squarely on the developers, because it’s an intentional decision by the developers. This should be always avoided, either by not creating this situation, or by fixing the UI. If this is not done, users will be annoyed, I think justifiably.
a reason to actively facilitate users having an easier time posting as often as possible
When users want to post, not facilitating that annoys them. If you actually knew that we want them to go away, you could’ve banned them already. You don’t actually know, that’s the whole issue here, so some of them are the reason there is a site at all, and it’s very important to be a good host for them.
Here’s my best guess for overall “moderation frame”, new this week, to handle the volume of users. (Note: I’ve discussed this with other LW team members, and I think there’s rough buy-in for trying this out, but it’s still pretty early in our discussion process, other team members might end up arguing for different solutions)
I think to scale the LessWrong userbase, it’d be really helpful to shift the default assumptions of LessWrong to “users by default have a rate limit of 1-comment-per day” and “1 post per week.”
If people get somewhat upvoted, they fairly quickly increase that rate limit to either “1 comment per hour” or “~3 comments per day” (I’m not sure which is better), so they can start participating in conversations. If they get somewhat more upvoted the rate limit disappears completely.
But to preserve this, you need to be producing content that is actively upvoted. If they get downvoted (or just produce a long string of barely-upvoted comments), they go back to the 1-per-day rate limit. If they’re getting significantly downvoted, the rate limit ratchets up (to 1 per 3 days, then once per week and eventually once-per month which is essentially saying “you’re sort of banned, but you can periodically try again, and if your new comments get upvoted you’ll get your privileges restored”)
Getting the tuning here exactly right to avoid being really annoying to existing users who weren’t doing anything wrong is somewhat tricky, but a) I think there are at least some situations where I think the rules would be pretty straightforward, b) I think it’s an achievable goal to the tune the system to basically work as intended.
When users have a rate limit, they get UI elements giving them some recommendations for what to do differently. (I think it’s likely we can also build some quick-feedback buttons that moderators and some trusted users can use, so people have a bit more idea of what to do differently).
Once users have produced a multiple highly upvoted posts/comments, they get more leniency (i.e. they can have a larger string of downvotes or longer non-upvoted back-and-forths before getting rate limited).
If we were starting a forum from scratch with this sort of design at it’s foundation, I think this could feel more like a positive thing (kinda like a videogame incentivizing good discussion and idea-generation, with built in self-moderation).
Since we’re not starting from scratch, I do expect this to feel pretty jarring and unfair to people. I think this is sad, but, I think some kind of change is necessary and we just have to pay the costs somewhere.
My model of @Vladimir_Nesov pops up to warn about negative selection here (I’m not sure whether he thinks rate-limiting is as risky as banning, for negative-selection reasons. It certainly still will cause some people to bounce off. I definitely see risks with negative selection punishing variance, but even the current number of mediocre comments has IMO been pretty bad for lesswrong, the growing amount I’m expecting in the coming year seems even worse, and I’m not sure what else to do.
Natural times I expect this to be frustrating are when someone’s written a post, got 20 comments, and tries to reply to 5 of them, but is locked after the first one. 1 per day seems too strong there. I might say “unlimited daily comments on your own posts”.
I also think I’d prefer a cut-off where after which you’re trusted to comment freely. Reading the positive-selection post (which I agree with), I think some bars here could include having written a curated post or a post with 200+ karma or having 1000 karma on your account.
I’m not particularly attached to these numbers, but fyi the scale I was originally imagining was “after the very first upvote, you get something like 3 comments a day, and after like 5-10 karma you don’t have a rate limit.” (And note, initially you get one post and one comment, so you get to reply to your post’s first comment)
I think in practice, in the world where you receive 4 comments but a) your post hasn’t been upvoted much and b) none of your responses to the first three comments didn’t get upvoted, my expectation is you’re a user I’d indeed prefer to slow down, read up on site guidelines and put more effort into subsequent comments.
I think having 1000 karma isn’t actually a very high bar, but yeah I think users with 2+ posts that either have 100+ karma or are curated, should get a lot more leeway.
Ah good, I thought you were proposing a drastically higher bar.
Here are some principles that are informing some of my thinking here, some pushing in different directions
Karma isn’t that great a metric – I think people often vote for dumb reasons, and they vote highest in drama-threads that don’t actually reflect important new intellectual principles. I think there are maybe ways we can improve on the karma system, and I want to consider those soon. But I still think karma-as-is is at least a pretty decent proxy metric to keep the site running smoothly and scaling.
Because karma is only a proxy metric, I’d still expect moderator judgment to play a significant role in making sure the system isn’t going off the rails in the immediate future
each comment comes with a bit of an attentional cost. If you make a hundred comments and get 10 karma (and no downvotes), I think you’re most likely not a net-positive contributor. (i.e. each comment maybe costs 1/5th of a karma in attention or something like that)
in addition, I think highly upvoted comments/posts tend to be dramatically more valuable than weakly upvoted comments/posts. (i.e. a 50 karma comment is more than 10 times as valuable as a 5 karma comment, most of the time [with an exception IMO for drama threads]
The current karma system kinda encourages people to write lots of comments that get slightly upvoted and gives them the impression of being an established regular. I think in most cases users with a total average karma of ~1-2 are typically commenting in ways that are persistently annoying in some way, in a way that’d be sort of fine with each individual comment but adds up to some kind of “death by a thousand cuts” thing that makes the site worse.
On the other hand, lots of people drawn to LessWrong have a lot of anxiety and scrupulosity issues and I generally don’t want people overthinking this and spending a lot of time worrying about it.
My hope is to frame the thing more around positive rewards than punishments.
I suggest not counting people’s comments on their own posts towards the rate limit or the “barely upvoted” count. This both seems philosophically correct, and avoids penalizing authors of medium-karma posts for replying to questions (which often don’t get much if any karma).
Yeah that probably makes sense.
There should be fast tracks that present no practical limits to the new users. First few comments should be available immediately upon registration, possibly regenerating quickly. This should only degrade if there is downvoting or no upvoting, and the limits should go away completely according to an algorithm that passes backtesting on first comments made by users in good standing who started commenting within the last 3-4 years. That is, if hypothetically such a rate-limiting algorithm were to be applied 3 years ago to a user who started commenting then, who later became a clearly good contributor, the algorithm should succeed in (almost) never preventing that user from making any of the comments that were actually made, at the rate they were actually made.
If backtesting shows that this isn’t feasible, implementing this feature is very bad. Crowdsource moderation instead, allow high-Karma users to rate-limit-vote on new users, but put rate-limit-level of new users to “almost unlimited” by default, until rate-limit-downvoted manually.
I’m less optimistic than Ray about rate limits, but still think they’re worth exploring. I think getting the limits/rules correct will be tricky since I do care about the normal flow of good conversation not getting impeded.
I think it’s something we’ll try soon, but not sure if it’ll be the priority for this week.
Imagine a system that lets a user write their comments or posts in advance, and then publishes comments according to these limits automatically. Then these limits wouldn’t be enough. On the other hand, if you want to write a comment, you want to write it right away, instead of only starting to write it the next day because you are out of commenting chits. It’s very annoying if the UI doesn’t allow you to do that and instead you need to write it down in a file on your own device, make a reminder to go back to the site once the timeout is up, and post it at that time, all the while remaining within the bounds of the rules.
Also, being able to reply to responses to your comments is important, especially when the responses are requests for clarification, as long as that doesn’t turn into an infinite discussion. So I think commenting chits should accumulate to a maximum of at least 3-4, even if it takes a week to get there, possibly even more if it’s been a month. But maybe an even better option is for all but one of these to be “reply chits” that are weaker than full “comment chits” and only work for replies-to-replies to your own comments or posts. While the full “comment chits” allow commenting anywhere.
I don’t see a way around the annoyance of feasibility of personally managed manual posting schedule workarounds other than implementing the queued-posting feature on LW, together with ability to manage the queue, arranging the order/schedule in which the pending comments will be posted. Which is pretty convoluted, putting this whole development in question.
LessWrong already stores comments you write in local storage so you can edit it over the rose of the day and post it later.
I… don’t see a reason to actively facilitate users having an easier time posting as often as possible, and not sure I understand your objection here.
An obvious issue that could be fixed by the UI but isn’t, that can be worked around outside the UI, is deliberate degradation of user experience. The blame is squarely on the developers, because it’s an intentional decision by the developers. This should be always avoided, either by not creating this situation, or by fixing the UI. If this is not done, users will be annoyed, I think justifiably.
When users want to post, not facilitating that annoys them. If you actually knew that we want them to go away, you could’ve banned them already. You don’t actually know, that’s the whole issue here, so some of them are the reason there is a site at all, and it’s very important to be a good host for them.