Have you or anyone else on the LW team written anywhere about the effects of your new rate-limiting infrastructure, which was IIRC implemented last year? E.g. have some metrics improved which you care about?
We haven’t written up concrete takeaways. My sense is the effect was relatively minor, mostly because we set quite high rate limits, but it’s quite hard to disentangle from lots of other stuff going on.
This was an experiment in setting stronger rate-limits using more admin-supervision.
I do feel pretty solid in using rate-limiting as the default tool instead of temporary bans as I think most other forums use. I’ve definitely felt things escalate much less unhealthily and have observed a large effect size in how OK it is to reverse a rate-limit (whereas if I ban someone it tends to escalate quite quickly into a very sharp disagreement). It does also seem to reduce chilling effects a lot (as I think posts like this demonstrate).
I think one outcome is ‘we’re actually willing to moderate at all on ambiguous cases’. For years we would accumulate a list of users that seemed like they warranted some kind of intervention, but banning them felt too harsh and they would sit there in an awkwardly growing pile and eventually we’d say ‘well I guess we’re not really going to take action’ and click the ‘approve’ button.
Having rate limits made it feel more possible to intervene, but it still required writing some kind of message which was still very time consuming.
Auto-rate-limits have done a pretty good job of handling most cases in a way I endorse, in a way that helps quickly instead of after months of handwringing.
The actual metric I’d want is ‘do users who produce good content enjoy the site more’, or ‘do readers, authors and/or commenters feel comment sections are better than they used to be?’. This is a bit hard to judge because there are other confounding factors. But it probably would be good to try checking somehow.
Have you or anyone else on the LW team written anywhere about the effects of your new rate-limiting infrastructure, which was IIRC implemented last year? E.g. have some metrics improved which you care about?
We haven’t written up concrete takeaways. My sense is the effect was relatively minor, mostly because we set quite high rate limits, but it’s quite hard to disentangle from lots of other stuff going on.
This was an experiment in setting stronger rate-limits using more admin-supervision.
I do feel pretty solid in using rate-limiting as the default tool instead of temporary bans as I think most other forums use. I’ve definitely felt things escalate much less unhealthily and have observed a large effect size in how OK it is to reverse a rate-limit (whereas if I ban someone it tends to escalate quite quickly into a very sharp disagreement). It does also seem to reduce chilling effects a lot (as I think posts like this demonstrate).
I think one outcome is ‘we’re actually willing to moderate at all on ambiguous cases’. For years we would accumulate a list of users that seemed like they warranted some kind of intervention, but banning them felt too harsh and they would sit there in an awkwardly growing pile and eventually we’d say ‘well I guess we’re not really going to take action’ and click the ‘approve’ button.
Having rate limits made it feel more possible to intervene, but it still required writing some kind of message which was still very time consuming.
Auto-rate-limits have done a pretty good job of handling most cases in a way I endorse, in a way that helps quickly instead of after months of handwringing.
The actual metric I’d want is ‘do users who produce good content enjoy the site more’, or ‘do readers, authors and/or commenters feel comment sections are better than they used to be?’. This is a bit hard to judge because there are other confounding factors. But it probably would be good to try checking somehow.