Please include a vote in your annual review comments! Often there are comments with mixed takes, or short comments with positive takes, but I don’t know how it shapes up. Is this strong positive only a +4? Is this weakly positive actually a +9? Is this highly mixed review a −1 or a +9? It helps to know for many reasons (for example, does this person think I should be giving a different vote to my current one?).
This isn’t literally always the right call, but I think it should probably be considered the default.
This seems backwards. Natural language review comments convey the substance of what people think; the one-dimensional review vote is an extremely lossy compression that only exists to help allocate reader attention and because we want to make it a competition. NLP tech is basically good enough already that in the future, you should do away with manual voting altogether: just embed the review comments into latent space, and let people project out a ranking along some dimension of interest if they want a ranking.
I don’t think a one dimensional vote is necessarily just a compression of the comment, because weighing all the points made in the comment against each other is an extra step that the commenter might not have done in the text.
E.g the comment might list two ways the post is good and two ways it’s bad but not say or even imply whether or not the bad outweighs the good. The commenter might not even have decided. The number forces them to decide and say.
The version of Ben’s argument I’d make is “put some effort into the top-line of your review to somehow summarize either what your overall takeaways were, or, summarize why someone might want to read your review in full.”
(We could leave the compression to the Machines but I don’t think they’d do that good a job yet. It is plausible they could do a good enough job next year, but, I also kinda think LW should err on the side of not outsourcing that sort of thing.)
Please include a vote in your annual review comments! Often there are comments with mixed takes, or short comments with positive takes, but I don’t know how it shapes up. Is this strong positive only a +4? Is this weakly positive actually a +9? Is this highly mixed review a −1 or a +9? It helps to know for many reasons (for example, does this person think I should be giving a different vote to my current one?).
This isn’t literally always the right call, but I think it should probably be considered the default.
This seems backwards. Natural language review comments convey the substance of what people think; the one-dimensional review vote is an extremely lossy compression that only exists to help allocate reader attention and because we want to make it a competition. NLP tech is basically good enough already that in the future, you should do away with manual voting altogether: just embed the review comments into latent space, and let people project out a ranking along some dimension of interest if they want a ranking.
I don’t think a one dimensional vote is necessarily just a compression of the comment, because weighing all the points made in the comment against each other is an extra step that the commenter might not have done in the text.
E.g the comment might list two ways the post is good and two ways it’s bad but not say or even imply whether or not the bad outweighs the good. The commenter might not even have decided. The number forces them to decide and say.
The version of Ben’s argument I’d make is “put some effort into the top-line of your review to somehow summarize either what your overall takeaways were, or, summarize why someone might want to read your review in full.”
(We could leave the compression to the Machines but I don’t think they’d do that good a job yet. It is plausible they could do a good enough job next year, but, I also kinda think LW should err on the side of not outsourcing that sort of thing.)
If this is recommended, then (eventually) the UI should allow you to voluntarily reveal the vote.
If you want it to be default, LW should enable it by default with a checkbox for “Hide score”
I agree, but I’m too busy atm to code it, so I thought this quick take was worth it anyway.