Voting Weight Discussion
One of the distinguishing features of LW 2.0 is Voting Weight. People with more karma will be able to give higher-weighted upvotes and downvotes.
This is part of an overall plan to make it better at sorting signal-from noise. But it’s a major change to the system and seemed worthy of a dedicated-discussion thread.
Intended Outcomes
There are roughly three goals for the improved karma system:
I. A reasonable pace of quality content on the front page, which is the single conversation locus. (We’re aiming for about 3-7 featured posts per week)
II. A reasonable opportunity for people (newcomers and otherwise) to get their ideas seen, receive feedback. (The front page involves an inherently limited supply of attention, so not everything can go there. But good posts should at least stick around on the Recent Posts page for awhile)
III. Implement goals I and II in a way that is (and feels) fair.
Current Implementation of Voting Weight
The problem is that LessWrong is an oddly specific community, and it won’t be necessarily obvious to a newcomer what are the sorts of post we want to incentivize. So we want more experienced users who have a proven track epistemic record to be able to draw attention to posts more easily.
In light of Goal III, we don’t want a high-karma user’s vote to be overwhelming. So the current mathematical implementation is:
floor(log_5(karma)+1)+1
What this means in non-math-speak is that is that your voting weight looks something like:
0 karma: 1 voting weight
5 karma: 2 voting weight
25 karma: 3 voting weight
125 karma: 4 voting weight
625 karma: 5 voting weight
…
… roughly capping out for practical purposes at approximately:
400,000 karma: 9 voting weight (currently no users are near this level)
You may have noticed that when you upvote or downvote a post, it changes by more than 1 point. This is why.
New posts also start with their user having upvoted them once, so if you have 25 karma, you’ll start with 3 points. This is to reflect that people who’ve been around longer are more likely to be writing things that are worth paying attention to. (Although note that this initial upvote doesn’t count towards their overall karma score)
Shifting to a voting-weight based system also makes it easier to notice people who are abusing the system and nullify their votes (preventing mass-downvote attacks)
Thoughts?
The LW 2.0 team has some thoughts about how to refine the voting system into something more fine-grained. But before sharing those ideas it seemed good to solicit a more general discussion. (Some of us might share our thoughts in the comments, but those thoughts will be from our perspectives as users rather than site developers)
I have noticed that starting posts at the poster’s karma creates a double counting: I think “Raymond posted this I should read it” and also “This has 3 points I should read it” but it has 3 points because Raymond wrote it. I then notice that it causes me to want to know how many points Raymond gives his initial posts so I can mentally subtract those 3 points and see if anyone else actually endorses this comment, so now I feel low-level pressured to memorize people’s karma scores. Which seems bad.
A question I find myself asking is “do people write posts they themselves would not wish to upvote?” I think the answer is yes, e.g. Raymond’s “good point, I will fix that” which is a good comment for him to write but which I do not think he would choose to upvote.
I do think that the sorting effect of attaching the person’s karma to the comment is actively good and would like to keep it. That could be kept distinct from the displayed score, same way it is kept distinct from the person’s karma score.
I also realize that I have a lot of strong preferences in such things, so we might want to consider (non-urgently) putting effort into giving people customization options for display of scores and sorting of comments.
I actually like the inflation quite a bit. A major part of why I didn’t feel that the original lesswrong was worth interacting with was that I didn’t feel like I would get large enough rewards from posting something epistemically valuable; higher-karma users being able to give and take more approval makes it feel more like my voice mattered to someone who I should care about. I do like the idea of being able to see +/- votes, because then you see ratio, but I definitely love the karma inflation.
On reddit, I get one point when I post a comment; getting downvoted puts me at 0 points, and so I wouldn’t want to send the signal that I got downvoted by un-upvoting my own post. This might be an argument for not upvoting one’s own posts by default, but if that’s the case, I would want “0 points” to be replaced with “no votes” if there aren’t any.
I also think karma should be kept under anti-goodhart treatment: it should have humans who have the option of totally changing the algorithm monitoring it for value alignment. Karma will be an optimization target for many users (attempted to make bet, too hard to specify), and displaying it seems important for keeping users who want social approval for their thinking coming back. On facebook, I have to get social approval by being funny; my hope is that lesswrong can be a place that I can get social approval for being epistemically useful. (In general, I don’t think I’ll care to interact with it if I don’t get social approval points back for it.)
Opposite preference here: I would very much like have the option to not see my own karma scores, or votes on anything that I post.
Upvoted among other things for attempting to operationalize a belief so it could be bet on.
update: no longer think I like vote inflation. I would have lost the virtual bet against Zvi.
Did something happen to change your opinion?
Noticed that, as [edit: Raymond, not Zvi] predicted, I started counting 3 as 1.
You can remove the upvote from your own comment or even downvote it.
I think the intent might actually be that the karma for a given comment isn’t displayed that prominently (i.e. a faint grey number buried in a string of numbers of icons). And that the idea is for most of the work being done by the sorting rather than by expecting people to read karma-count.
(although this is the case for comments, less so for posts)
This seems like another reason to display net upvotes on the page (in an easier-to-read font), and hide karma weighting behind a thing you have to click/hover to view. Net upvoters matter less than karma, so you’re giving boring information in an easy-on-the-eyes fashion, rather than giving interesting information in a hard-to-see fashion.
I noticed this. This seems to belong to a class where if you’re going to ignore the karma number and trust the sorting, being hard to see is good, but if you’re going to squint and look at it anyway, then making it hard to see is just a tax on eyes. I need to live with it for longer before saying for sure but I worry that this catches us in the bad middle. Can we actually avoid looking at all? So far my answer is no, I can’t.
I think the relative weighting is good and if anything is too weak, and I would consider taking out the initial point, which would mean that you need 5 Karma to vote at all. This seems fine since 5 Karma is one good comment, and the system can put your vote in retroactively. If anything it sends the right message that we want to know your opinion but do not yet put any weight on it! During the beta this might have odd effects since it does not appear that Karma has transferred from 1.0 (e.g. my posts on my first day here are starting at 1) so for now we should give everyone at least 1 point.
Inflation worries me a lot, though. As Raymond notes, when I see a 7 I instinctively think of that as 7 people voting up (or 7 more up than down). My brain knows what to think about that. If 20 is the new 7, then every time I see a number I’m doing math to translate back to the scale of people. This is together with the issue that comments start at 1+ rather than 0. The starting at 0 and going up slowly (since downvoting and stinginess) helped with the atmosphere and culture that LW1.0 projected. Each point mattered and getting them was hard and meaningful. It sent the message that one should be thoughtful (perhaps too thoughtful, but I am not worried about that error in 2.0 at least for now).
I find smaller numbers that map onto basic instincts, where I know what 1 means, much better and enabling of discrete, clean thinking than larger numbers. I despise the ‘Over 9000’ thing, and really appreciate Dragon Quest’s starting you out dealing 4 damage and maxing out at 99 versus Final Fantasy starting you out dealing 121 and maxing out at 9999. Too much precision! Too many significant figures! I get that a Reddit post having 2730 upvotes actually got upvoted that many times but what does that even mean beyond going viral? How do I think about that?
Thus I think we should consider normalizing the scale, at least such that when you see a 10, on average it means 10 people voted up. If we get more people maybe even normalize such that the scale maxes out at 1 rather than starting at 1. We might even think about displaying on a log scale (although we should not do the actual long-term calculation that way, that would be quite bad).
Hmm. Reading this I think pushes me towards:
Display number of upvotes, use karma only for sorting
This would let the visual-indicator tell us the most concrete thing about a post, while getting most of the benefits of the karma-as-sorting-mechanism, without having to figure out a karma-scale that was human-readable-yet-also-accurate.
Hmm. The main disadvantage I see of this is that it then makes it harder (or at least somewhat more confusing) to implement the “incremental upvote” thing (where you can upvote something up to a number of times equal to your karma)
Bonus suggestion: Reinterpret “∧” and “∨” as meaning “increase the amount of karma I give this post/comment” and “decrease it”, respectively, not as meaning “upvote” and “downvote” per se. So e.g. if you find a new comment and click ∧ once, that gives +1 karma; if you click ∧ a second time, then if your user karma is high enough, it goes up to +2 karma; etc. This means that clicking ∧ once followed by ∨ once results in 0, not in −1; this is the method for removing your vote entirely.
Giving <0 karma to something counts as a “downvote”; giving >0 karma counts as an “upvote”; giving 0 counts as not voting.
If you click and hold on the ∧ or ∨, a scale appears and you can drag the karma weighting of your vote along a -n to +n bar, where moving it to 0 will remove your vote.
There’s a clear visual animation that pops up when you click the ∧ or ∨ and successfully increase someone’s karma, whereas an inconspicuous “nothing happened!” bubble pops up if you try to up or down something past the point where you’re allowed to allocate that much karma or negkarma.
Suggestion: replace “n points” with “n votes,” displaying the net number of upvotes a post/comment has received. Hovering your mouse over “n votes” shows the number of upvotes, the number of downvotes, and the net karma gained. E.g., you might find a comment low on the listing that says “+6 votes”, hover your mouse over that text, and have this display:
8 upvotes
2 downvotes
+7 karma
(Representing, say, downvotes valued at −5 and −2, and upvotes valued at +1, +1, +1, +1, +2, +2, +2, +4. It’s not obvious to me that it’s good or necessary to make it clear this is the breakdown, though.)
> I would consider taking out the initial point, which would mean that you need 5 Karma to vote at all.
This is actually moderately harder to implement in a way that makes sense. If you set the initial thing to 0, people get confused about whether the buttons are working (it just looks like a bug, or if you make it explicit that it doesn’t work because they are new, it’s a fairly offputting initial attempt to engage with the discussion)
One possibility is to let people upvote things, and see the upvote, but it doesn’t actually start weighting things for other people until they’ve gotten at least 5 karma. This might be better but is more work to implement.
Another possibility (with some added work) is to not display the voting buttons at all until someone has 5 karma. Hacker News does this for downvotes, I believe.
My worry on letting their upvote only appear for them is that it’s dishonest, in addition to any coding issues.
I agree that the button doing nothing visible seems bad, so we’d want to have some other visual indicator that it worked, such as displaying number of upvotes somewhere—they’d still count for that.
I agree with this. I also think that needing to get 5 karma before voting isn’t that discouraging, and the kinds of users it discourages (ones who are too impatient to want to hang out and read stuff before they get to influence where things go; ones who will just lurk forever no matter what; ones who are unusually easily discouraged or unambitious...) might be outweighed by the kinds of users it encourages to comment who otherwise would just lurk. Particularly if the voting requirement is some low, obviously reachable karma level, I could imagine it feeling like a tantalizing prize that would make me want to start commenting.
Also, keeping brand-new users from voting would have some advantages for limiting sockpuppet shenanigans, ‘we just got linked from the front page of reddit’ mayhem, and the like.
Strong endorsement of the principle that not every potential user is a net positive, and driving away potential new users is not automatically bad. Maximizing views/users/etc too much is a classic trap metric.
I think that principle would apply to ‘leaves because they couldn’t vote without 5 karma’ but could be convinced otherwise by people worth keeping saying ‘this would drive me away if I were new.’
This did drive me away from both stackoverflow and lesswrong for a long time, and what finally made me feel able to interact with them is learning that it is possible to make good posts and get karma. I think it’s a reasonable restriction, and that it creates a problem that is likely best solved another way.
Huh—making sure I get this: you were driven away by inability to upvote, but you think it is probably still better to have that restriction and find some other way to reassure people they can get karma and participate?
Assuming I understand you correctly this is a stronger-than-average endorsement of the karma restriction.
StackExchange also has a minimum reputation requirement for votes to count. When you try to vote on something it displays a box saying the vote was recorded but doesn’t change the publicly displayed vote count.
What I don’t like about the way it is implemented on StackExchange is that it seems it’s not possible to take back a vote until you have enough reputation to vote at all.
Besides protecting the vote count from being distorted by newcomers, I think the main advantage is that it makes it much harder to farm a bunch of karma with sockpuppet accounts.
Reactions—Vote with Emojis or Tags
Karma currently conflates multiple possible reations into a single datum. An upvote currently could mean “me too” or “I updated” or “I agree” or “others should read this”, or “LOL”, etc. A downvote could mean “fallacy” or “poor quality” or “disagree”, etc. It’s hard for posters and readers to discern intent.
Facebook and Github (and probably others) now have a small number of Emoji reactions, instead of just +/- or “Like”. This is a more fine-grained feedback mechanism than a simple vote, but still easy (and familiar) enough for everyone to understand and use.
These give us additional low-effort feedback mechanisms that normally wouldn’t be worth a written reply. Facebook’s selection isn’t appropriate for Less Wrong, but we could choose some better ones. They also need not be icons. Words or short phrases (i.e. “tags”) will do. We’d probably want to pull memes about good discourse and their failure modes from the Sequences, e.g. #applause-light or #updated, etc., as well as all the well-known logical fallacies and biases. (These could also be links to the wiki for the uninitiated.)
For maximum flexibility, we could also give users the option to reply with a custom tag. Popular options could be added to the default suggestions, either automatically, or curated by the Sunshine Regiment. My concern with this approach is that custom tags may not have clear meanings, but a curated set could be linked to agreed-upon definitions in the wiki.
Tags also give us a good mechanism for straw polls. I’ve seen karma used this way on the old Less wrong, with a yes/no question asking for karma votes, (and a reply by the same author to counter balance it with opposite votes). The new weighted karma kind of destroys this feature. But with tags, we could vote with #yes #no or even #A #B #C #D, etc. for multiple-choice questions.
What happens if a user with 15 Karma upvotes something, gets to 30 karma, and then removes the upvote? Does it remove the original two karma, or the new three? Or are increases in karma weight retroactive, so your two vote becomes worth three when you reach 25 karma?
It would remove the original two karma, and if you would vote again the new vote would be worth three. I.e. increases in karma are not retroactive.
I think this is broken. I tested it on a post of mine with 1 upvote (from me), when I removed the upvote it went down to −2.
Nitpicking a misplaced parenthesis: I think the equation is probably
floor(log_5(karma+1))+1
rather than
floor(log_5(karma)+1)+1
I don’t think so. The second equation is negative infinity for karma = 0, which seems not right.
It sounds like you’re agreeing with Unnamed but think that you’re disagreeing?
(Another minor piece of evidence that the second equation is wrong, is that it could be more simply written floor(log_5(karma))+2.)
I believe the equation was constructed specifically to avoid that scenario (but also I don’t know the math to check it myself)
you can think of log (base b) as inverting exponentiation. Here’s a graph to refer to when reading these.
solve for x, and you get:
in other words:
If you try to invert exponentiation where the result of the exponentiation was 0:
Then the only thing that could produce that result is -inf, because anything greater than -inf just produces a fractional result. So if our full function is floor(log_5(karma)+1)+1, then when someone had 0 karma, they would have -inf voting power.
I have observed that this upvote does not effect the posters karma, this is a good design, but not clear from the text of this post.
Oh, good point. I’ll edit that in.
Okay, the thoughts that originally prompted me to write this post:
1) Point Inflation—it currently feels weird/wrong to me that I’m getting more points than usual. And then, when I realize that 17 points actually just means maybe 5 people upvoted my post, that feels a little sad. However, this seems like a temporary problem, and over time I’d get used to it.
The new paradigm seems like “3-weight votes” is basically what you’ll get as soon as you get the hang of things.
2) Not knowing how many people actually upvoted. I’m not sure how much I’d want this if I wasn’t used to having it, but it seems like something that might be useful to actually be able to check.
3) ‘Having’ to upvote/downvote more than feels fair. I’ve upvoted a post and felt a little weird because it was a *slightly* good post that I wanted to incentivize it, but giving it 3 karma felt overkill for what I wanted to give.
Alternately: if something is hovering at 2 karma, and I think it was a little bit wrong or poorly-done in some way, I sometimes want to downvote it but then feel bad about kicking it into the negatives, which results in me not downvoting it at all.
4) The existing of voting weight isn’t very intuitive—it feels at first like the site is just buggy, and this makes me want it to be more explicitly spelled out.
I have some thoughts on solutions, which I’ll make separate comments so they’re easier to upvote/downvote independently.
Karma-Weight as Max-Upvote, rather than Standard Upvote
Medium recently implement something called “claps” (see here), where you can basically “like” something as many times as you want. If you like it a lot, “clap” a lot. This is neat because it’s somewhat costly signaling—it actually takes time to clap things, there’s only so much you’d do it unless something actually seems really good to you.
A possible variation on this is something like “you can upvote something multiple times, and the limit of how much you upvote it is limited by your vote-weight”. (So, someone with 25 karma can upvote/downvote something up to 3 times).
This would resolve issue #3 above, and I think might add some additional value as a costlier signal.
If we went this route, there’s a few variations. Maybe you let people upvote a lot if it matters to them (which would mean even more point inflation). Maybe you can only upvote at the current log_5 scale, but make each additional vote require a lengthier (and perhaps more visually satisfying) click.
I’d probably lean towards your own posts still starting with your max-upvote.
(Note: Medium also doesn’t allow you to like something as many times as you want. Claps have a current maximum of 50, which is reached after about 1.5 minutes of clapping)
Ah, good to know.
Meta: this is the sort of thing I’d have wanted to give a simple like or thumbs-up to to indicate to you that I’d seen it without having to respond with a comment.
(This is part of an mostly-unrelated issue of “should upvotes be public, or should there be some way to do something upvote-like that is public?” that’s been discussed a bit in a few places)
I really like the system on Slack, Discord etc. where you can react to comments with various graphical icons and it shows who posted which reaction. Would be fantastic to have something similar here.
I want to push back on that, on two fronts. Such a system would be very bad and potentially rises to the dealbreaker level for me.
First point is that I think graphical icons are quite distracting and loud, and take us away from the kind of atmosphere of real discourse that we want. Icons and emoji and such are strongly associated for good reason with casual slash ephemeral conversation. They send the message that we’re texting, not debating.
Second point is that reactions of this type being non-anonymous seems very bad. I have come to the conclusion that social networks are toxic and should be considered harmful. If LW becomes a de facto social network rather than a place for discussion we’ve failed. Thinking about which people had which reaction to which things, and who is going to see you have which reaction to which thing, is central to that toxicity, and is again part of a very different type of thinking (and one I want to avoid, not embrace).
There are certainly times when it is valuable to tell someone “I have seen this” so there is an argument for a button that sends that message to the original poster (and only the original poster) but the counter-argument to that is that this puts subtle (or non-subtle) pressure on people to indicate whether they have seen things, and thus to check the site frequently. I very much do not want LW to be another site that people feel the need to keep current and constantly check for small updates. This is another big reason why I consider social networks toxic, as they reward constant refreshes. Email has this problem too and solving it while still being reachable quickly is an unsolved problem; some people such as Paul Christiano accept one-day turnaround times to avoid this issue. So overall, I prefer the equilibrium where there is no button, and if it is important that someone know you’ve seen something, you can comment/message to that effect and edit later.
I don’t think they necessarily need to be: see e.g. the small “agree, respectfully disagree, helpful” icons at the bottom of posts over at Paradox forums (sample thread). They look pretty nice and unobtrusive to me.
There seem to be both advantages and disadvantages to non-anonymity, and it’s not clear to me which one dominates. E.g. over on LW, a lot of people have mentioned that anonymous karma counts on their posts are pretty bad for motivation, and that a couple of named posters making comments such as “nice post” can feel much more rewarding than having lots of upvotes.
I agree that this can also have a negative effect, but given that one of the reasons why LW1.0 died seems to have been that people didn’t find it rewarding enough to put in the work of writing quality content, having more emotionally compelling feedback mechanisms seems worth considering.
I find it interesting that you’re worried the feedback isn’t compelling enough, and I’m worried it will be too compelling in bad ways. I strongly resonate with the idea that someone taking the time to write ‘nice post’ feels much better than getting a like or upvote. That seems good so long as doing so is rare and someone failing to do this does not feel like information, since it involves far more conscious effort.
I also like Rob’s idea of collapsing ‘minor’ comments, with my additional suggestion that the person you’re replying to defaults to seeing them in expanded form, and likely they start at sorting power −1 for other people. This could also be useful for things like “you have a typo or math error.”
I think both private non-anonymous reactions and public anonymous reactions are likely to be valuable, whereas public non-anonymous reactions could be potentially harmful and private anonymous reactions seem mostly useless.
“I’ve seen this” coming from the parent poster and “nice post” are valuable feedback for the author of the post/comment, but less useful information for other people so it would best be private and non-anonymous.
Reactions that say something about the content of a comment, like “interesting” or “confusing” are more useful if they are public and anonymous.
Proposal: instead of buttons, have a feature to mark a comment you’re making as “minor”, which causes it to be collapsed by default. A stack of 10 little collapsed comments saying “I agree” or “+1” or “SGTM” or “I’ve seen this” might be manageable in a way that uncollapsed comments wouldn’t be. (A feature like this might also encourage shy or uncertain people to comment more?)
Something like this might also encourage people to think of collapsed comments as boring, and not do the “oooh, this comment is collapsed, it must be super interesting and salacious” thing. (If comments at a sufficient depth get collapsed, then you might want them to display differently from comments that are minor or downvoted.)
This could also be useful if we have buttons, but only for a very limited set of reactions, and/or if all or most buttons are anonymous.
Are High Karma People Trustworthy? Does Karma Even Do The Thing We Want?
The hope is for high karma people to actually represent the kind of judgment we want. This is… more true than I think it’d be in most communities, but not overwhelmingly true to the degree I’d like it to be.
I’m interested in upvote-options that reflect different styles of agreement/endorsement (i.e. in many ways “I disagree with this person but think they are saying a valuable thing” is more valuable than “I agree with this person”, and definitely more valuable than “that post was funny”)
It’s possible to implement something like this with something like Facebook Reacts, and weight them in ways that seem more to “get at the thing we care about.”
Using karma for like/dislike and having a separate Arbital-style probability meter for every post and comment (plus an option to insert a probability meters into a specific part of your post) might get you a lot of the value of this, and it seems like a different genre from “buttons” in the sense Zvi is talking about.
Specifically, I would have every post and comment automatically come with a probability meter at the bottom for “How confident are you that all the key claims in this (post/comment) are true?”, which can maybe be disabled on a case-by-case basis. I’d expect people to normally just intuit what the key claims are, but people could also be encouraged by the existence of this feature to include an explicit tl;dr at the bottom or top of their post listing what they see as their key claims. (I’d consider it valuable to have the ‘whole-post/comment’ probability meter consistently ask this question, so that people can easily scroll quickly down a page and eyeball the probability meter for each comment/post without the extra mental overhead of needing to worry about whether the ‘whole-post/comment’ probability meter represents something fundamentally different in each case.)
Separately, I would also include a feature for inserting an arbitrary number of probability meters into the body of a post or comment, explicitly associated with particular claims.
(When people have multiple “key claims” and choose to explicitly list them, it can also be standard to include a probability meter for each one, in which case the whole-post/comment probability meter may not be useful (unless some people feel comfortable weighing in on the conjunction but not the conjuncts, which I could imagine happening for a variety of reasons). I see this redundancy as more or less harmless, and would still consider it useful in this case to make each-post/comment-gets-a-single-top-level-probability-meter a site-wide standard, both because it would make it possible to see the probabilities different entire posts or comments got in indexes of those posts and comments (e.g., “All posts” or “Featured posts”), and because maintaining this norm would make it more normal and cognitively available to do a lot of probability assignments.)
I would suggest making people’s probability judgments non-anonymous by default (to encourage accountability and conversation, and to keep the site from devaluing globally unpopular opinions that are held by high-status contrarians), but allowing them to switch to anonymous judgments if they want. I would also suggest marking when each probability judgment was made, and encouraging people to make periodic follow-up comments in the case of important/interesting things they’ve updated on, in which they give their new probability judgments for the exact same claim, without overwriting the record of their old judgment.
(I think overwriting is OK too, though, provided that there’s some way for users to access their own old judgment in case it’s important later (or for mods to find them), and provided that the probability judgments receive dates. The ability to “remove” an old probability judgment seems generally good insofar as it makes people less worried that they’ll have an embarrassing mistake immortalized on the Internet forever, and therefore encourages them to try assigning probabilities to things more. Ideally people would never remove old judgments, even if they misunderstood the thing they were answering—the probability you’re misunderstanding a question should always be factored into your probability assignments—but I don’t think this is a case where we want to force people to do the ideal thing.)
I assume a lot of people would abuse probability meters for all sorts of trivial/weird/meta things, once they became commonplace. E.g., “Proposition: I did a good job summarizing my key claims above.” “Proposition: Most people will assign <50% probability to this proposition.” “Proposition: You found OP confusing.” “Proposition: If you threw a dart at a random point on a dartboard whose target was as large as a proportion of dartboard as the amount you liked the new Blade Runner movie is high compared to how much you like your favorite movies, then you would hit the target.” “Proposition: You’ll join me at the MIRIxLA meetup tomorrow at noon.” I consider this an actively good thing and want to be in a culture where probability assignment is so ubiquitous that it gets incorporated into trifles and in-jokes and gets appropriated for a variety of uses.
Most of these ideas seem good in isolation (or pointing in the direction of good things). I think they’d add up to significant complexity cost for the page, so figuring out to what degree they are worth adding to the overall cognitive load for the site will be an issue.
I agree that we should be very careful about adding complexity, especially when the complexity is added to every single post and comment. I can think of a few things that might reduce the complexity:
1. Instead of having a big eye-catching Arbital-style visualization of the probabilities displayed for every post and comment on the site, display an aggregate probability in boring grey text that matches the other text, and have people hover/click on that text to view/predict. E.g., your comment header could look like this:
Raemon +2 votes ∧ ∨ 𝗣 ≈ 0.9 6h
You could still include the full Arbital-style visualization within posts and comments, but it would be a deliberate choice by the post/comment author, rather than being a default. In cases where not enough people have assigned probabilities to the post/comment for the system to think it’s worth displaying an aggregate probability, the default visualization will be 𝗣 = ?.
2. Use a functionally similar (though visually distinct) click-and-drag sliding scale for the voting system’s karma-weighting that we use (see above) and for assigning probabilities, so some of the basic habits and motor intuitions people build up with karma can also be used for probability.
In general: My vague understanding of Oliver and co.’s vision for LessWrong is that LessWrong is to be a site where probability assignments, predictions, cruxes, bets, etc. play a huge role. Having easy infrastructure for making and comparing probability assignments might be more of a core feature than the full range of “buttons”, particularly if some of the key buttons can themselves be implemented as probability assignments.
More specifics of how I might implement a probability system like this:
Before you can assign probabilities, you need a “level-1 calibration” badge, achieved via hitting a certain calibration level in a LW/CFAR game/app. (You can then get level-2, level-3, etc. calibration badges for even better performance, maybe unlocking other site features like karma-betting markets.)
Everyone’s probability assignments (which are non-anonymous by default) can always be viewed by clicking or hovering on a “collapsed” probability (i.e., one that look like 𝗣 = foo instead of like an Arbital probability distribution image).
Collapsed probabilities will display as 𝗣 = ? until, e.g., some number of users with at least 2000 karma between them have assigned probabilities to that comment/post. Some aggregated probability will then be displayed for “foo” in 𝗣 = foo, with better-calibrated users (according to badge count) receiving more weight in the aggregation.
The main purpose of hiding the probabilities behind 𝗣 = ? until enough high-karma users have weighed in is to discourage low-karma users from getting really excited and wasting time running around and assigning probabilities to every comment on the site. If someone feels like doing that, that’s totally fine — maybe they’ll learn something from the process — but those probabilities shouldn’t be prominently displayed, because a lot of comments on the site are things like “I agree!” or “Woah.” where it doesn’t really matter if someone decides to waste their time adding silly probability assignments, but it does start carrying a cost if this makes silly probability assignments distracting and visible to anyone visiting the page.
(Note that users’ karma totals do not affect the weight users’ assignments receive in the probability aggregation at all, even though it affects how prominently the probabilities are displayed. On the other hand, it might be fine to weight users more if they have badges for things other than calibration, e.g., a “general knowledge” badge reflecting that you’re unusually good at answering Jeopardy questions or what-have-you.)
Is there an existing CFAR/LW calibration app that we consider good? (I know there have been attempts but haven’t actually used them myself)
New thing: https://www.openphilanthropy.org/blog/new-web-app-calibration-training
This one broke for me a few times :/
I actually also think the UI design and feedback mechanisms are a lot worse, so I would recommend that people still use the old one.
LW has a lot of old content, with old votes on it. This change will introduce a big disparity between the power of votes in the (pre-LW2) past and votes in the (post-LW2) future. I don’t know that this is a problem, but it’s a bit weird. (Consequence 1: when interpreting the score on a post or comment, you’ll need to think about whether it’s mostly been voted on recently or in the past. Consequence 2: when something has been voted on both recently and in the past, past voters’ votes will have less influence on its score. Consequence 3: there will be a temptation once everything is integrated to go back to old material and retract-and-redo one’s votes; people who choose to do this will in effect have more voting power—as regards that old material—than people who don’t, and this may have quite a substantial effect on the karma scores of the people whose posts and comments those votes are made on.)
Bayesian Karma
Attention is a limited resource. I don’t have the time or interest to read every comment on LessWrong. So what is karma even for? I use the karma score for one simple yes-no question: “Is reading this worth my time?”.
Is displaying the number of upvotes minus downvotes really the best way to answer the question? Karma should not be about mere popularity, or path dependence based on whoever voted first. The weighting system is an improvement, but I think we can do better.
Display the estimated probability (as a percentage) that the post is worth my time.
Users will start with a reasonable personal karma score (say less that 50%), while established users with high karma on the old system will start with somewhat higher personal scores. Then the initial (prior) score of a post is the score of its author.
Users can then upvote or downvote posts, and this will be taken as Bayesian evidence about the quality of that post, shifting its score from its prior in the direction of the vote.
The probability that any given vote is accurate will be basted on the voters’ karma percentage. Those with high personal karma will be assumed to have better judgement (because they write high karma posts), so their vote will have more weight.
And finally, the user’s personal karma percentage will be adjusted from their prior by using the karma percentage of their posts as Bayesian evidence. This means that the personal karma percentage of a user is the estimated probability that the user’s next post is worth reading.
This is an interesting idea, although I’m not sure if the degree-of-(in)accuracy of the percentage would end up being more useful than the current weighted karma. (i.e, converting upvotes into a percentage-of-worth-it-ness seems like it’d have to go through several iterations before reaching something that was accurate enough to be better)
I’m not exactly sure what you mean by “iterations” here. Is it about getting enough votes? Or about what conversion function to use when grandfathering established users?
I think it would be possible to experiment with the current data. You have a record of the dates of all posts and votes so far. Rather than grandfathering in established users with some human-estimated prior, give everyone the same starting score, and try computing their current karma% from scratch. See if it gives you reasonable answers. See if it finds hidden gems. Try a different prior (with enough votes, any reasonable choice should get similar results). This won’t answer questions about incentives, but it will give you a good comparison to the current unbounded karma system.
A not-so-obvious consequence of weighted voting (in the form we currently have, as opposed to Rob’s suggestion which I think I like a lot): if you know or suspect that something has been voted on only once since it was posted (or: since you last looked) then you can tell what voting weight the person who voted on it had, and hence get some information about who it was.
I’ve been thinking a lot about karma, as any numerical incentive system risks people going down incentive gradients, checking numbers and otherwise aiming for the number rather than the thing we want to target. This is a Hard Problem, since we 100% need karma for LW to work at all. The first level solution (in addition to weighting, which likely will help) is for everyone to be thinking hard about this and vote accordingly rather than voting locally and instinctively, but that feels insufficient. I’m planting this flag to encourage others to think more about karma problems outside of the weighting and inflation questions, which I’ll address in a distinct comment while noting as an example that the Karma system encourages splitting comments in this way (which in this case I think is good).
On StackOverflow there’s a minimum of 15 reputation to be able to cast upvotes. I think it’s better when newly registered accounts don’t have the ability to vote.
What happens to users with negative karma? (I have noticed one such user commenting on my posts.)
Currently, their voting weight goes to zero but their ability to post is unaffected. My personal off-the-cuff opinion is something like “if you get −20 karma or something in that range, your comments should start being auto-collapsed”, but that is not currently the case and would require some dev work.
My karma from the old LessWrong did not port over. Is this normal?
It’s a temporary issue (nobody’s karma has ported over as-of-now). It turns out migrating millions of upvotes from one system to another is hard.
That makes sense. On a mostly unrelated note, is there any way to get notified when someone replies to my comment?
Quick check: if you click on the Bell icon at the top-right, do you see any notifications about replies?
(Last I checked, notifications appeared, we just hadn’t created a thing to turn the bell red or anything when they happen)
Ditto.
Per-Paragraph Voting
I sometimes find myself agreeing (or disagreeing) with only part of a long comment.
Current options include replying with a quotation of the part in question, or voting on the whole thing based on the part. Sometimes this isn’t worth the effort or seems unfair and I wish I could just vote on that one part.
Conciseness in comments is a virtue given our limited free time and attention, but I don’t want to turn Less Wrong into Twitter. Due to inferential gaps, some concepts really do take long comments to get across, but aren’t worth a top-level post. Our karma system should accommodate that.
I don’t think it should be so fine-grained that we could insert a vote between every character. That’s noise. It could really make a popular post hard to read. On the fine scale, per-sentence, or perhaps even per-punctuation mark is more reasonable.
But I suspect that the appropriate granularity is per paragraph. There’s already a visual gap there, so you can insert karma votes (or tags) without hurting readability.
I’m not sure how to handle the case of edits. Minor typo fixes aren’t a problem, but changing the number of paragraphs, or moving sentences around could be a problem. But there’s nothing stopping us now from completely rewriting a comment to say something completely different. This is bad form, of course. And it might be bad form to change that in a deceptive way once it’s voted on, but honest mistakes are possible.
I think edited posts (even comments) should have a publicly-visible history, like a wiki.
I don’t know if this is worth the cost of implementing, but I absolutely like the idea. In particular, in the hypothetical world where this was in place, if one had e.g. two karma to spend per post (upvote strength of two) they could at most upvote or downvote two particular paragraphs, such that they’d left the same “weight” of opinion, but in a more detailed way.
Oliver, Ben and I were actually talking about this a few days ago. (I actually ran into the “man I really wish I could upvote this paragraph and downvote this other paragraph a week ago)
A few thoughts came up (epistemic status: current thoughts, nothing clearcut)
a) it might incentivize certain writing styles. Exact details depend on the execution, but regardless you’re fiddling with Goodheart’s Demon. It could have weirder consequences than you intend. And this might subtly make people with different writing styles (in particular ones with more meandering styles rather than clearly bulleted points) feel like the site didn’t want them to write the way they felt most comfortable, which leads to them writing less.
b) you might be able to get most of the same value by implementing claims as an object type. i.e. let people write their article without worry about how pieces of it will get upvoted/downvoted, but afterwards you have the option of adding a summary of explicit claims, which get listed as short bullet points and potentially top-level-comments to respond to. Those claims can be either upvoted/downvoted, or potentially (see Robby Bensinger comment somewhere on this page), get a “probability this is correct” a la Arbital.
The Claim Thing is also fiddling with incentives that might have unintended consequences, but seemed like it might be pointing in a direction that was a) easier to implement, and b) if done well could have other valuable downstream effects, c) you probably don’t want to implement both because they are occupying similar niches, and each feature adds to the cognitive load of the site.
Is there a plan to import our old karma? I see I currently have none when I definitely have more than none on old LW.
Kinda saying the obvious here, but just in case: did anyone consider an “eigentrust” or analogous system?