Coding day in and out on LessWrong 2.0
Yep, on this page you can see all nominations and reviews, plus all the posts with at least two nominations: https://lesswrong.com/reviews
(I confirm this is not a moderator, and does not have any special arrangement with moderators)
Yeah, this seems reasonable. One of the nice things about this migration is that it’s now very easy for us to adjust the rules and then just rerun the history again. So now is a pretty good time for suggestions for how the rules should change.
(I am promoting this site-meta post to the frontpage, so that users who have personal blogposts filtered out aren’t surprised that suddenly a lot of scores are different. But generally we try to keep most site-meta stuff that isn’t crucial for people to know about off the frontpage.)
This also strikes me as backwards, and the literature seems to back this up. Learning rates seem to differ a lot between different people, and also be heavily g-loaded.
I think in-practice there are lots of situations where you can confidently create a kind of pocket-universe where you can actually consider hypotheses in a bayesian way.
Concrete example: Trying to figure out who voted a specific way on a LW post. You can condition pretty cleanly on vote-strength, and treat people’s votes as roughly independent, so if you have guesses on how different people are likely to vote, it’s pretty easy to create the odds ratios for basically all final karma + vote numbers and then make a final guess based on that.
It’s clear that there is some simplification going on here, by assigning static probabilities for people’s vote behavior, treating them as independent (though modeling some subset of independence wouldn’t be too hard), etc.. But overall I expect it to perform pretty well and to give you good answers.
(Note, I haven’t actually done this explicitly, but my guess is my brain is doing something pretty close to this when I do see vote numbers + karma numbers on a thread)
So I’m not sure how we distinguish what’s ruled out from what isn’t.
Well, it’s obvious that anything that claims to be better than the ideal bayesian update is clearly ruled out. I.e. arguments that by writing really good explanations of a phenomenon you can get to a perfect understanding. Or arguments that you can derive the rules of physics from first principles.
There are also lots of hypotheticals where you do get to just use Bayes properly and then it provides very strong bounds on the ideal approach. There are a good number of implicit models behind lots of standard statistics models that when put into a bayesian framework give rise to a more general formulation. See the Wikipedia article for “Bayesian interpretations of regression” for a number of examples.
Of course, in reality it is always unclear whether the assumptions that give rise to various regression methods actually hold, but I think you can totally say things like “given these assumption, the bayesian solution is the ideal one, and you can’t perform better than this, and if you put in the computational effort you will actually achieve this performance”.
I think it definitely changed a bunch of stuff for me, and does at least a bit invalidate some of the things that Eliezer said, though not actually very much.
In most of his writing Eliezer used bayesianism as an ideal that was obviously unachievable, but that still gives you a rough sense of what the actual limits of cognition are, and rules out a bunch of methods of cognition as being clearly in conflict with that theoretical ideal. I did definitely get confused for a while and tried to apply Bayes to everything directly, and then felt bad when I couldn’t actually apply bayes theorem in some situations, which I now realize is because those tended to be problems where embededness or logical uncertainty mattered a lot.
My shift on this happened over the last 2-3 years or so. I think starting with Embedded Agency, but maybe a bit before that.
Nope, still broken. When I try to access them the site asks me for a password (i.e. if I go directly to the link where they are hosted) so that’s probably related. I expect turning off that password protection will probably make them visible again.
Promoted to curated: I’ve been meaning to follow up on the state of anti-aging work for a while, and this was really a quite good overview. I also know of a number of other people who found it useful. Thank you for your work when compiling this overview!
I liked this dialogue quite a bit.
Yeah, they were hosted on Matt’s website, and are now down. Though it probably also means they can still be restored.
This feels generally related to the problems covered in Scott and Abram’s research over the past few years. One of the sentences that stuck out to me the most was (roughly paraphrased since I don’t want to look it up):
In order to be a proper bayesian agent, a single hypothesis you formulate is as big and complicated as a full universe that includes yourself
I.e. our current formulations of bayesianism like solomonoff induction only formulate the idea of a hypothesis at such a low level that even trying to think about a single hypothesis rigorously is basically impossible with bounded computational time. So in order to actually think about anything you have to somehow move beyond naive bayesianism.
We got a lot of complaints about it, and one of the first things we did was to add ways to filter out coronavirus related stuff from the frontpage. Of course not everyone minded, but I very distinctly had a sense that it was difficult to talk about anything else. It was also all very news-focused, which is something I care a lot about avoiding with LessWrong, because I think becoming news-focused is a very strong attractor for online communities that usually has pretty bad effects on overall intellectual progress.
I decided that Zvi’s content is bypassing the usual frontpage guidelines for now, in an effort strike a balance between informing people of important and urgent developments, while not overwhelming the whole site with political and coronavirus-related content again.
Here is the relevant comment where I announced this policy: https://www.lesswrong.com/posts/vkvaAXHN2zPXhDjJC/covid-12-3-land-of-confusion?commentId=dR8FYiztgL4pzwpCs
To make it easier to read, the full text:
Given rising case counts and the importance of keeping everyone up-to-date with COVID stuff given the holiday season, I am making an exception to our usual frontpage guidelines and promoting this to the frontpage (as well as subsequent update posts in the coming weeks). This does not apply to other COVID content, and is only temporary until case counts either drop substantially again, or we decide for some other reason that these should no longer be on the frontpage. Zvi still has dictatorial control over moderation. Feel free to comment on this decision here, happy to talk about it and pretty open to changing my mind. This really wasn’t an obvious call.
I agree with most of this review, and also didn’t really like this post when it came out.
I think the first one could plausibly be a reason that we would want to promote this on LW. Unfortunately, I think it is wrong: I do not think that people should usually feel upon themselves the burden of bucking bad incentives. There are many, many bad incentives in the world; you cannot buck them all simultaneously and make the world a better place. Rather, you need to conform with the bad incentives, even though it makes your blood boil, and choose a select few areas in which you are going to change the world, and focus on those.
Just for the record, and since I think this is actually an important point, my perspective is that indeed people cannot take on themselves the burden of bucking bad all bad incentives, but that there are a few domains of society where not following these incentives is much worse than others and where I currently expect the vast majority of contributors to be net-negative participants because of those incentives (and as such establishing standards of “deal with it or leave it” is a potentially reasonable choice).
I think truth-seeking institutions are one of those domains, and that in those places, slightly bad incentives seem to have larger negative effects, and also that it is very rarely worth gaining other resources in exchange for making your truth-seeking institutions worse.
For almost any other domain of the world (with the notable exception of institutions that are directly responsible for handling highly dangerous technologies), I am much less worried about incentives and generally wouldn’t judge someone very much for conforming to most of them.
I agree with this, and was indeed kind of thinking of them as one post together.
Yeah, I think DT is very unrepresentative. I also think COVID really sucked for everyone, and increased the variance of everything by a lot. I am definitely extremely glad I wasn’t living alone during COVID and had friends in my house that allowed me to maintain basic social functions during the harshest parts of quarantine, but it also definitely created conflict and was stressful for many.
This post surprised me a lot. It still surprises me a lot, actually. I’ve also linked it a lot of times in the past year.
The concrete context where this post has come up is in things like ML transparency research, as well as lots of theories about what promising approaches to AGI capabilities research are. In particular, there is a frequently recurring question of the type “to what degree do optimization processes like evolution and stochastic gradient descent give rise to understandable modular algorithms?”.