LessWrong team member / moderator. I’ve been a LessWrong organizer since 2011, with roughly equal focus on the cultural, practical and intellectual aspects of the community. My first project was creating the Secular Solstice and helping groups across the world run their own version of it. More recently I’ve been interested in improving my own epistemic standards and helping others to do so as well.
Raemon
Karma: 60,707
fwiw I’d value it if this were crossposted in full over here, most people don’t click through linkpost links and it’s easier to have a discussion about it if the text is all here.
Yeah, but I feel like Screwtape was acknowledging this two comments ago and I’m not sure why you think there’s still more updating to be done here.
(I think his two-comments-ago-comment was saying “learning to implementing bubble sort, and then implementing bubble sort, might cost time, but not implementing bubble sort might mean your servers are taking way longer to answer queries and costing more compute and costing you more money than your hourly rate for the time it took to learn and implement bubble sort)
fwiw I did enjoy reading the first ~half of this (up through the last conversation with Mr Asi, and the beginning of the second conversation with Tessa). But after we started getting into each individual fallacy, I was like “okay I get it” and couldn’t bring myself to keep reading. (I skipped to the end, which was a fine end, given the genre).
When I write regular blogposts, I often try to fit in examples that cover every major type of objection or concern I anticipate, but my impression is that people have the patience for more like 3 examples (rather than the 6-10 I end up writing), and just skim the rest, and it’s sort of a fabricated option to cover everything.
Nod, though I expect there’s value in various flavors of politician / public figures understanding these concepts who can put pressure on the CEOs.
(I dunno that that’s the top theory-of-change I’d be spending my effort on, but, if my assumption is right that this is more like “getting marginally more utils out of low-effort-side-project-time”, that’s not exactly a crux)
Serious question: (well, it’ll start as “more of a comment, really”, but, at the end I do have a question)
The comment: I think the world is bottlenecked on people understanding the sort of concepts in posts like these. I don’t think the world is particularly bottlenecked on current-gen-Yudkowsky-shaped dialogue essays about it. They appeal to a small set of people.
My guess is you write them anyway because they are pretty easy to write in your default style and maybe just mostly fun-for-their-own-sake. And when you’re in higher-effort modes, you do also write things like If Anyone Builds It, that are shaped pretty different. And, probably these essays still help some people, and maybe they help workshop new analogies that eventually can be refined into If Anyone style books or podcast interviews.
But, that said, my questions are:
How much have you experimented with finding low-energy-but-different formats, that might reroll on who finds them compelling?
(I’m particularly interested in if there turns out to be anything short in this reference class)
How much have you (or, anyone else, this doesn’t have to be you) systematically thought about how to improve the distribution channel of this sort of essay so it reaches more people?
Both of these are presumably high effort. I’m not sure if the the first one is a better use of your high-effort time than other things, or how likely it is to work out. But, wondering if this is an area you think you’ve already checked for low or mid-hanging fruit it.
(Also, having now thought about it for 5 min, I think this sort of thing would actually make a good youtube video that the Rational Animations people might do. That could be mostly outsourced)
“What’s hard about this? What can I do about that?” (Recursive)
Yeah, I agree there’s a whole-ass “how to actually do this” post that would be a lot more effort to write. And I agree, because this is fairly costly, you should put some thought it.
For now, I just wanted to get it into people’s option-space. If you’re trying a lot of microoptimizations but you have a nagging feeling that your life situation is fundamentally off, try fixing your overall life situation with more drastic macro-moves.
One thing to look for, is, when you talk to your friends about your problems, instead of being like “oh yeah I have those problems too”, are they more like “uh, are you sure you’re okay? This doesn’t seem normal.”
For example, when I was working at a job that was stressful and I came home crying a couple times a week, my then girlfriend said “hey, um, this isn’t okay. This is not how jobs are supposed to be.” And I’d say “but I think there’s a rare opportunity at this job to do something really meaningful” and she said “idk man it’s not that rare, there are other places you could work that do similar kinds of things.”
(This is a clue in cases where it’s, like, really clear cut. There are are subtler cases, that I don’t have off-the-cuff examples about)
I think that could probably also use to be a short post with a 5 word title encapsulating it.
Ah, gotcha. I think I hadn’t been tracking this sort of thing (i.e. incidental meta-white-lies) in the first paragraph. I think I happen to not do them much, but will keep an eye out for them.
I think I did track things like the second paragraph, but, I resolved them more on the “regular honesty” rather than “meta honesty level”, i.e. figuring out what I could say that addressed the thing they needed without lying. (although it sounds like you maybe got a harder case than I’ve recently dealt with).
On the object-level-tangent, you’ve maybe figured out your own version of this sort-of-thing now, but, the way I handle the second-case is: If it seems like they need a particular thing, try to give them that particular thing. If it looks like they’re currently an emotional vortex that’s not going to be happy no matter what I say, then I tell them (in an empathetic voice, trying to radiate reassurance):
“Hey man, it currently feels like you’re spiralling/fixating on this in a way that doesn’t actually make sense. I’m not going to give you the specific kind of reassurance you’re asking for here because I don’t think it’ll help. But I think you’re being fatalistic here in a way that doesn’t make sense. I think you’re currently living in a depression hole where you can’t imagine anything good happening, and there will be days where you are less in this hole and making progress will be more possible.” (Meanwhile: “wanna go for a walk, or get help getting an appointment with a psychiatrist booked?” depending on situation and how much effort I actually want to invest)
Some specific things that have come up for me, some of which are about “metahonesty-qua-metahonesty” and some is just… idk, getting more intentional about honesty.
I’ve generally switched to “never lie” as a strong default and noticing white lies and similar more reflectively, and trying to find more honest ways of accomplishing whatever my white-lie goal was.
Most people in my social circle understand and accept glomarization as something you do sometimes.
I am still confused about whether/when to lie to bureaucracies. I do sometimes lie to bureaucracies. I am particularly worried about it when it seems likely other people will get caught up in the lies, but bureaucracies are often designed in a way to disempower me and I don’t feel automatically obligated to play by their rules
Honesty oaths
Sometimes, I think someone is maybe lying to me (usually in a fun prankstery way, sometimes more importantly). Sometimes, when it’s actually pretty important to me to know if it’s a lie, I ask “do you swear that’s true upon your honor as a guy who cares about being able to credibly coordinate about things truthfully sometimes?”
and sometimes they say “yes” and sometimes they say “no, because I have a general policy of not being pressured by that sort of question” and sometimes they say “hmm, I’m not sure whether I should glommarize here.”
sometimes, I instead say “do you swear on your honor as a guy who cares about credibly coordinating but also cares about fun pranks and gets, a few free passes on lying in answer to this sorta question?”
Mostly this has just come up for fun, but I like it as social tech.
“Honest relationships”, vs “Other random kinds of relationships”
Generally, I want to have honest relationships with my close friends. But, there’s a cluster of people (usually but not always non-rationalists, old friends, family, etc) who seem like they just don’t actually want a relationship with rationalist-levels-of-honesty. If I were fully honest with them they’d be annoyed or sad, and it doesn’t seem like this even bothers them. Mostly I still don’t lie to them, I just am not as open and don’t correct all inaccurate assumptions, but I lie sometimes.
Neat.
I think I have basically been trying to be meta-honest since the meta-honesty post came out… but, like, not trying super scrupulous committed about it, just, like, keeping it as the obviously-correct-thing to be aspiring to, and being more intentional about when/why to lie.
I didn’t find that degree of effort super costly. (I think “specifically tracking meta-honesty” basically just didn’t come up as a thing I had to do, because indeed people mostly don’t ask questions about when I’m honest)
(You said there was a lot of mental overhead, and I’m not sure if this was more about being “meta-honest” or just “honest”?)
I do think some benefits accrue, not just to you, but, to the rationalists-and-associates as a whole for taking honesty seriously. I don’t think the rest of the world cares about “meta-honesty” as an intellectual concept, but, it’s a true fact that if you’re the sort of person who takes honesty seriously you need to somehow handle the sorts of problems meta-honesty is designed to handle, so it’s sort of part-and-parcel to accurately gaining a reputation for serious honesty.
Re-rolling environment
Seems reasonable split, although I try to gesture at / share compressed versions of the background knowledge.
Yeah, I asked for this split precisely because, usually with a LessWrong post I already have at least the gist of the background knowledge, and what I really want to know is “what is the new stuff here?”.
But yeah I like the dream of “keep track of the stuff you know, and explain the diff between what you know.” But I think for the immediate future, being able to see at a glance “okay, what background context might I not have that, if I’m lost, I might want to read up on separately?”
I feel like I want here is a campaign to make sure history remembers the specific people who let this happen – names of board members, Attorney General Kathy Jennings, etc.
It feels achievable and correct to me for this to be a thing where, if you’re going to do this, a lot of people associate your name with enabling the theft.
Do you have own off-the-cuff guesses about how you’d tackle the short feedbackloops problem?
Also, is it more like we don’t know how to do short feedbackloops, or more like we don’t even know how to do long/expensive loops?
(Putting the previous Wei Dai answer to What are the open problems in Human Rationality? for easy reference, which seemed like it might contain relevant stuff)
I think I meant a more practical / next-steps-generating answer.
I don’t think “academia is corrupted” is a bottleneck for a rationalist Get Gud At Philosophy project. We can just route around academia.
The sorts of things I was imagining might be things like “figure out how to teach a particular skill” (or “identify particular skills that need teaching”, or “figure out how test whether someone has a particular skill), or “solve some particular unsolved conceptual problem(s) that you expect to unlock much easier progress.”
In your mind what are the biggest bottlenecks/issues in “making fast, philosophically competent alignment researchers?”
In this case I spot checked a few random strings from it.
For my personal browsing AI prompt-library-tool I use, it has the ability to click on a highlight, and scroll to that corresponding paragraph. It fails to work if there are any errors (although usually the errors are just “slightly different punctuation”), so it’s actually pretty easy to click through and verify.
But if I were building this into an reliably tool I wanted to use at scale, I’d follow it up with a dumb script that checks if the entire paragraphs match, and if not, if there are random subparts that match a given paragraph from the original content, and then reconcile them. (the sort of thing I’m imagining here is a thing that generates interesting highlights from LessWrong posts and some scaffolding for figuring out if you need to read prerequisites)
Just edit into this one :)