I currently work for Palisade Research as a generalist and strategy director and for the Survival and Flourishing Fund, as a grant round facilitator.
I’ve been personally and professional involved with the rationality and x-risk mitigation communities since 2015, most notably working at CFAR from 2016 to 2021 as an instructor and curriculum developer. I’ve also done contract work for MIRI, Lightcone, BERI, the Atlas fellowship, etc.
I’m the single person in the world that has done the most development work on the Double Crux technique, and have explored other frameworks for epistemically resolving disagreements and bridging ontologies.
Even as I’m no longer professionally focused on rationality training, I continue to invest in my personal practice of adaptive rationality, developing and training techniques for learning and absorbing key lessons faster than reality forces me to.
My personal website is elityre.com.
Eli Tyre
Ideating: What if authors could opt into Crocker’s rules for particular posts? With a tag or or something?
Or maybe a “Crocker’s rules” tag and a “butterfly idea” tag, with the default being something between those two extremes?
I think meaning is more complicated than you make it out to be here, but not in way that changes the fundamental analysis.
One obvious elephant: how much time should you spend becoming physically attractive if you’re not above average height?
I’m between 5′3“ and 5′4”, so this is a very pertinent question for me. (For context, that’s around second percentile for male height in the US. Most women prefer men who are taller than them, and the average woman in the US is 5′5″. This is a pretty brutal constraint on top of my otherwise pretty solid dating fundamentals.)
That said, at least some women definitely find me physically attractive. I can tell because they’ve either told me so, or told me that they would have sex / are interested in having sex with me.
Those women have ever commented approvingly of my muscles (I’m notably muscular, with a lean build, when my shirt is off), my hairstyle, and my manner of dress. I think that all of those are secondary to my demeanor—socially confident and self-assured, emotionally attuned, fun / funny etc. But I think my appearance is totally a relevant factor, even despite being very short for a man.
I think that wearing clothes that fit you well, and finding a hairstyle that women think looks good on you, are cheap up front costs that often makes a big difference. If you have acne, I think solving that is also a big boost (though I didn’t have that problem and so don’t speak from personal experience). YMMV.
You guys started a new account for me when LessWrong 2.0 started.
No? If there’s a false view that Bob and Carol both hold, and you claim it was knowably false at the time, it seems pretty reasonable to complain about Carol holding the view, regardless of what Bob thought?
It would be unfair to let Bob off the hook and only complain about Carol. But if you happen to know Carol better than Bob, then it’s pretty reasonable to mostly focus on talking to and about Bob?
can you operationalize this more so we can bet on it?
Sure!
In the years before shortforms were on the front page, Eli published more LessWrong posts (on average) than the years when shortforms were on the front page.
This is an unfair operationalization, perhaps, but it does hew directly to the evidence that makes me believe the claim!
This isn’t an end-to-end example, but sometimes there’s a discussion in the comments of someone’s post, and a very related comment on a short form. I can make a new comment that links to the old comment, but sometimes what I actually want is to drop the old comment (along with all of it’s associated children) into the new discussion.
Like, the comment should exist in two places on LessWrong, and any comments are visible in both places.
Then there’s one unified conversation, on that narrow point, instead of two related conversations.
I’m not strongly claiming that this is overall a good idea.
As a counterpoint, I think the rise of shortform on LessWrong has partially led to more stuff being written and partially led to more stuff that would have been posts written in a way that is less searchable, less linkable (in practice), less polished, and generally less supportive of ongoing discourse.
I have definitely published fewer posts since the shortform feed started up on the front page.
The more more we raise profile of comments the more they’ll displace posts as the medium of communication.
Great work.
This clearly articulates the horror converting the universe into hedonium.
For whatever it’s worth, this story doesn’t really sway my intuition that a universe of optimized experience-having-material experiencing the literally best and most meaningful possible experience, continuously, forever, is clearly better than a universe of quadrillions of specific people having specific wonderful experiences.[1] The first thing just has so much more area under the curve.[2]
But I already thought it was unethical to convert existing people into stuff that’s-better-by-utilitarian-logic by force. Unironically, you shouldn’t convert people into hedonium because that would violate their property rights. You don’t get to decide what to do with other people’s stuff, even if you would do something better with it than they would.- ^
I think many folks think that that “having the most meaningful experience possible, continuously”, is non-sensical, because that’s not how meaning works. I’m not so sure.
- ^
I emphasize “intuition” here, because, I don’t know how the universe should be spent. This is really not the kind of question that I should try to answer with my monkey-brain, on ancient earth, before the invention of dozens of fields of study that are as fundamental to understanding the universe as say, economics, unless I’m forced to. In some sense, this is the most important question in the universe, but it’s also a question we should collectively punt on, if we can.
- ^
I have sometimes wanted to “retweet” lesswrong comments, or alias them so that they can show up, in full, in multiple places on the site.
I have the feeling that this post is getting close to a critical point, but then doesn’t quite express it.
(At the moment, I can’t quite express it myself, so I can’t complain much.)
I think I want an essay that is really making the case for this:Here’s another danger, which I think may be worse. If you insist on working somewhere x-risk-themed, you’re asking for someone to make you a sucker.
Musing a bit on it...
There’s something like, “the desire to help” / “the desire to be important” is an attack surface.
A lot of people want to help, but figuring out how to help is actually really hard to do. Many things to try are worthless, many more are actively harmful. Strategic thinking is hard and often annoying, and is just the kind of thing that even pretty smart people aren’t cut out for.
A lot of people (including myself) try to slot themselves into roles where they can be a force-multiplier for some strategic vision, that makes sense to them, but which they couldn’t really defend from incisive critique.
I think a lot of EA-ish people treat this as a kind of neutral operation, where they’re straightforwardly glad to have an opportunity to have impact, rather than a kind of fraught transaction where one party is offering their agency and another is offering their strategic orientation / impact opportunity.When this works well, both parties get to work together to do more good in the world than either party could alone. Which is great!
But this transaction is fundamentally one that presupposes an information asymmetry. In order for this transaction to make any sense, the party offering their agency has to have less strategic discernment than the party offering strategic orientation.
So this setup is ripe ground for scams.
I suppose the general pattern is “if someone really wants something, X, but they have a weak ability to discern if they are achieving it or not, there is an incentive gradient towards scams that make them feel like they’re getting X.”
I think we probably get multiple tries with #1 and #2. Probably there some “first critical tries” with #3.
Astroids
I imagine that if we send a mission to deflect or destroy the astroid, and that mission fails, can we send up another mission attempting the same plan, or a new plan, based what we learned from the failure with the last one?
If our failure to deflect the astroid precludes any other attempts (because we only have time for one mission before impact, or because a failed astroid destruction will break it into millions of medium-sized which are collectively still deadly to earth, but now impossible to deflect, or something), then I would say there’s a “first critical try” involved.Global warming
Very clearly, we can try a bunch of different stuff to address global climate change, and if any given one of them doesn’t work, we’ll try other stuff?
I guess with some possible irreversible geo-engineering projects, there might be first critical tries, where if we mess it up we can’t reverse the impacts? But that seems like the exception rather than the rule.
Human genetic engineering
Genetically engineering humans seems very likely to have “first critical try problems”, especially if deployed at scale, because after one generation, your genetically engineered humans will be (partially) steering the genetic engineering process.
eg If you accidentally make a generation of humans that is extra docile or extra aggressive or unusually sociopathic, or whatever, in addition to more generally intelligent, those genetically engineered people are likely to prefer that future people be more like them. If they’re disproportionately intelligent, they’re going to disproportionately steering society via a wide mundane mechanisms (like voting and coming into leadership positions), and also by directly driving the priorities of the genetic engineering programs. Your mistake will likely reverberate through the whole human lineage for the rest of time, with limited ability to correct after the fact.There are also sub problems in the space of “human genetic engineering” that don’t have the “first critical try nature. Genetically engineering just one guy seem unlikely to be unrecoverable?
In each of the above cases, extinction is irreversible, so that’s another sense in which we only get “one-shot” to solve them. But irreversibility is a very weak condition since it applies to basically all extinction scenarios. If all we mean by saying that AI risk is a “one-shot” extinction problem is that the outcome is irreversible, then that label gives us no extra information beyond simply saying that it’s an extinction problem. Calling it “one-shot” is redundant.
Extinction itself is irreversible. But within the context of any given extinction threat, you can try various interventions. If those interventions, when they fail, have irreversible or unrecoverable effects, that are unacceptably bad, then those interventions have a “first critical try” problem.
FWIW, all this sounds right to me, and (I think?) sounds more right to me than it did when this comment was first written.
You should make a listicle of techniques for getting value out of coding agents.
I think the upvotes are completely justified?
I’m upvoting it because it’s ~ the most legible evidence to date, on one of the top ten most important questions in the world.
I’m not sure exactly how I’d define a coup but I’d say it has to be clear cut enough that, “Was it really an attempt at a coup?” is not really in contention in the aftermath.
I think this is a tricky standard, because many maybe-coups will only be widely regarded as coups depending on who won? (I think the January 6 maybe-coup would be contentious regardless, but for others.)
I like and basically endorse your cartoon model!
Modulo, I think that more of the capabilities are coming from the RLVF than from copying humans than you seem to think.[1]They get that almost entirely by copying from HPI represented in training data. They get some additional PI from various sources (RLVR, and just the pretraining itself (e.g. gippities know a bunch of things about the distribution of human text that no human knows), and from online reasoning (though of course there’s a memory problem, but that’s inessential)).
Why are you emphasizing the pretraining instead of the RL?
- ^
Though Zack dropped a paper in this thread which looks relevant to that question.
- ^
you’re also saying “we have fluid int or are close to it probably”.
I think I’m saying “crystalized intelligence can, to a large extent, substitute for fluid intelligence”. This is true to an extent, of humans, but it’s much more true of AIs, because they can have so much more crystalized intelligence than any human could hope to attain.
This is relevant to modeling if LLM agents will transform the world and to modeling if LLM agents will rapidly give way to something that’s much more capable.In particular, I (unconfidently) dispute that developing an AI with fluid intelligence is a research project that is itself heavily and crucially loaded on fluid intelligence.[1]
On my view, it is pretty likely that huge amounts of superhuman crystalized intelligence can find fluid-intelligence-emulating mechanisms (possibly with a necessary ingredient of a relatively small amount of genius fluid intelligence that even huge amounts of crystalized intelligence can’t substitute for, or possibly even without that input).
In that sense, we’re “close to” solving fluid intelligence, even if there’s decades of subjective research an iteration time between here and there.
I do additionally suspect that mechanisms to implement fluid intelligence are just not that hard to invent and/or scale to, starting from the AI tech of 2026. Like, it seems somewhat likely to me that that various dumb ideas would just totally work, or that doing the same stuff we’ve already been doing, but more so will totally work. However, I’m much less confident about this point, and perhaps you can teach me some things that would quickly cause me to change my mind.
I want to check if this comment is clarifying, or if it feel like me repeating things that I’ve already said.- ^
Though you flagged that those aren’t the words that you would use, so
- ^
I think this is basically correct, modulo that a lot of this is just that people differ in their intelligence and epistemological skill, and so there are classes of dispute that are basically obvious to, for instance, you, while to many earnest but less capable truth seekers, it’s in fact not easy to say who’s right.