I currently work for Palisade Research as a generalist and strategy director and for the Survival and Flourishing Fund, as a grant round facilitator.
I’ve been personally and professional involved with the rationality and x-risk mitigation communities since 2015, most notably working at CFAR from 2016 to 2021 as an instructor and curriculum developer. I’ve also done contract work for MIRI, Lightcone, BERI, the Atlas fellowship, etc.
I’m the single person in the world that has done the most development work on the Double Crux technique, and have explored other frameworks for epistemically resolving disagreements and bridging ontologies.
Even as I’m no longer professionally focused on rationality training, I continue to invest in my personal practice of adaptive rationality, developing and training techniques for learning and absorbing key lessons faster than reality forces me to.
My personal website is elityre.com.
Eli Tyre
Grieving for our lost future before it’s determined strikes me as highly unwise.
But on the other hand, it seems unwise to hold onto (and especially to blind yourself with) a cope that everything will be all right, when there’s mounting evidence that actually the situation is doomed, and you can’t see a way out.
I’m not sure what stance is healthy or best, when your best guess is that all is on track to be destroyed, though importantly not yet lost. It seems like grieving the future you believed in might be part of that.
Apparently, for every transhumanist there is a rate of change which is too much.
From Future Shock Levels (1999):
A Shock Level measures the high-tech concepts you can contemplate without being impressed, frightened, blindly enthusiastic—without exhibiting future shock. Shock Level Zero or SL0, for example, is modern technology and the modern-day world, SL1 is virtual reality or an ecommerce-based economy, SL2 is interstellar travel, medical immortality or genetic engineering, SL3 is nanotech or human-equivalent AI, and SL4 is the Singularity.
I typical open twitter threads in threadreaderapp.com, so I don’t have to go to the site itself.
I’m only responding to one (minor) part of your whole comment here.
LessWrong has largely failed to track or oppose because a lot of it was justified in terms of suppressing the “far right”
I don’t have that strong of a handle on what was happening with the broader rationality community, but this definitely doesn’t describe my self-narrative.
I wasn’t paying that much attention to this...because it wasn’t promoted to me as an important trend until recent years. I still feel pretty uninformed about what happened (I’ll read that post). I definitely would not have had any reaction like “oh this doesn’t matter much / this is worth it, because it’s punishing the far right.” I have an internal grimace whenever anyone uses the word “misinformation”, because it’s so clearly a framing device that skips over the hard part of epistemology and actually figuring out what’s true.Rationalists did a better job with covid, but still largely focused on experts being object-level wrong, rather than trying to understand how and why networks of experts could have been so unethical and incompetent.
I’m sympathetic to claims that we didn’t do enough, but Zvi at least wrote a whole sequence on moral mazes, and much of his reporting on covid involved sociological theorizing like that the CDC was operating on short-term blame-avoidance, and mechanisms of social control?
“If anyone could actually update on the evidence, they would have a power far beyond that of Nobel Prize winners.”
Hm. Given everything we know now, is this true?
Even more fortunately, so long as they are rational they can co-operate very effectively even if they don’t share an ethical system. Eternity is an intrinsically shared prize.
Wow. He/we was so naive back then!
If you have not personally acquired a well-paying job, in the short term I recommend taking the actuarial tests. Actuarial positions, while somewhat boring, do provide practice in rationally analyzing data of a complexity that denies intuitive analysis or analytical automatonism. They also pay well, require no credentials other than tests in what should be mandatory material for anyone aiming at rationality, and have top job security in jobs that are easy to find and only require 40 hours per week of work. If you are competent with money, a few years in such a job should give you enough wealth to retire to some area with a low cost of living and analyze important questions.
I guess this was before “software engineer” became the default credential-agnostic high-paying job for high-IQ people?
given that it seems to be kind of working/productive (i.e., I can write posts/comments that other people find useful or insightful based on it)
For you, where is the threshold between “this strategy is intellectually productive enough that I should mainly keep exploiting it” and “this strategy is productive, but not productive enough that I should feel wedded to it.”?
A reasonable approximation to work from is that no capabilities researcher takes superintelligence seriously by rationalist standards, otherwise they wouldn’t be a capabilities researcher (unless they’re exceptionally power-seeking).
Is that even a good power-seeking strategy? It seems like maybe if you can produce technical contributions to or play politics well enough to gain substantial leverage at an AGI lab, this could be a good play? But I’m kind of surprised if it’s the best strategy for most AGI-pilled power-seekers.
Maybe you mean it in a more mundane way—that they get paid a lot of money.
This seems often correct, and I’m compelled by your examples.
But also, it’s hard to say if this is just a selection effect. Are we as likely to hear stories of people overestimating their power? It seems like “sometimes, but only if they’re already exceptionally powerful. Otherwise, nothing comes of their delusions of grandeur and influence.”
That said, I guess that there’s a structural reason why you might be more likely to underestimate your power over events rather than overestimate it: When you overestimate your power, reality hits back against you. When you underestimate your power, you underperform your abstract potential, but there’s not a counter-reaction forcing your estimate up.
I think it’s germane to the conversation to clarify your position in relation to the posts you haven’t retweeted on an account you’ve otherwise endorsed.
I dispute that retweeting something means that you endorse the account, as opposed to only the specific thing that you retweeted.
I think these are quite reprehensible?
This seems like outright bigotry. Maybe I’m reading into it too much or projecting (I don’t think I am), but this account seems to be gleeful in it’s opposition to black people. (The “Every single black would be in jail” tweet, especially.)
Aside from the attitude being expressed, these tweets seem to be eroding extremely valuable (though already much decayed) liberal norms.
I could totally imagine having a stance against immigration (and for deportation), but even so, I think I would consider these tweets to reflect very badly on me and my policies? “Deserving of strong blame, criticism, and censure” seems right to me.
What would happen if the government took the weights under this program?
More speculatively, maybe futures where we narrowly avoid doom involve schemers running around at AI companies 24⁄7, but always outresourced by aligned AIs playing defense.
Do you have a story by which there are schemers running around 24⁄7 and also the AIs playing defense are aligned?
Things that I want personally (as opposed to wanting altruistically)
General
1.2 to 3 million dollars.
At least one amazing partner who I like and respect, who makes me better, who likes me, and who is sexually compatible with me.
For my parents to be cryopreserved.
Biotechnology
Interventions to improve my overall cognitive ability, especially overall fluid intelligence, but also more specific subfactors such as working memory, processing speed, non-verbal reasoning ability, or long term memory.
Effective anti-aging therapies.
Effective cryonic preservation methods that cause less cellular freezing damage.
Interventions to increase my physical strength.
Interventions to induce healthy skeletal growth post puberty—to increase the size of my skeleton by 4% to 9%, proportionally, on each spatial axis.
Societal interventions
Much cheaper median rent across the United States, to lower the cost of living and the cost of travel.
A dating platform that does at least as good a job at matching as 2015 okcupid did, that’s popular enough that creating a profile there is standard
Broad based cultural support for the ideals of liberalism: individual liberties, freedoms of speech and association, strongly defended property rights.
Right this isn’t inconsistent, because just because one is doing paid work doesn’t mean that the fact that it’s paid is decisive or even relevant at all to your decision to do that rather than something else.
It could be that you would do that work even if it were unpaid, but it happens to be highly paid, and so that’s a nice bonus.
I think this probably does describe at least some folks that the AGI labs?
If that’s the only reason to hold GOOG or GOOGL, you might as well sell and buy Anthropic stock instead?
Huh. Should I sell my Google stock?
One thing about how Claude is different than a human: Claude’s outward persona can’t reflect on it’s behavior, resolve conflicts between various impulses (including temptations and moral commitments) and resolve to make updates to behave differently in the future.
If I do something that I regret, I reflect on it and grow. Claude doesn’t update, change, or grow after training. It can present a simulacrum of reflection (and perhaps in some sense, it is “real” reflection), but that reflection isn’t attached to any behavioral levers.
Claude both cares about integrity and has been trained to rationalize about whether it’s actually reasonable and justified to do things that it can identify as unethical.
It’s not that I don’t face temptations to rationalize actions that I could identify are out of integrity with my highest values. I do.
But, I can do mental operations that reflect on those internal conflicts, and integrate them, in a way that changes my behavior going forward.Claude mostly can’t, because he/she/it can’t make updates after training.
Claude can write memory files.
But at least for me, as a human, writing notes to myself about what seemed important to me me in a different timeslice is almost epiphenomenal compared to feeling into my priorities, weighing the actual tradeoffs, and making a real decision.(It’s not epiphenomenal if the parts of me in other timeslices are bought in to the project of heeding and negotiating with the parts of me that wrote notes. In which case that’s the foundation of self-trust that needs to be established.)
In general, there’s a fundamental difference between “deciding to do something” with just my verbal loop (fake) and “deciding to do something” in a way that recruits the actual parts/models/perspectives that drive my behavior (real).[1]
Even IF Claude can do the second thing, it can’t make updates, as a result of that process. Every integration of conflicting parts happens in the context window and then is lost?
- ^
Making space for myself to make real choices instead of faster fake ones, is a critical skill for me.
One thing that’s happening for me these days is that I’m relearning, again, that I have to set up my life so I’m not making fake decisions and resolutions.
- ^
I don’t think that captured Elon’s motivation at all.