LessWrong team member / moderator. I’ve been a LessWrong organizer since 2011, with roughly equal focus on the cultural, practical and intellectual aspects of the community. My first project was creating the Secular Solstice and helping groups across the world run their own version of it. More recently I’ve been interested in improving my own epistemic standards and helping others to do so as well.
Raemon
Karma: 55,981
I do agree it’s suggestive, I’d be interested to see practical → different practical.
This sounds like it might be a Paper Trauma -ish thing, which might have a different specific mechanism.
I’m not sure if this is fiction. I realize there’s something nice about opening in media res but I think this could use better indication of who this is for or what it is.
Nod, but fwiw if you don’t have a cached answer, I am interested in you spending like 15 minutes thinking through whether there exist startup-centric approaches to helping with x-risk that are good.
Well yeah, but the question here is “what should be community guidelines on specifically how to approach startups that are aimed at specifically helping with AI safety” (which may or may not include AI), not “what kinds of AI startups should people start, if any?”
I didn’t track this previously: how did they incentive themselves to reach the cap?
Against strict kantian deontologists I admit no version of this argument could be persuasive and they’re free to bite the other bullet and fail to achieve any good outcomes.
Note that this is very different from what you said in your post, which is “sometimes you will lose.” (And this one seems obviously false)
I think I understood your article, and was describing which points/implications seemed important.
I think we probably agree on predictions for nearterm models (i.e. that including this training data makes it more likely for them to deceive), I just don’t think it matters very much if sub-human-intelligence AIs deceive.
FYI I think there are a set of cues that move you from ‘pretty unlikely to be interested’ to ‘maybe interested’, but not that get you above like 25% likely.
FYI this was an April Fools joke.
Curated. I’ve been following this project for awhile (you can see some of the earlier process in Daniel’s review of his “What 2026 looks like” post, and on his comment on Tom Davidon’s What a Compute-centric framework says about AI takeoff). I’ve participated in one of the wargames that helped inform what sort of non-obvious things might happen along the path of AI takeoff.
(disclosure, Lightcone did a lot of work on the website of this project, although I was only briefly involved)
Like others have said, I appreciate this for both having a lot of research behind it, and for laying out something concrete enough to visualize and disagree with. Debating individual “event X will happen” predictions isn’t exactly the point, since some of them are merely illustrative of “something similar that might happen.” But, it’s helpful for debating underlying models about what sort-of-events are likely to happen.
One of the central, obvious debates here is “does it actually make sense to just extrapolate the trends the way this way, or is AGI takeoff dependent on some unrelated progress?”. Recent posts like A Bear Case and Have LLMs Generated Novel Insights?[1] have argued the opposite view). I lean towards “the obvious trends will continue and the obvious AGI approaches will basically work”, but only put it at bit over 50%. I think it’s reasonable to have a lower credence there. But one thought I’ve had this week is: perhaps longer-time-folk (with some credence on this) should to spend the next year-or-so focusing more on plans that help in short-timeline worlds, and then return to longer time-horizon plans if a year from now, it seems like progress has slowed and there’s some missing sauce.[2]
I think it would have been nicer if a third scenario was presented – I think the current two-scenario setup comes across as more of a rhetorical device, i.e. “if y’all don’t change your actions you will end up on the doomy racing scenario.” I believe Daniel-et-al that that wasn’t their intent, but I think a third scenario that highlighted some orthogonal axis of concern would have been helpful for getting people into the mindset of actually “rolling the simulation forward” rather than picking and arguing for a side.
- ^
Notably, written before AI 2027 came out, although I think they were reacting to an intellectual scene that was nontrivially informed by earlier drafts of it.
- ^
On the other hand, if most of your probability-mass is on mediumish timelines, and you have a mainline plan you think you could barely pull off in 10 years, such that taking a year off seems likely to make the difference,
- ^
Yeah I introduced Baba is You more as a counterbalance to empiricism-leaning fields. I think ‘practice forms of thinking that don’t come natural to you’ is generally valuable so you don’t get in ruts.
I’m curious how it went in terms of ‘do you think you learned anything useful?’
I do think the thing you describe here is great. I think I hadn’t actually tried really leveraging the current zeitgeist to actively get better at it, and it does seem like a skill you could improve at and that seems cool.
But I’d bet it’s not what was happening for most people. I think the value-transfer is somewhat automatic, but most people won’t actually be attuned to it enough. (might be neat to operationalize some kind of bet about this, if you disagree).
I do think it’s plausible, if people put more deliberate effort it, to create a zeitgeist where the value transfer is more real for more people.
A thing that gave me creeping horror about the Ghiblification is that the I don’t think the masses actually particularly understand Ghibli. And the result is an uneven simulacrum-mask that gives the impression of “rendered with love and care” without actually being so.
The Ghibli aesthetic is historically pretty valuable to me, and in particular important as a counterbalanacing force against “the things I expect to happen by default with AI.”
Some things I like about Ghibli:
The “cinematic lens” emphasizes a kind of “see everything with wonder and reverence” but not in a way that papers over ugly or bad things. Ugliness and even horror are somehow straightforwardly depicted, but in a way that somehow makes both seem very normal and down to earth, and also supernaturally majestic. (See On green, and The Expanding Moral Cinematic Universe).
The main characters are generally “low-ish neuroticism.” (This youtube analysis I like argues that the women in particular are ‘non-neurotic’, and the men tend to be “low compared to modern city-dwelling standards.”)
There’s a bit of awkwardness where Miyazaki is particularly anti-transhumanist, where I disagree with him. But I feel like I could argue with him about it on his terms – I have an easy time imagining how to depict spirits of technology and capitalism and bureaucracy as supernatural forces that have the kind of alien grandeur, not on humanity’s side or the “natural world’s side”, but still ultimately part of the world.
For years, I have sometimes walked down the street and metaphorically put on “Miyazaki goggles”, where I choose to lean into a feeling of tranquility, and I choose to see everything through that “normal but reverent” stance. I imagine the people that live in each house doing their day to day things to survive and make money and live life. And seeing the slightly broken down things (a deteriorating fence, a crumbling sidewalk) as part of a natural ebb and flow of the local ecosystem. And seeing occasional more “naturally epic” things as particularly majestic and important.
So, the wave of “ghiblify everything” was something I appreciated, and renewed a felt-desire to live more often in a ghibli-ish world. But, also, when I imagine how this naturally plays out, I don’t think it really gets us anything like a persistent reality transfer the way you describe. Mostly we get a cheap simulacra that may create some emotion / meaning at first, but will quickly fade into “oh, here’s another cheap filter.”
...
That all said, I do feel some intrigue at your concept here. I’m still generally wrapping my mind around what futures are plausible, and then desirable. I feel like I will have more to say about this after thinking more.
Seems at odds with longhairism.
Curated. I think this is a pretty important point. I appreciate Neel’s willigness to use himself as an example.
I do think this leaves us with the important followup questions of “okay, but, how actually DO we evaluate strategic takes?”. A lot of people who are in a position to have demonstrated some kind of strategic awareness are people who are also some kind of “player” on the gameboard with an agenda, which means you can’t necessarily take their statements at face value as an epistemic claim.
I think I agree with a lot of stuff here but don’t find this post itself particularly compelling for the point.
I also don’t think “be virtuous” is really sufficient to know “what to actually do.” It matters a lot which virtues. Like I think environmentalism’s problems wasn’t “insufficiently virtue-ethics oriented”, it’s problem was that it didn’t have some particular virtues that were important.
Or: when the current policy stops making sense, we can figure out a new policy.
In particular, when the current policy stops making sense, AI moderation tools may also be more powerful and can enable a wider range of policies.
Curated. This post’s framing resonated a lot with my own framing. I think the questions of how to cultivate impact, agency and taste are some of the more important questions that LessWrong tackles.
Much of this post were phenomena I’d observed myself, but, a few particular framings stood out to me as helpful crystallizations:
The first was “Don’t rely too much on permission or encouragement.” I think a few Lightcone employees have also been slowly learning something along these lines. Our CEO has a lot of taste and vision, but sometimes one of us comes up with an idea that doesn’t immediately resonate with him, and it’s not until we actually build some kind of prototype ourselves that other people start to believe in it.
Another was:
I had heard this sort of idea before, but this was the first time I parsed it as a technique you could explore on purpose. (i.e. actively investigate what things people seem mysteriously bad at, and use that to guide where you can maybe trust your taste more).
Finally:
Recently I’ve been exploring the “think real hard about things” paradigm. This paragraph helped flesh out for me that there are some prerequisites for “think real hard” to work. I think some of this are skills that work across domains (i.e. “notice when you can’t actually explain something very clearly” → “you’re still confused, try to break the confusion down”). But, it makes sense that some of it is domain specific.
There’s an important question I previously would have framed: “If you’re tackling an unfamiliar domain, the question is ‘how much mileage can you get from general, cross-domain reasoning skills?”. But, a different question is “what’s the minimum amount of ‘domain specific’ skill you need in order for ‘think about it in advance’ to really help?”.