Huh, I knew there wasn’t the sort of plan you’d naively expect where the US gov/military command observes the response of the Japanese gov/military to one of their cities being destroyed by unthinkable godlike powers and then decides what to do next. I didn’t know that president Truman literally didn’t know about/have implicit preemptive control over the 2nd bombing.
David J Higgs
Karma: 23
Is there a reliable way to distinguish between [remembers more facts] and [infers more correct facts from remembered ones]? If there isn’t, then using remembered facts as an estimate of base model size would be even more noisy than you’d already expect.
I know I get far more questions right on exams than chance would predict when I have 0 direct knowledge/memory of the correct answer. I assume reasoning models have at least some of this kind of capability
I think with respect to utopia, especially via AI, mourning can make sense now, but not after it actually happens. Now you can see all the things that will never be, but you can’t see all the things that are even better that will actually be.
After you will see and feel and live all that is better, and it will be obvious that it is better. Only gratitude makes sense, or perhaps some form of nostalgia, but not genuine grief (for life without AI).
I’m glad this kind of content exists on LessWrong: writing that doesn’t shy away from an explicit focus on personal virtue, in a “how algorithms feel from the inside” kind of way. I used to devour anything I could find written by C.S. Lewis as a still-religious teenager, because I felt a certain quality of thought and feeling emanating from each page. I felt the sincerity of effort orienting towards truth, beauty, and goodness.
Unfortunately, much of his worldview turned out to be importantly wrong, but his writings are hardly alone in that compared to other genuine historical truth seekers. I hope that like this post, my future thinking can manage to orient in that direction which Lewis was among the first to bring before my attention.
This strikes me as a good sort of constructive feedback, but one that didn’t apply in my case, and I’ll try to explain why. Thinking real instead of fake seems like a canonical example of rationality that is especially contingent upon emotions and subjective experience, and intervening on that level is extremely tricky and fraught.
In my case, the copious examples, explanations of why the examples are relevant, pointers to ways of avoiding the bad/getting at the good, etc. mostly seemed helpful in conveying the right pre-rational mental patterns for allowing the relevant rational thoughts to occur (either by getting out of the way or by participating in the generation of rational thought directly).
It was also simply enjoyable throughout, in a way that harkened back a little to when I would read C.S. Lewis as a still-religious teenager. Not imitation of Lewis’s writing, but rather pointing in the direction Lewis was trying to point (toward truth, beauty and goodness). This last element seems potentially load bearing, in that I don’t know whether I’d have found the continuous details helpful if I hadn’t found them intellectually pleasant in this particular way.
I’m guessing “species” is there mainly as emphasis that we are NOT talking about (mere) tool AI, and also maybe to marginally increase the clickbait for Twitter/X purposes.
Don’t forget intellectual charity, which might actually be the most LW distinguishing feature relative to other smart online communities.
Counter-counterpoint: big groups like bureaucracies are not composed of randomly selected individuals from their respective countries. I strongly doubt that say, 100 randomly selected Google employees (the largest plausible bureaucracy that might potentially develop AGI in the very near term future?) would answer extremely similarly to 100 randomly selected Americans.
Of course, in the only moderately near term or median future, something like a Manhatten Project for AI could produce an AGI. This would still not be identical to 100 random Americans, but averaging across the US security & intelligence apparatus, the current political facing portion of the US executive administration, and the leadership + relevant employee influence from a (mandatory?) collaboration of US frontier labs would be significantly closer on average. I think it would at least be closer to average Americans than a CCP Centralized AGI Project would be to average Chinese people, although I admit I’m not very knowledgeable on the gap between Chinese leadership and average Chinese people other than basics like (somewhat) widespread VPN usage.
If you haven’t already, you should consider reading the Timelines Forecast and Takeoff Forecast research supplements linked to on the AI 2027 website. But I think there are a good half dozen (not necessarily independent) reasons for thinking that if AI capabilities start to takeoff in short timeline futures, other parts of the overall economy/society aren’t likely to massively change nearly as quickly.
The jagged capabilities frontier in AI that already exists and will likely increase, Moravec’s Paradox, the internal model/external model gap, the lack of compute available for experimentation + training + synthetic data creation + deployment, the gap in ease of obtaining training data for tasks like Whole Brain Emulation versus software development & AI Research, the fact that diffusion/use of publicly available model capabilities is relatively slow for both reasons of human psychology & economic efficiency, etc.
Basically, the fact that the most pivotal moments of AI 2027 are written as occurring mostly within 2027, rather than say across 2029-3034, means that it’s possible for substantial RSI in terms of AI capabilities before substantial transformations occur in society overall. I think the most likely way AI 2027 is wrong on this matter is that not nearly as fast of an “intelligence explosion” occurs, not that the speed of societal impacts that occur simultaneously is underestimated. The reasons for thinking this are basically taking scaling seriously & priors (which are informed by things like the industrial revolution).
In addition to the option of spending effort on reducing the chance the world ends, one could also reframe from “leaving a mark on the world that outlives you” to “contributing to something bigger and beyond yourself.” The world is bigger than you, more important than you and exists outside of you right now, as well as up until the world ends (if/when it does).
Helping the world right now, and helping the world after you are gone, are morally equivalent, and quite possibly equivalent at the level of fundamental physics. I’m not sure what, other than a false sense of personal immortality (legacy as something beyond the actual beneficial effects on the world), is tied to benefiting the world later than your own time of existence. But perhaps that’s my own ignorance.