I wonder how much it depends on the details of the world state when alignment happens, and how alignment happens? I’ve played with poking Claude into simulating the first ten years after the Slowdown ending of AI 2027. But it just seems like there’s so much to model, though! What are the actual bottlenecks to various things?
AI 2027 sort of handwaves things like brain uploading or nanobots being invented, but, if you’re trying to worldbuild a successful scenario, I wonder how much the details of these things matter? There’s such a kitchen sink of “new things are invented” that it gets very confusing.
Similarly, who holds power matters; is a world where humans are alive but under the grip of a locked-in Oversight Committee one worth calling a utopia?
And the details of alignment itself matter; alignment to who? Via what world model? Under which system of metavalues or CEV or whatever? And so on.
Probably this has been examined in detail in lots of other posts, and I just need to put in the work (or have Claude put in the work) of reading it all and synthesizing it. It seems like such a large project, though...
I’ve played with poking Claude into simulating the first ten years after the Slowdown ending of AI 2027. But it just seems like there’s so much to model, though! What are the actual bottlenecks to various things?
Neat! I’d be curious to hear your takeaways if you have any.
I wonder how much it depends on the details of the world state when alignment happens, and how alignment happens? I’ve played with poking Claude into simulating the first ten years after the Slowdown ending of AI 2027. But it just seems like there’s so much to model, though! What are the actual bottlenecks to various things?
AI 2027 sort of handwaves things like brain uploading or nanobots being invented, but, if you’re trying to worldbuild a successful scenario, I wonder how much the details of these things matter? There’s such a kitchen sink of “new things are invented” that it gets very confusing.
Similarly, who holds power matters; is a world where humans are alive but under the grip of a locked-in Oversight Committee one worth calling a utopia?
And the details of alignment itself matter; alignment to who? Via what world model? Under which system of metavalues or CEV or whatever? And so on.
Probably this has been examined in detail in lots of other posts, and I just need to put in the work (or have Claude put in the work) of reading it all and synthesizing it. It seems like such a large project, though...
Neat! I’d be curious to hear your takeaways if you have any.