I’m not getting your 35% to 5% reference? I just have no hope of getting as low as 5%, but a lot of hope for improving on just letting the labs take a swing.
i.e, if basically anything other than a long pause will be insufficient to actually work, you might as well swing for the pause.
It drives me a bit nuts that many of our otherwise best thinkers are confident of aligniing LLM AGI being almost impossible, when the arguments they’re citing just don’t stack to near certainty even with active steelmanning. I’ve been immersing myself in the arguments for inner misalignment as a strong default. They’re strong, they should make you afraid, but they’re nowhere near certainty.
Few people who take that aligment difficulty seriously are even proposing ways around it for LLM AGI. We have barely begun working on the most relevant hard problem. Calling it hopeless without working on it is… questionable. I see why you might, for various reasons, but at this point I think it’s a huge mistake.
We can call for pause/slowdown and emphasize the difficulty, while also working on alignment on the default path. We’re in a bad situation, and that looks to me like our biggest potential out by an order of magnitude.
i.e, if basically anything other than a long pause will be insufficient to actually work, you might as well swing for the pause.
Sure. If you’re confident of that.
It drives me a bit nuts that many of our otherwise best thinkers are confident of aligniing LLM AGI being almost impossible, when the arguments they’re citing just don’t stack to near certainty even with active steelmanning. I’ve been immersing myself in the arguments for inner misalignment as a strong default. They’re strong, they should make you afraid, but they’re nowhere near certainty.
Few people who take that aligment difficulty seriously are even proposing ways around it for LLM AGI. We have barely begun working on the most relevant hard problem. Calling it hopeless without working on it is… questionable. I see why you might, for various reasons, but at this point I think it’s a huge mistake.
We can call for pause/slowdown and emphasize the difficulty, while also working on alignment on the default path. We’re in a bad situation, and that looks to me like our biggest potential out by an order of magnitude.