I wonder if creating a superintelligence requires applying a way to increase capabilities at the cost of alignment. At the time of writing, Anthropic’s ECI increases linearly with time and I suspect some constraints related to model sizes. Therefore, I don’t expect acceleration of capabilities progress in any way besides introducing new architectures undermining alignment techniques (e.g. neuralese, as in AI 2027) or severe scaling which poses its problems like models learning to reason without the CoT or by using a faked CoT.
P.S. Could you describe your proposed alignment approaches in more detail? Something like many more novel methods in mechinterp?
I wonder if creating a superintelligence requires applying a way to increase capabilities at the cost of alignment. At the time of writing, Anthropic’s ECI increases linearly with time and I suspect some constraints related to model sizes. Therefore, I don’t expect acceleration of capabilities progress in any way besides introducing new architectures undermining alignment techniques (e.g. neuralese, as in AI 2027) or severe scaling which poses its problems like models learning to reason without the CoT or by using a faked CoT.
P.S. Could you describe your proposed alignment approaches in more detail? Something like many more novel methods in mechinterp?
That seems false to me. If we had a textbook from the future on AI alignment, I think we would certainly be able to build an aligned ASI.