I think it’s well understood by the people around who want an international treaty that it isn’t a stable end state
My impression of the common narrative is that nation states agreeing to limit training run sizes is presented as a kind of holy grail achieved through the very arduous journey of trying to solve a difficult global coordination problem. It’s where the answer to “well, what should be done?” terminates.
I heard “stop the training runs”, but not “stop new algorithms”, or “collective roll back to 22nm lithography”.
This is why they advocate for a crash program in adult human intelligence enhancement—to very rapidly make people are smart enough to get alignment right on the first try, before the international regime breaks down.
Further, only other detailed, written, plan that I’m aware of, explicitly expects to be able to maintain the international capability limiting regime for only about one decade, after which the plan is to handoff to trusted AIs. (I’m not citing that one since it’s not published yet.)
I’m not personally aware of anyone that thinks that an international ban or slowdown is a permanent equilibrium.
The AI alignment problem does not look to us like it is fundamentally unsolvable.
I wonder what the basis for this belief is? Rice’ theorem suggests that there is no general algorithm for predicting semantic properties in programs, and that the only way to know what it does is to actually run it.
My impression of the common narrative is that nation states agreeing to limit training run sizes is presented as a kind of holy grail achieved through the very arduous journey of trying to solve a difficult global coordination problem. It’s where the answer to “well, what should be done?” terminates.
I heard “stop the training runs”, but not “stop new algorithms”, or “collective roll back to 22nm lithography”.
From the online resources of IABIED:
This is why they advocate for a crash program in adult human intelligence enhancement—to very rapidly make people are smart enough to get alignment right on the first try, before the international regime breaks down.
Further, only other detailed, written, plan that I’m aware of, explicitly expects to be able to maintain the international capability limiting regime for only about one decade, after which the plan is to handoff to trusted AIs. (I’m not citing that one since it’s not published yet.)
I’m not personally aware of anyone that thinks that an international ban or slowdown is a permanent equilibrium.
In the link,
I wonder what the basis for this belief is? Rice’ theorem suggests that there is no general algorithm for predicting semantic properties in programs, and that the only way to know what it does is to actually run it.