I agree with all of these, but most of these problems would be equally bad whether was or rather than . What I’m disputing is the argument from “Look at this large number of things which we cannot individually measure”.
(I was tempted to say that disappearing polymorphs are an example, because if we had times as many atoms, then it would take much less time for the first crystal of the stable state to appear, but then we just wouldn’t have discovered the meta-stable state of that particular molecule at all. Instead, in world we’d run into problems with a different polymorph which would take years to disappear in our universe.)
Linking micro models to bulk dynamics has been intellectually fruitful in many fields.
However, there seem to be vast differences between fields in terms of what dynamics are of interest and how predictable they are.
Model weights can in fact be individually measured and combinations studied and examined in isolation. That‘s the trivial counter to the argument as your follow-up comment presents it.
An advantage unique to AI interpretability is the tractability of experimentation. As long as we stay in the domain of bits, we can in principle arbitrarily tweak inputs and weights at a low cost per data point. Other fields work in the domain of atoms and don’t have nearly the same level of freedom.
Nevertheless, the point remains that some points of interest scale in prediction difficulty with the size of the system. I think the concern is really that what we most want to predict—large AI model behaviors in the world of atoms with recursive inputs, where each AI, context and harness differs—seems likely to become harder to predict with increased model size, extended time horizons and a wider range of contexts.
Chemistry is a potentially misleading analogy, because when we study it or implement industrial chemical methods, we do our best to rigorously control the environment in which it’s applied because what we want is a specific, monotonous, consistent product. AI is different. We want adaptability to diverse contexts and the ability to satisfy multiple, often competing objectives. So the interest is different.
I agree with all of these, but most of these problems would be equally bad whether
(I was tempted to say that disappearing polymorphs are an example, because if we had
Linking micro models to bulk dynamics has been intellectually fruitful in many fields.
However, there seem to be vast differences between fields in terms of what dynamics are of interest and how predictable they are.
Model weights can in fact be individually measured and combinations studied and examined in isolation. That‘s the trivial counter to the argument as your follow-up comment presents it.
An advantage unique to AI interpretability is the tractability of experimentation. As long as we stay in the domain of bits, we can in principle arbitrarily tweak inputs and weights at a low cost per data point. Other fields work in the domain of atoms and don’t have nearly the same level of freedom.
Nevertheless, the point remains that some points of interest scale in prediction difficulty with the size of the system. I think the concern is really that what we most want to predict—large AI model behaviors in the world of atoms with recursive inputs, where each AI, context and harness differs—seems likely to become harder to predict with increased model size, extended time horizons and a wider range of contexts.
Chemistry is a potentially misleading analogy, because when we study it or implement industrial chemical methods, we do our best to rigorously control the environment in which it’s applied because what we want is a specific, monotonous, consistent product. AI is different. We want adaptability to diverse contexts and the ability to satisfy multiple, often competing objectives. So the interest is different.