Noise is certainly a problem, but the biggest problem for any sort of atomic modelling is that you quickly run into an n-body problem. Each one of of n electrons in an atom interacts with every other electron in that atom and so to describe the behavior of each electron you end up with a set of 70 something coupled differential equations. As a consequence, even if you just want a good approximation of the wavefunction, you have to search through a 3n dimensional Hilbert space and even with a preponderance of good experimental data there’s not really a good way to get around the curse of dimensionality.
Am I understanding the relevance of the curse of dimensionality to this correctly: Generally, our goal is to find a simple pattern in some high-dimensional data. However, due to the high dimensionality there are exponentially many possible data points and, practically, we can only observe a very small fraction of that, so curse is that we are often left with an immense list of candidates for the true pattern. All we can do is to limit this list of candidates with certain heuristic priors, for example that the true pattern is a smooth, compact manifold (that worked well e.g. for relativity and machine learning, but for example quantum mechanics looks more like that the true pattern is not smooth but consists of individual particles).
The true pattern (i.e. the many-particle wavefunction) is smooth. The issue is that the pattern depends on the positions of every electron in the atom. The variational principle gives us a measure of the goodness of the wavefunction, but it doesn’t give us a way to find consistent sets of positions. We have to rely on numerical methods to find self-consistent solutions for the set of differential equations, but it’s ludicrously expensive to try to sample the solution space given the dimensionality of that space.
It’s really difficult to solve large systems of coupled differential equations. You run into different issues depending on how you attempt to solve them. For most machine-learning type approaches, those issues manifest themselves via the curse of dimensionality.
Noise is certainly a problem, but the biggest problem for any sort of atomic modelling is that you quickly run into an n-body problem. Each one of of n electrons in an atom interacts with every other electron in that atom and so to describe the behavior of each electron you end up with a set of 70 something coupled differential equations. As a consequence, even if you just want a good approximation of the wavefunction, you have to search through a 3n dimensional Hilbert space and even with a preponderance of good experimental data there’s not really a good way to get around the curse of dimensionality.
Am I understanding the relevance of the curse of dimensionality to this correctly: Generally, our goal is to find a simple pattern in some high-dimensional data. However, due to the high dimensionality there are exponentially many possible data points and, practically, we can only observe a very small fraction of that, so curse is that we are often left with an immense list of candidates for the true pattern. All we can do is to limit this list of candidates with certain heuristic priors, for example that the true pattern is a smooth, compact manifold (that worked well e.g. for relativity and machine learning, but for example quantum mechanics looks more like that the true pattern is not smooth but consists of individual particles).
The true pattern (i.e. the many-particle wavefunction) is smooth. The issue is that the pattern depends on the positions of every electron in the atom. The variational principle gives us a measure of the goodness of the wavefunction, but it doesn’t give us a way to find consistent sets of positions. We have to rely on numerical methods to find self-consistent solutions for the set of differential equations, but it’s ludicrously expensive to try to sample the solution space given the dimensionality of that space.
It’s really difficult to solve large systems of coupled differential equations. You run into different issues depending on how you attempt to solve them. For most machine-learning type approaches, those issues manifest themselves via the curse of dimensionality.