The presumption of complete reducibility is, with some great certainty, unclear at best and at worst absolutely impossible,
Oh, I fully agree. But “complete” is not necessary to achieve the change in categorization from “unknown and magical” to “just (big/difficult) math”.
I don’t know how much playing people around here have done with Mandelbrot set coding, but it’s a useful comparison in this. It’s very clearly NOT magic in any literal sense—the calculation is trivial (and even the iterations to determine converge/diverge is pretty easy to understand). But the results remain captivating and astounding (to me) that they come from such simple rules.
In this sense, I suspect many complex systems will remain impressive and astounding, no matter how good we get at modeling and understanding their components. In the sense that knowing the underlying rules DOES turn it from “fully magical” into “an interesting corner of math”, this will probably happen to current LLMs, and likely eventually to primate intelligence.
Oh, I fully agree. But “complete” is not necessary to achieve the change in categorization from “unknown and magical” to “just (big/difficult) math”.
I don’t know how much playing people around here have done with Mandelbrot set coding, but it’s a useful comparison in this. It’s very clearly NOT magic in any literal sense—the calculation is trivial (and even the iterations to determine converge/diverge is pretty easy to understand). But the results remain captivating and astounding (to me) that they come from such simple rules.
In this sense, I suspect many complex systems will remain impressive and astounding, no matter how good we get at modeling and understanding their components. In the sense that knowing the underlying rules DOES turn it from “fully magical” into “an interesting corner of math”, this will probably happen to current LLMs, and likely eventually to primate intelligence.