Responding to some of the overarching evolution ideas.
A while back I went down the rabbit hole of “how life started” and how this incredible evolutionary process, in feats only describable as magic, was somehow able to turn dirt into conscious, questioning beings, which then might eventually evolve into “basically god”. Drawing a path all the way from the original LUCA. What an astounding phenomena of the universe.
Maybe what helped lift this naivety, and I think it was explained well by Nexus, is that “evolutionary” selection algorithms are one type of optimization tool/pattern. Comparatively, not even a very good one. Evolution’s “prime quality” is just that it is the only one which seems to occur naturally (in absence of design). Actually (again referencing Nexus), evolutionary algorithms are rather brutal by comparison: think predator / prey scenarios we see in nature.
Responding to the essay, I’m not sure there is good reason to believe evolution or evolutionary algorithms will play too much a role in future AI, at least not in the manner suggested. Software is already copied with perfect fidelity a gazzillion times a day and we’ve solved this control problem handily—cosmic radiation is not going to “flip bits” to cause evolution nor certainly would hardware churn. Even extending to model weights.
The other primary ingredient for evolution, aside from random deviation which we control for, is a selection function which also loses meaning when AI is temporal / short lived, etc.
So it seems more likely that development beyond this point is going to be intentional, and if we do eventually enter some new paradigm where model weights (or whatever substrate) can change dynamically and fluidly, this would still not imply evolution.
To be honest, you are not actually responding to ideas in this essay. That’s okay, just want to flag this.
I’m sorry this was your takeaway, but feel free to return to my OP for deeper reflection at any point. The general idea, which I tried to put sensitively, is you are the one “misunderstanding evolution” at a rather deep level.
And yes I saw your comment that was partly what I was replying to