“This is the real point of the cell boundary, not “protection from the environment”—it keeps the fruits of chemical labor inside a spatial boundary.”
What basis does Yudkowsky have for this claim?
For example, the speed of biological/chemical processes is related to diffusion, and the speed of diffusion is related to the size of cells. By restricting diffusion to a relatively small chunk of water, you can get much faster reactions. If this reason is the dominant “why” of the evolution of cells, then cells would sound more like an object-level innovation.
I suspect that Yudkowsky is ascribing more knowledge to the researchers into early life than the researchers actually have; most disciplines look more impressive from a distance than up close.
It seems to me that there’s an aspect to this that isn’t getting much attention: the domain.
Example domains include chess and Go, certainly. But probabilistic games surely should not be excluded. There is a spectrum of domains which go from “fair roulette” (which is not manipulatable by intelligence), though blackjack (slightly manipulable), and only at one end reach highly manipulatable games like chess and Go.
I’m sure Eliezer understands this, but his presentation doesn’t spend much time on it.
For example, how do the calculations change when you admit that the domain may make some desirable situations impossible?