I think I do want to make my agent-like architecture general enough to include evolution. However, there might be a spectrum of agent-like-ness such that you can’t get much more than Sphex behavior with just evolution (without having a mesa-optimizer in there)
I think you can guarantee that, probabilistically, getting a specific outcome requires information about that outcome (no free lunch), which implies “search” on a “world model.”
Yeah, but do you think you can make it feel more like a formal proof?
I think I do want to make my agent-like architecture general enough to include evolution. However, there might be a spectrum of agent-like-ness such that you can’t get much more than Sphex behavior with just evolution (without having a mesa-optimizer in there)
Yeah, but do you think you can make it feel more like a formal proof?