I’m not sure that’s quite right. A genetic mutation is “one thing” but it can easily have many different effects especially once you consider that it’s active for an entire lifetime.
And doesn’t gradient descent also demand that each weight update is beneficial? At a much more fine grain than evolution does… then again I guess there’s grokking so I’m not sure.
I’m not sure that’s quite right. A genetic mutation is “one thing” but it can easily have many different effects especially once you consider that it’s active for an entire lifetime.
Wile this can happen, empirically speaking (at least for eukaryotes), genetic mutations are mostly much more modular and limited to one specific thing by default, rather than it affecting everything else in a tangled way, and this is due to genetics research discovering that things are mostly linear and compositional for genetic effects, in the sense that the best way to predict what will happen if you add two genes together is that their effects are summed up, not interacting in a nonlinear way.
I’m not sure that’s quite right. A genetic mutation is “one thing” but it can easily have many different effects especially once you consider that it’s active for an entire lifetime.
And doesn’t gradient descent also demand that each weight update is beneficial? At a much more fine grain than evolution does… then again I guess there’s grokking so I’m not sure.
Wile this can happen, empirically speaking (at least for eukaryotes), genetic mutations are mostly much more modular and limited to one specific thing by default, rather than it affecting everything else in a tangled way, and this is due to genetics research discovering that things are mostly linear and compositional for genetic effects, in the sense that the best way to predict what will happen if you add two genes together is that their effects are summed up, not interacting in a nonlinear way.