Are evolution analogies really that much of a crux? It seems like the evidence from evolution can’t get us that far in an absolute sense (though I could imagine evolution updating someone up to a moderate probability from a super low prior?), so we should be able to talk about more object level things regardless.
Yeah, I agree with this, we should be able to usually talk about object level things.
Although (as you note in your other comment) evolution is useful for thinking about inner optimizers, deceptive alignment etc. I think that thinking about “optimizers” (what things create optimizers? what will the optimizers do? etc) is pretty hard, and at least I think it’s useful to be able to look at the one concrete case where some process created a generally competent optimizer
Are evolution analogies really that much of a crux? It seems like the evidence from evolution can’t get us that far in an absolute sense (though I could imagine evolution updating someone up to a moderate probability from a super low prior?), so we should be able to talk about more object level things regardless.
Yeah, I agree with this, we should be able to usually talk about object level things.
Although (as you note in your other comment) evolution is useful for thinking about inner optimizers, deceptive alignment etc. I think that thinking about “optimizers” (what things create optimizers? what will the optimizers do? etc) is pretty hard, and at least I think it’s useful to be able to look at the one concrete case where some process created a generally competent optimizer