I don’t think the random training would be very good at mitigating the anthropic capture risk because the AI will probably be able to make reasonable guesses about whether it’s in the random training examples or somewhere the agents might try to anthropically capture.
I agree that concern for remote influence is a complicated and unlikely generalization for within a forward pass, but once you have a competent general recurrent reasoner spending a bit of time thinking about what it wants to do, it seems plausible. Reflecting and deciding to do FDT instead of CDT also seems plausible.
I don’t think the random training would be very good at mitigating the anthropic capture risk because the AI will probably be able to make reasonable guesses about whether it’s in the random training examples or somewhere the agents might try to anthropically capture.
I agree. The thought is that if it’s unwilling to risk any probability of being in training at all (which it would have to be to always be maximally fit under these assumptions, although not necessarily for any particular finite random dataset), then this would prevent it from being remotely influenceable. However, most likely it would not be remotely influenceable anyway in that case.
I don’t think the random training would be very good at mitigating the anthropic capture risk because the AI will probably be able to make reasonable guesses about whether it’s in the random training examples or somewhere the agents might try to anthropically capture.
I agree that concern for remote influence is a complicated and unlikely generalization for within a forward pass, but once you have a competent general recurrent reasoner spending a bit of time thinking about what it wants to do, it seems plausible. Reflecting and deciding to do FDT instead of CDT also seems plausible.
Makes sense!
I agree. The thought is that if it’s unwilling to risk any probability of being in training at all (which it would have to be to always be maximally fit under these assumptions, although not necessarily for any particular finite random dataset), then this would prevent it from being remotely influenceable. However, most likely it would not be remotely influenceable anyway in that case.