P(A bit flips in the AI that leads to it convincing a human panel to do something bad) is always less than P(A bit flips anywhere that leads to a human panel deciding to do something bad), (the former is a subset of the latter).
The point of the cosmic ray statement is not so much that that might actually happen, but is just demonstrating that the Outcome-Pump-2.0-universe doesn’t necessarily result in a positive outcome, just that it is a universe that has had the “Outcome” accepted, and also that the Outcome being accepted doesn’t imply that the universe is one we like.
P(A bit flips in the AI that leads to it convincing a human panel to do something bad) is always less than P(A bit flips anywhere that leads to a human panel deciding to do something bad), (the former is a subset of the latter).
The point of the cosmic ray statement is not so much that that might actually happen, but is just demonstrating that the Outcome-Pump-2.0-universe doesn’t necessarily result in a positive outcome, just that it is a universe that has had the “Outcome” accepted, and also that the Outcome being accepted doesn’t imply that the universe is one we like.