Hi Seth, Thanks for the comment! I agree that empirical evidence is very important for both airplane and AI Safety. IMO a strong case for Option 2 also has to include empirical tests as described in Option 1. I could’ve written that better and will edit thx.
To uncover weird failure modes (eg like Moonrise edge cases), it’s essential to have a well-resourced and capable reviewer team that is incentivised to think of such edge cases. However, this would probably still leave some edge cases that nobody ever thought of. I think such black swan events are just hard to anticipate for Risk Management esp when one doesn’t have a lot of historical evidence about similar systems.
Hi Seth, Thanks for the comment! I agree that empirical evidence is very important for both airplane and AI Safety. IMO a strong case for Option 2 also has to include empirical tests as described in Option 1. I could’ve written that better and will edit thx.
To uncover weird failure modes (eg like Moonrise edge cases), it’s essential to have a well-resourced and capable reviewer team that is incentivised to think of such edge cases. However, this would probably still leave some edge cases that nobody ever thought of. I think such black swan events are just hard to anticipate for Risk Management esp when one doesn’t have a lot of historical evidence about similar systems.