Exactly that! Sorry to say, Chris Santos-Lang, but your post feels dangerously naive to me. Starting with the premise that our inability to come up with a winning evil startegy in a limited game is an empirical proof of some, when the whole point is that we cannot anticipate what novel stategies an ASI might come up with! And all the sneaky actions it might invent that bypass anything you have modeled in your simplified game scenario!
Exactly that! Sorry to say, Chris Santos-Lang, but your post feels dangerously naive to me. Starting with the premise that our inability to come up with a winning evil startegy in a limited game is an empirical proof of some, when the whole point is that we cannot anticipate what novel stategies an ASI might come up with! And all the sneaky actions it might invent that bypass anything you have modeled in your simplified game scenario!