If developing interpretable ASIs is beyond us, we might need to strive towards making them extremely difficult to interpret, even for themselves.
Intuitively, I think that if developing interpretable ASI is beyond us, then developing provably-obscure-to-ASI ASI is beyond us too.
Interesting point, and you might be right. Could get very complicated because ideally an ASI might want to convince other ASIs that it has one utility function, when in fact it has another, and of course all the ASIs might take this into account.
Intuitively, I think that if developing interpretable ASI is beyond us, then developing provably-obscure-to-ASI ASI is beyond us too.
Interesting point, and you might be right. Could get very complicated because ideally an ASI might want to convince other ASIs that it has one utility function, when in fact it has another, and of course all the ASIs might take this into account.