Thanks. This sounds like a more peripheral interest/concern, compared to Eliezer/LW’s, which was more like, we have to fully solve DT before building AGI/ASI, otherwise it could be catastrophic due to something like the AI falling prey to an acausal threat or commitment races, or can’t cooperate with other AIs.
we have to fully solve DT before building AGI/ASI, otherwise it could be catastrophic due to something like the AI falling prey to an acausal threat or commitment races, or can’t cooperate with other AIs.
These seem to me like much lower priority problems than ensuring that our AI agent don’t stage a takeover. In comparison, this seems like an exotic failure mode. Further, this is a problem that a self-modifying AGI might very well be able to solve on it’s own before being extorted.
Which isn’t to say that there are not catastrophic possibilities here, but I’m surprised that these were the reasons given at the time for Decision Theory getting top billing.
Am I missing something or is this indeed much lower priority than subproblems that are more directly about preventing AI takeover?
Thanks. This sounds like a more peripheral interest/concern, compared to Eliezer/LW’s, which was more like, we have to fully solve DT before building AGI/ASI, otherwise it could be catastrophic due to something like the AI falling prey to an acausal threat or commitment races, or can’t cooperate with other AIs.
These seem to me like much lower priority problems than ensuring that our AI agent don’t stage a takeover. In comparison, this seems like an exotic failure mode. Further, this is a problem that a self-modifying AGI might very well be able to solve on it’s own before being extorted.
Which isn’t to say that there are not catastrophic possibilities here, but I’m surprised that these were the reasons given at the time for Decision Theory getting top billing.
Am I missing something or is this indeed much lower priority than subproblems that are more directly about preventing AI takeover?