Well, A (the developer) has substantial discretion over a powerful technology which has impact on both A and B (other people), which is a moral hazard. This applies in both cases in different ways (corrigible-to-them internal-only access and prosocial-according-to-them external deployment). In my reading of Daniel’s response, the hypothesis is that in both contexts the tech would be able to exert takeover-assisting or -implementing influence.
I agree that some sort of monitoring scheme internally might mitigate that. And some sort of visible spec with auditing and/or other validation might help in the external case.
Well, A (the developer) has substantial discretion over a powerful technology which has impact on both A and B (other people), which is a moral hazard. This applies in both cases in different ways (corrigible-to-them internal-only access and prosocial-according-to-them external deployment). In my reading of Daniel’s response, the hypothesis is that in both contexts the tech would be able to exert takeover-assisting or -implementing influence.
I agree that some sort of monitoring scheme internally might mitigate that. And some sort of visible spec with auditing and/or other validation might help in the external case.