This seems closely related to inside-view versus outside-view. The think-lobe of the brain comes up with a cunning plan. The plan breaks an ethical rule but calculation shows it is for the greater good. The executive-lobe of the brain then ponders the outside view. Every-one who has executed an evil cunning plan has run a calculation of the greater good and had their plan endorsed. So the calculation lack outside-view credibility.
What kind of evidence could give outside-view credibility? Consider a plan with lots of traceability to previous events. If it goes badly, past events will have to be re-interpreted, and much learning will take place. Well, people generally don’t learn from the past. If the think-lobe’s cunning plan retains enough debugging information to avoid going wrong and later going wrong again, that distinguishes it from what people usually do and gives it outside-view credibility.
Randomised controlled trials of medical treatments can be attacked on ethical grounds from both sides. They deny some patients medical treatments that is quite likely beneficial. They inflict unproven and potentially dangerous treatment on others. Both attacks lack outside-view credibility. We always think we know. The randomised trial itself has outside-view credibility. It will place us in the position that we can do the right thing without having to use our judgement or be clever.
This seems closely related to inside-view versus outside-view. The think-lobe of the brain comes up with a cunning plan. The plan breaks an ethical rule but calculation shows it is for the greater good. The executive-lobe of the brain then ponders the outside view. Every-one who has executed an evil cunning plan has run a calculation of the greater good and had their plan endorsed. So the calculation lack outside-view credibility.
What kind of evidence could give outside-view credibility? Consider a plan with lots of traceability to previous events. If it goes badly, past events will have to be re-interpreted, and much learning will take place. Well, people generally don’t learn from the past. If the think-lobe’s cunning plan retains enough debugging information to avoid going wrong and later going wrong again, that distinguishes it from what people usually do and gives it outside-view credibility.
Randomised controlled trials of medical treatments can be attacked on ethical grounds from both sides. They deny some patients medical treatments that is quite likely beneficial. They inflict unproven and potentially dangerous treatment on others. Both attacks lack outside-view credibility. We always think we know. The randomised trial itself has outside-view credibility. It will place us in the position that we can do the right thing without having to use our judgement or be clever.