This raises a good point, especially with ethics and blame/credit. Before the Rootclaim debate, I gave a lab leak at about 65% confidence. After the debate and some thought, I put it much lower, <10% with a lot of my uncertainty based on my relatively low biology knowledge. If someone asks, I may say ~about 10% but I’m not enough of an expert. (I also expect if I was more of an expert or spent a lot more time, my 10% would go down, in tension to rationality)
HOWEVER that does not mean I can ethically judge the GoF researchers as if they had taken a 10% chance at killing >20 million people or about equal to killing 2 million for certain. (I do think they were reckless, biased, unethical, generally bad etc but just not capable enough to cause such harm).
There seems to be a bit of a Pascal mugging like thing going on here—if you are not an expert I can convince you that x has a ~1% chance of ending the world, therefore anyone involved is the worst person in history.
An objection not mentioned here that I saw from Emmett Shear was in general against assuming the AI will misbehave out of distribution. That isn’t explicitly mentioned here, but is often in the literature, especially older texts such as Superintelligence by Bostrom. His argument was that generalizing OOD well is an essential feature of intelligence and any superintelligence will in fact do it better than humans.