I wish you had entitled / framed this as “here are some disanalogies between biorisk and AI risk”, rather than suggesting in the title and intro that we should add up the analogies and subtract the disanalogies to get a General Factor of Analogizability between biorisk and AI risk.
We can say that they’re similar in some respects and different in other respects, and if a particular analogy-usage (in context) is leaning on an aspect in which they’re similar, that’s good, and if a particular analogy-usage (in context) is leaning on an aspect in which they’re different, that’s bad. For details and examples of what I mean, see my comments on a different post: here & here.
Fundamentally, OP is making the case that Biorisk is an extremely helpful (but not exact) analogy for AI risk, in the sense that we can gain understanding by looking the ways it’s analogous, and then get an even more nuanced understanding by analyzing the differences.
The point made seems to be more about it’s place in the discourse than about the value of the analogy itself? (E.g. “The Biorisk analogy is over-used” would be less misleading then)
On net, the analogies being used to try to explain are bad and misleading.
I agree that I could have tried to convey a different message, but I don’t think it’s the right one. Anyone who wants to dig in can decide for themselves, but you’re arguing that ideal reasoners won’t conflate different things and can disentangle the similarities and differences, and I agree, but I’m noting that people aren’t doing that, and others seem to agree.
This is an 800-word blog post, not 5 words. There’s plenty of room for nuance.
The way it stands right now, the post is supporting conversations like:
Person A: It’s not inconceivable that the world might wildly under-invest in societal resilience against catastrophic risks even after a “warning shot” for AI. Like for example, look at the case of bio-risks—COVID just happened, so the costs of novel pandemics are right now extremely salient to everyone on Earth, and yet, (…etc.).
Person B: You idiot, bio-risks are not at all analogous to AI. Look at this blog post by David Manheim explaining why.
Or:
Person B: All technology is always good, and its consequences are always good, and spreading knowledge is always good. So let’s make open-source ASI asap.
Person A: If I hypothetically found a recipe that allowed anyone to make a novel pandemic using widely-available equipment, and then I posted it on my blog along with clearly-illustrated step-by-step instructions, and took out a billboard out in Times Square directing people to the blog post, would you view my actions as praiseworthy? What would you expect to happen in the months after I did that?
Person B: You idiot, bio-risks are not at all analogous to AI. Look at this blog post by David Manheim explaining why.
Is this what you want? I.e., are you on the side of Person B in both these cases?
disanalogies listed here aren’t in and of themselves reasons that similar strategies cannot sometimes be useful, once the limitations are understood. For that reason, disanalogies should be a reminder and a caution against analogizing, not a reason on its own to reject parallel approaches in the different domains.
You seem to be simultaneously claiming that I had plenty of room to make a more nuanced argument, and then saying you think I’m saying something which exactly the nuance I included seems to address. Yes, people could cite the title of the blog post to make a misleading claim, assuming others won’t read it—and if that’s your concern, perhaps it would be enough to change the title to “Biorisk is Often an Unhelpful Analogy for AI Risk,” or “Biorisk is Misleading as a General Analogy for AI Risk”?
It’s not just the title but also (what I took to be) the thesis statement “Comparisons are often useful, but in this case, I think the disanalogies are much more compelling than the analogies”, and also “While I think the disanalogies are compelling…” and such.
For comparison:
Question:Are the analogies between me (Steve) and you (David) more compelling, or less compelling, than the disanalogies between me and you?
I think the correct answer to that question is “Huh? What are are you talking about?” For example:
If this is a conversation about gross anatomy or cardiovascular physiology across the animal kingdom, then you and I are generally extremely similar.
If this is a conversation about movie preferences or exercise routines, then I bet you and I are pretty different.
So at the end of the day, that question above is just meaningless, right? We shouldn’t be answering it at all.
I don’t think disanalogies between A and B can be “compelling” or “uncompelling”, any more than mathematical operations can be moist or dry. I think a disanalogy between A and B may invalidate a certain point that someone is trying to make by analogizing A to B, or it may not invalidate it, but that depends on what the point is.
FWIW I do think “Biorisk is Often an Unhelpful Analogy for AI Risk,” or “Biorisk is Misleading as a General Analogy for AI Risk” are both improvements :)
I agree that your question is weird and confused, and agree that if that were the context, my post would be hard to understand. But I think it’s a bad analogy! That’s because there are people who have made analogies between AI and Bio very poorly, and it’s misleading and leading to sloppy thinking. In my experience seeing discussions on the topic, either the comparisons are drawn carefully and the relevant dissimilarities are discussed clearly, or they are bad analogies.
To stretch your analogy, if the context were that I’d recently heard people say “Steve and David are both people I know, and if you don’t like Steve, you probably won’t like David,” and also “Steve and David are both concerned about AI risks, so they agree on how to discuss the issue,” I’d wonder if there was some confusion, and I’d feel comfortable saying that in general, Steve is an unhelpful analog for David, and all these people should stop and be much more careful in how they think about comparisons between us.
I wish you had entitled / framed this as “here are some disanalogies between biorisk and AI risk”, rather than suggesting in the title and intro that we should add up the analogies and subtract the disanalogies to get a General Factor of Analogizability between biorisk and AI risk.
We can say that they’re similar in some respects and different in other respects, and if a particular analogy-usage (in context) is leaning on an aspect in which they’re similar, that’s good, and if a particular analogy-usage (in context) is leaning on an aspect in which they’re different, that’s bad. For details and examples of what I mean, see my comments on a different post: here & here.
Fundamentally, OP is making the case that Biorisk is an extremely helpful (but not exact) analogy for AI risk, in the sense that we can gain understanding by looking the ways it’s analogous, and then get an even more nuanced understanding by analyzing the differences.
The point made seems to be more about it’s place in the discourse than about the value of the analogy itself? (E.g. “The Biorisk analogy is over-used” would be less misleading then)
On net, the analogies being used to try to explain are bad and misleading.
I agree that I could have tried to convey a different message, but I don’t think it’s the right one. Anyone who wants to dig in can decide for themselves, but you’re arguing that ideal reasoners won’t conflate different things and can disentangle the similarities and differences, and I agree, but I’m noting that people aren’t doing that, and others seem to agree.
This is an 800-word blog post, not 5 words. There’s plenty of room for nuance.
The way it stands right now, the post is supporting conversations like:
Or:
Is this what you want? I.e., are you on the side of Person B in both these cases?
I said:
You seem to be simultaneously claiming that I had plenty of room to make a more nuanced argument, and then saying you think I’m saying something which exactly the nuance I included seems to address. Yes, people could cite the title of the blog post to make a misleading claim, assuming others won’t read it—and if that’s your concern, perhaps it would be enough to change the title to “Biorisk is Often an Unhelpful Analogy for AI Risk,” or “Biorisk is Misleading as a General Analogy for AI Risk”?
It’s not just the title but also (what I took to be) the thesis statement “Comparisons are often useful, but in this case, I think the disanalogies are much more compelling than the analogies”, and also “While I think the disanalogies are compelling…” and such.
For comparison:
Question: Are the analogies between me (Steve) and you (David) more compelling, or less compelling, than the disanalogies between me and you?
I think the correct answer to that question is “Huh? What are are you talking about?” For example:
If this is a conversation about gross anatomy or cardiovascular physiology across the animal kingdom, then you and I are generally extremely similar.
If this is a conversation about movie preferences or exercise routines, then I bet you and I are pretty different.
So at the end of the day, that question above is just meaningless, right? We shouldn’t be answering it at all.
I don’t think disanalogies between A and B can be “compelling” or “uncompelling”, any more than mathematical operations can be moist or dry. I think a disanalogy between A and B may invalidate a certain point that someone is trying to make by analogizing A to B, or it may not invalidate it, but that depends on what the point is.
FWIW I do think “Biorisk is Often an Unhelpful Analogy for AI Risk,” or “Biorisk is Misleading as a General Analogy for AI Risk” are both improvements :)
I agree that your question is weird and confused, and agree that if that were the context, my post would be hard to understand. But I think it’s a bad analogy! That’s because there are people who have made analogies between AI and Bio very poorly, and it’s misleading and leading to sloppy thinking. In my experience seeing discussions on the topic, either the comparisons are drawn carefully and the relevant dissimilarities are discussed clearly, or they are bad analogies.
To stretch your analogy, if the context were that I’d recently heard people say “Steve and David are both people I know, and if you don’t like Steve, you probably won’t like David,” and also “Steve and David are both concerned about AI risks, so they agree on how to discuss the issue,” I’d wonder if there was some confusion, and I’d feel comfortable saying that in general, Steve is an unhelpful analog for David, and all these people should stop and be much more careful in how they think about comparisons between us.