The two paths to thing X might also be non-equivalent for reasons other than quantity/scale.
If for example learning about biology and virology from textbooks and professors is more difficult, and thereby acts as a filter to selectively teach those things to people who are uncommonly talented and dedicated, and if that correlates with good intentions.
Or if learning from standard education embeds people in a social group that also to some extent socialises its members with norms of ethical practice, and monitors for people who seem unstable or dangerous (whereas LLM learning can be solitary and unobserved)
It’s not just good intentions, but also the temptations of capitalism. Anyone smart and conscientious enough to engineer a pandemic is probably also able to get a well-paid job and live a comfortable life, so the temptation to kill everyone (including themselves) is small.
EDIT:
To expand on the concept of “temptations of capitalism”, the idea is roughly that skills necessary to overthrow a regime are often also useful for making money. For example, if you can get thousands of followers, you could sell them a book or a seminar, get tons of money, and live comfortably. The more followers you can get, the more resources you can extract from them. Or if you are great at organizing, you could found a company or become a CEO of an existing one. -- On the other hand, if the regime prevents you from doing these things (because the positions of comfortable life are reserved for those of noble origin, or for those whose parents and grandparents were Party members), you may be tempted to use your followers or your organizational skills to support a revolution. As an example, V. I. Lenin originally wanted to be a lawyer, but was not allowed for political reasons, so he used his talents to overthrow the regime instead.
This can’t be universally true: Aum Shinrikyo famously recruited mostly from top university students, Osama bin Laden inherited a lot of money and attended an elite high school (Wikipedia says accounts differ in his success at university). In general, often rich people have some void of meaning that can be filled by religious or ideological movements, which can motivate violence.
The two paths to thing X might also be non-equivalent for reasons other than quantity/scale.
If for example learning about biology and virology from textbooks and professors is more difficult, and thereby acts as a filter to selectively teach those things to people who are uncommonly talented and dedicated, and if that correlates with good intentions.
Or if learning from standard education embeds people in a social group that also to some extent socialises its members with norms of ethical practice, and monitors for people who seem unstable or dangerous (whereas LLM learning can be solitary and unobserved)
That could be true, but I’m not actually that optimistic about elite morals and didn’t want the counter-argument to rely on that.
It’s not just good intentions, but also the temptations of capitalism. Anyone smart and conscientious enough to engineer a pandemic is probably also able to get a well-paid job and live a comfortable life, so the temptation to kill everyone (including themselves) is small.
EDIT:
To expand on the concept of “temptations of capitalism”, the idea is roughly that skills necessary to overthrow a regime are often also useful for making money. For example, if you can get thousands of followers, you could sell them a book or a seminar, get tons of money, and live comfortably. The more followers you can get, the more resources you can extract from them. Or if you are great at organizing, you could found a company or become a CEO of an existing one. -- On the other hand, if the regime prevents you from doing these things (because the positions of comfortable life are reserved for those of noble origin, or for those whose parents and grandparents were Party members), you may be tempted to use your followers or your organizational skills to support a revolution. As an example, V. I. Lenin originally wanted to be a lawyer, but was not allowed for political reasons, so he used his talents to overthrow the regime instead.
This can’t be universally true: Aum Shinrikyo famously recruited mostly from top university students, Osama bin Laden inherited a lot of money and attended an elite high school (Wikipedia says accounts differ in his success at university). In general, often rich people have some void of meaning that can be filled by religious or ideological movements, which can motivate violence.