This occurs to me as an important reason to not release an AI system capable of long-term goal setting, nor build it in a way that risks getting leaked.
However it might also be a reason in favor of privately building an AI system that occupy all the low hanging fruit opportunities for money making on the internet, which both makes money for whoever builds the system and ensures that they are not usable by rogue AI worms or those controlled by bad actors. However:
And returns might be larger for agents without legal or ethical guardrails- cue a deluge of scams and ransomware attacks.
This is the biggest problem imo, not just because scams and ransomware are bad but also because these evil money-making opportunities will not be depleted by legitimate people running automated businesses with AI. This means that computer security and scam protection are extremely important.
This occurs to me as an important reason to not release an AI system capable of long-term goal setting, nor build it in a way that risks getting leaked.
However it might also be a reason in favor of privately building an AI system that occupy all the low hanging fruit opportunities for money making on the internet, which both makes money for whoever builds the system and ensures that they are not usable by rogue AI worms or those controlled by bad actors. However:
This is the biggest problem imo, not just because scams and ransomware are bad but also because these evil money-making opportunities will not be depleted by legitimate people running automated businesses with AI. This means that computer security and scam protection are extremely important.