You “sound” perfectly LessWrongy to me! (We don’t have a ‘style guide’ or anything :))
I was hoping more for ‘brainstorm’ type responses I guess. (A somewhat comprehensive answer would be wonderful of course.)
I’m not actually worried about this. I don’t expect any particular ‘immediate’ tragedy to result. I’m thinking of this as more of ‘security puzzle’ than anything else, e.g. a ‘call to action’.
This isn’t a political campaign, or activism, or me attempting to pull any kind of ‘alarm’ about this being an impending catastrophe!
I’m really just curious.
Your details about likely defenses is interesting – pretty obvious, but good to point out anyways.
I was curious in particular about “detecting what compounds are being created and to prevent or block creation of bad stuff” – both the concrete practical specifics of how this would or could work, as well as, e.g. hard limits even in principle about what could be done.
Maybe the lab will only perform ‘verified’ reactions? Or maybe they have some protocol for handling ‘novel reactions’ too? I would expect the latter (or, rather hope they had a (good) protocol because it seems like part of the point of these labs being enabling chemistry experiments, i.e. potentially novel chemical reactions.
Also, since it’s not a General Availability thing, just like OpenAI is not exactly GA, the applications are probably controlled and monitored for negative use, which might cause negative press for this new system.
How well do those kinds of protections work generally? (I don’t know myself.)
I was thinking more that the more likely vulnerability is a ‘vetted’ customer’s account being hacked. I’d expect that to, sadly, be ridiculously easy to do (for a determined and resourceful attacker).
The site/backend-system itself being hacked also seems like an inevitability. What are the security risks when that happens?
What’s the risk analysis for labs like this in the case of an attack ‘on the order of’ something like Stuxnet?
You “sound” perfectly LessWrongy to me! (We don’t have a ‘style guide’ or anything :))
I was hoping more for ‘brainstorm’ type responses I guess. (A somewhat comprehensive answer would be wonderful of course.)
I’m not actually worried about this. I don’t expect any particular ‘immediate’ tragedy to result. I’m thinking of this as more of ‘security puzzle’ than anything else, e.g. a ‘call to action’.
This isn’t a political campaign, or activism, or me attempting to pull any kind of ‘alarm’ about this being an impending catastrophe!
I’m really just curious.
Your details about likely defenses is interesting – pretty obvious, but good to point out anyways.
I was curious in particular about “detecting what compounds are being created and to prevent or block creation of bad stuff” – both the concrete practical specifics of how this would or could work, as well as, e.g. hard limits even in principle about what could be done.
Maybe the lab will only perform ‘verified’ reactions? Or maybe they have some protocol for handling ‘novel reactions’ too? I would expect the latter (or, rather hope they had a (good) protocol because it seems like part of the point of these labs being enabling chemistry experiments, i.e. potentially novel chemical reactions.
How well do those kinds of protections work generally? (I don’t know myself.)
I was thinking more that the more likely vulnerability is a ‘vetted’ customer’s account being hacked. I’d expect that to, sadly, be ridiculously easy to do (for a determined and resourceful attacker).
The site/backend-system itself being hacked also seems like an inevitability. What are the security risks when that happens?
What’s the risk analysis for labs like this in the case of an attack ‘on the order of’ something like Stuxnet?