I’ve seen the “ML gets deployed carelessly” narrative pop up on LW a bunch, and while it does seem accurate in many cases, I wanted to note that there are counter-examples. The most prominent counter-example I’m aware of is the incredibly cautious approach DeepMind/Google took when designing the ML system that cools Google’s datacenters.
This seems to be careful deployment. The concept of deployment is going from an AI in the lab, to the same AI in control of a real world system. Suppose your design process was to fiddle around in the lab until you make something that seems to work. Once you have that, you look at it to understand why it works. You try to prove theorems about it. You subject it to some extensive battery of testing and will only put it in a self driving car/ data center cooling system once you are confident it is safe.
There are two places this could fail. Your testing procedures could be insufficient, or your AI could hack out of the lab before the testing starts. I see little to no defense against the latter.
I’ve seen the “ML gets deployed carelessly” narrative pop up on LW a bunch, and while it does seem accurate in many cases, I wanted to note that there are counter-examples. The most prominent counter-example I’m aware of is the incredibly cautious approach DeepMind/Google took when designing the ML system that cools Google’s datacenters.
This seems to be careful deployment. The concept of deployment is going from an AI in the lab, to the same AI in control of a real world system. Suppose your design process was to fiddle around in the lab until you make something that seems to work. Once you have that, you look at it to understand why it works. You try to prove theorems about it. You subject it to some extensive battery of testing and will only put it in a self driving car/ data center cooling system once you are confident it is safe.
There are two places this could fail. Your testing procedures could be insufficient, or your AI could hack out of the lab before the testing starts. I see little to no defense against the latter.