I think it could be misinterpreted to mean “pause all AI development and deployment”, which results in a delayed deployment of “sponge safe” narrow AI systems that would improve or save a large number of people’s lives. There’s a real cost to slowing things down.
This cost is trivial compared to the cost of AGI Ruin. It’s like going on a plane to see your family on a plane where the engineers say they think there’s a >10% chance of catastrophic failure. Seeing your family is cool, but ~nobody would think it’s reasonable to go on such a plane. There are other ways to visit your family, they just take longer.
The analogy breaks down when it comes to trying to fix the plane. We understand how airplanes work; we do not understand how AI works. It makes sense to ground the plane until we have such understanding, despite the benefits of transportation.
I would love to have all the cool AI stuff too, but I don’t think we’re capable of toeing the line between safe and ruinous AI at acceptable risk levels.
I think in this analogy the narrow AI models would be like texting your parents instead of flying to see them. Obviously not as good as visiting them in person, but you avoid the 10% x-risk. I think Rob is saying let’s make sure we don’t stop the development of texting/calling as collateral.
So yes, don’t get on the plane, but let’s be very specific about what we’re trying to avoid.
There seems to be a trade-off in policy-space between attainability and nuance (part of what this whole dialogue seems to be about). The point I was trying to make here is that the good of narrow AI is such a marginal gain relative to the catastrophic ruin of superintelligent AI that it’s not worth being “very specific” at the cost of potentially weaker messaging for such a benefit.
Policy has adversarial pressure on it, so it makes sense to minimize the surface area if the consequence of a breach (e.g. “this is our really cool and big ai that’s technically a narrow ai and which just happens to be really smart at lots of things...”) is catastrophic.
This cost is trivial compared to the cost of AGI Ruin. It’s like going on a plane to see your family on a plane where the engineers say they think there’s a >10% chance of catastrophic failure. Seeing your family is cool, but ~nobody would think it’s reasonable to go on such a plane. There are other ways to visit your family, they just take longer.
The analogy breaks down when it comes to trying to fix the plane. We understand how airplanes work; we do not understand how AI works. It makes sense to ground the plane until we have such understanding, despite the benefits of transportation.
I would love to have all the cool AI stuff too, but I don’t think we’re capable of toeing the line between safe and ruinous AI at acceptable risk levels.
I think in this analogy the narrow AI models would be like texting your parents instead of flying to see them. Obviously not as good as visiting them in person, but you avoid the 10% x-risk. I think Rob is saying let’s make sure we don’t stop the development of texting/calling as collateral.
So yes, don’t get on the plane, but let’s be very specific about what we’re trying to avoid.
There seems to be a trade-off in policy-space between attainability and nuance (part of what this whole dialogue seems to be about). The point I was trying to make here is that the good of narrow AI is such a marginal gain relative to the catastrophic ruin of superintelligent AI that it’s not worth being “very specific” at the cost of potentially weaker messaging for such a benefit.
Policy has adversarial pressure on it, so it makes sense to minimize the surface area if the consequence of a breach (e.g. “this is our really cool and big ai that’s technically a narrow ai and which just happens to be really smart at lots of things...”) is catastrophic.