Very interesting exercise on modeling, with some great lessons. I don’t really like the AI analogy though.
The ramp problem is a situation where idealizations are well-understood. The main steps to solving it seem to be realizing that these idealizations are very far from reality and measuring (rather than modeling) as much as possible.
On the first step, comparing with AI progress and risks there, nobody thinks they have a detailed mechanistic model of what should happen. Rather, most people just assume there is no need to get anything right on the first try because that approach has “worked” so-far for new technology. People also anticipate strong capabilities to take a lot longer to develop and generally underdeliver on the industry’s promises, again because that’s how it usually goes with new hyped technology. You could say “that’s the model one should resist using there” but the analogy is very stretched in my opinion. It only applies if the “potential models to be resisted” are taken to be extremely crude base-rate estimates from guestimated base-rates for how “technological development in general” is supposed to work. Such a model would be just as established as the fact that “such very crude models give terribly inaccurate predictions”. There is no temptation there.
On the second step, I don’t seee what one might reasonably try to measure concerning AI progress. E.g., extrapolating some curve of “capability advancement over time” rather than just being sceptical-by-default isn’t going to make a difference for AI risk.
I think a better metaphorical experiment/puzzle relevant to AI risk would be one where you naively think you have a lot of tries, but it turns out that you only get one due to some catastrophic failure which you could have figured out and mitigated if only you thought and looked more carefully. In the ramp problem, the “you get one try” part is implicit in how the problem is phrased.
My argument is based on models concerned with the question whether “you only get one try for AI”. Maybe some people are unconcerned because they assume that others have detailed and reasonably accurate models of what a given AI will do. I doubt that because “it’s a blackbox” is the one fact one hears most often about current AI.
Very interesting exercise on modeling, with some great lessons. I don’t really like the AI analogy though.
The ramp problem is a situation where idealizations are well-understood. The main steps to solving it seem to be realizing that these idealizations are very far from reality and measuring (rather than modeling) as much as possible.
On the first step, comparing with AI progress and risks there, nobody thinks they have a detailed mechanistic model of what should happen. Rather, most people just assume there is no need to get anything right on the first try because that approach has “worked” so-far for new technology. People also anticipate strong capabilities to take a lot longer to develop and generally underdeliver on the industry’s promises, again because that’s how it usually goes with new hyped technology. You could say “that’s the model one should resist using there” but the analogy is very stretched in my opinion. It only applies if the “potential models to be resisted” are taken to be extremely crude base-rate estimates from guestimated base-rates for how “technological development in general” is supposed to work. Such a model would be just as established as the fact that “such very crude models give terribly inaccurate predictions”. There is no temptation there.
On the second step, I don’t seee what one might reasonably try to measure concerning AI progress. E.g., extrapolating some curve of “capability advancement over time” rather than just being sceptical-by-default isn’t going to make a difference for AI risk.
I think a better metaphorical experiment/puzzle relevant to AI risk would be one where you naively think you have a lot of tries, but it turns out that you only get one due to some catastrophic failure which you could have figured out and mitigated if only you thought and looked more carefully. In the ramp problem, the “you get one try” part is implicit in how the problem is phrased.
My argument is based on models concerned with the question whether “you only get one try for AI”. Maybe some people are unconcerned because they assume that others have detailed and reasonably accurate models of what a given AI will do. I doubt that because “it’s a blackbox” is the one fact one hears most often about current AI.