It’s a combination of 1 and 2. Which is to say, regulations require a high level of safety, and we don’t have that yet because of point 2.
Robustness is very hard! And in particular perception is very hard when you don’t have a really solid world model, because when the AI sees something new it can react or fail to react to it in surprising ways.
The car companies are working on this by putting the cars through many millions of miles of simulated content so that it has seen most things, but that last mile (heh) problem is still very hard with today’s technology.
You can think of this as an example of why alignment is hard. Perfect driving in a simulated environment doesn’t equal perfect driving in the real world, and while perfection isn’t the goal (being superhuman is the goal), our robust perceptual and modeling systems are very hard to beat, at least for now.
It’s a combination of 1 and 2. Which is to say, regulations require a high level of safety, and we don’t have that yet because of point 2.
Robustness is very hard! And in particular perception is very hard when you don’t have a really solid world model, because when the AI sees something new it can react or fail to react to it in surprising ways.
The car companies are working on this by putting the cars through many millions of miles of simulated content so that it has seen most things, but that last mile (heh) problem is still very hard with today’s technology.
You can think of this as an example of why alignment is hard. Perfect driving in a simulated environment doesn’t equal perfect driving in the real world, and while perfection isn’t the goal (being superhuman is the goal), our robust perceptual and modeling systems are very hard to beat, at least for now.