What makes driving on surface streets so much different than driving on highways such that current state of the art ML techniques wouldn’t be able to handle it with slightly more data and compute?
Unlike natural language processing, AI doctors or household robots, driving seems like a very limited non-AGI-complete task to me because a self-driving car never truly interacts with humans or objects beyond avoiding hitting them.
we also need to notice objects which are going to move into the lane; we need trajectory tracking and forecasting. And we need the trajectory-tracker to be robust to the object classification changing (or just being wrong altogether), or sometimes confusing which object is which across timesteps, or reflections or pictures or moving lights, or missing/unreliable lane markers, or things in the world zig-zagging around on strange trajectories, or etc.
I would claim all of the above are also required for driving on the highway.
This is secondhand, but… two years ago I worked with a guy who had been on Tesla’s autopilot team. From the sound of it, they stayed in the lane mainly via some hand-coded image processing which looked for a yellow/white strip surrounded by darker color. For most highway driving, that turned out to be good enough.
I’m not sure how much state-of-the-art ML techniques (i.e. deep learning) are even being used for self-driving. I’m sure they’re used for some subtasks, like object recognition, but my (several-years-out-of-date and secondhand) understanding is that current projects aren’t actually using it end-to-end; it’s just specific subcomponents. Slightly more data/compute don’t matter much when key limiting pieces aren’t actually using ML.
From the sound of it, they stayed in the lane mainly via some hand-coded image processing which looked for a yellow/white strip surrounded by darker color.
That is what I heard about other research groups but a bit surprising coming from Tesla, I’d imagine things have changed dramatically since then considering this video, albeit insufficient as any sort of safety validation, still demonstrates they’re way beyond just following lane markings. According to Musk they’re pushing hard for end-to-end ML solutions. It would make sense seeing the custom hardware they’ve developed and also the data leverage they have with their massive fleet, combined with over-the-air updates.
It’s certainly plausible that things have changed dramatically, although my default guess is that they haven’t—a pile of hacks can go a surprisingly long way, and the only tricky-looking spot I saw in that video was a short section just after 1:30. And Musk saying that they’re “pushing hard for end-to-end ML” is exactly the sort of thing I’d expect to hear if such a project was not actually finding any traction. I’m sure they’re trying to do it, but ML is finicky at the best of times, and I expect we’d hear it shouted from the rooftops if end-to-end self-driving ML was actually starting to work yet.
What makes driving on surface streets so much different than driving on highways such that current state of the art ML techniques wouldn’t be able to handle it with slightly more data and compute?
Unlike natural language processing, AI doctors or household robots, driving seems like a very limited non-AGI-complete task to me because a self-driving car never truly interacts with humans or objects beyond avoiding hitting them.
I would claim all of the above are also required for driving on the highway.
This is secondhand, but… two years ago I worked with a guy who had been on Tesla’s autopilot team. From the sound of it, they stayed in the lane mainly via some hand-coded image processing which looked for a yellow/white strip surrounded by darker color. For most highway driving, that turned out to be good enough.
I’m not sure how much state-of-the-art ML techniques (i.e. deep learning) are even being used for self-driving. I’m sure they’re used for some subtasks, like object recognition, but my (several-years-out-of-date and secondhand) understanding is that current projects aren’t actually using it end-to-end; it’s just specific subcomponents. Slightly more data/compute don’t matter much when key limiting pieces aren’t actually using ML.
That is what I heard about other research groups but a bit surprising coming from Tesla, I’d imagine things have changed dramatically since then considering this video, albeit insufficient as any sort of safety validation, still demonstrates they’re way beyond just following lane markings. According to Musk they’re pushing hard for end-to-end ML solutions. It would make sense seeing the custom hardware they’ve developed and also the data leverage they have with their massive fleet, combined with over-the-air updates.
It’s certainly plausible that things have changed dramatically, although my default guess is that they haven’t—a pile of hacks can go a surprisingly long way, and the only tricky-looking spot I saw in that video was a short section just after 1:30. And Musk saying that they’re “pushing hard for end-to-end ML” is exactly the sort of thing I’d expect to hear if such a project was not actually finding any traction. I’m sure they’re trying to do it, but ML is finicky at the best of times, and I expect we’d hear it shouted from the rooftops if end-to-end self-driving ML was actually starting to work yet.