Any system that takes a huge amount of input data and reduces it to some sort of representation will have input cases it doesn’t handle well. The reduction throws away data of a certain, supposedly unimportant, variety. Input cases are bound to exist where the data thrown away by the reduction algorithm are, in fact, important. Visual illusions are such cases for the human visual system. Those that work on autonomous vehicles have to deal with such cases. Humans that understand how such recognition systems work can purposefully construct such cases in order to “hack” them. It’s a jungle out there.
Any system that takes a huge amount of input data and reduces it to some sort of representation will have input cases it doesn’t handle well. The reduction throws away data of a certain, supposedly unimportant, variety. Input cases are bound to exist where the data thrown away by the reduction algorithm are, in fact, important. Visual illusions are such cases for the human visual system. Those that work on autonomous vehicles have to deal with such cases. Humans that understand how such recognition systems work can purposefully construct such cases in order to “hack” them. It’s a jungle out there.