An important caveat is that many non-AI systems have humans in the loop somewhere that can intervene if they don’t like what the automated system is doing. Some examples:
we shut down stock markets that seem to be out of control
employees ignore standard operating procedures when they get into corner cases and SOPs would have them do something that would hurt them or that they’d get in trouble for
an advertiser might manually override their automated ad bidding algorithm if it tries to spend too much or too little money
customer service reps (or their managers) are empowered to override automated support systems (e.g. talk to the operator when an automated phone system can’t handle your request)
Much of the concern about AI systems is when they lack support for these kind of interventions, whether it be because they are too fast, too complex, or can outsmart the would-be intervening human trying to correct what they see as an error.
Much of the concern about AI systems is when they lack support for these kind of interventions, whether it be because they are too fast, too complex, or can outsmart the would-be intervening human trying to correct what they see as an error.
All of these possible causes for the lack of support are valid. I would like to add one more: when the humans that could provide this kind of support don’t care about providing it or have incentives against providing it. For example, I could report a bug in some system, but this would cost me time and only benefit people I don’t know, so I will happily ignore it :-).
An important caveat is that many non-AI systems have humans in the loop somewhere that can intervene if they don’t like what the automated system is doing. Some examples:
we shut down stock markets that seem to be out of control
employees ignore standard operating procedures when they get into corner cases and SOPs would have them do something that would hurt them or that they’d get in trouble for
an advertiser might manually override their automated ad bidding algorithm if it tries to spend too much or too little money
customer service reps (or their managers) are empowered to override automated support systems (e.g. talk to the operator when an automated phone system can’t handle your request)
Much of the concern about AI systems is when they lack support for these kind of interventions, whether it be because they are too fast, too complex, or can outsmart the would-be intervening human trying to correct what they see as an error.
All of these possible causes for the lack of support are valid. I would like to add one more: when the humans that could provide this kind of support don’t care about providing it or have incentives against providing it. For example, I could report a bug in some system, but this would cost me time and only benefit people I don’t know, so I will happily ignore it :-).