Scaling down is an interesting challenge. To consider what happens in natural intelligent agents like humans and dogs, to the extent dogs are scaled down humans dogs reliably make mistakes about human values that we might not consider graceful failures. For example, a guard dog might bite an intruder it doesn’t recognize but a human might notice this “intruder” is wearing a police uniform and would not want to attack the intruder. The usual solution to this is to then either train the dog about police or restrain the dog in some way so that it can’t cause harm until a human approves its actions, such as by putting it on a leash. In AI this might mean containment for scaled down AI by more powerful systems (either humans or more powerful AI) to verify its actions before it takes them.
Scaling down is an interesting challenge. To consider what happens in natural intelligent agents like humans and dogs, to the extent dogs are scaled down humans dogs reliably make mistakes about human values that we might not consider graceful failures. For example, a guard dog might bite an intruder it doesn’t recognize but a human might notice this “intruder” is wearing a police uniform and would not want to attack the intruder. The usual solution to this is to then either train the dog about police or restrain the dog in some way so that it can’t cause harm until a human approves its actions, such as by putting it on a leash. In AI this might mean containment for scaled down AI by more powerful systems (either humans or more powerful AI) to verify its actions before it takes them.