Density of brush and trees in a forest. Too many and light is blocked, Too few and there’s room to grow. Fires are an example of a perturbation, a big enough fire will destroy all the trees and push us into a new equilibrium.
A computer system which is in a stable state of performance until it’s overwhelmed with requests and crashes. Pushing the “ball” out of this equilibrium amounts to a denial of service attack.
Take an image classifier and an image correctly classified as a cat, perturbing the cat image with small amounts of random noise does not change the classification but large amounts does.
Regarding (3) suppose we want to change the equilibrium (make the network classify the image correctly). Perhaps you could apply the same methods used to create adversarial examples to find the smallest perturbation that makes the network classify the image correctly, then maybe you can do interpretability stuff on the perturbation? or look for systemic errors by finding patterns in the perturbations? To make this work well you’d need some vision meaningful definition of “smallest perturbation” I think
Density of brush and trees in a forest. Too many and light is blocked, Too few and there’s room to grow. Fires are an example of a perturbation, a big enough fire will destroy all the trees and push us into a new equilibrium.
A computer system which is in a stable state of performance until it’s overwhelmed with requests and crashes. Pushing the “ball” out of this equilibrium amounts to a denial of service attack.
Take an image classifier and an image correctly classified as a cat, perturbing the cat image with small amounts of random noise does not change the classification but large amounts does.
Regarding (3) suppose we want to change the equilibrium (make the network classify the image correctly). Perhaps you could apply the same methods used to create adversarial examples to find the smallest perturbation that makes the network classify the image correctly, then maybe you can do interpretability stuff on the perturbation? or look for systemic errors by finding patterns in the perturbations? To make this work well you’d need some vision meaningful definition of “smallest perturbation” I think