This is a really useful way of looking at deep neural networks. An extension to non-ReLU activation functions would be interesting, although for most, I feel that the only difference would be to make the boundaries between polytopes smooth (e.g., with Swish or GELU) or to make it so affine transformations only occur near the boundaries (e.g., with sigmoid, tanh, or tent).
Notice that outside of the inner sphere of polytopes, there will inevitably be an infinite region of unbounded polytopes, where the “leftover” affine transformations continue on to more extreme values forever. This may be a source of adversarial examples, where the network reaches very high confidence in predicting the wrong class with even seemingly small perturbations. I imagine that a lot of adversarial training involves tamping down these unbounded polytopes. Maybe constraining all neurons to output 0 when the input goes too far out of distribution would solve this.
Extending the polytope lens to Swish or GELU activation functions is, fortunately, the subject of a paper by Baleistriero & Baraniuk (2018) https://arxiv.org/abs/1810.09274
We wrote a few sentence about this at the end of Appendix C:
“In summary—smooth activation functions must be represented with a probabilistic spline code rather than a one-hot binary code. The corresponding affine transformation at the input point is then a linear interpolation of the entire set of affine transformations, weighted by the input point’s probability of belonging to each region.”
RE adversarial examples:
It certainly seems possible that adversarial examples might arise from polytopes far from the origin. My intuition for this is that some small norm perturbations will happen to be in directions that cross lots of polytope boundaries, which means that later activations will be in quite different directions. This is somewhat tautological, though, given the definition of polytope boundaries is literally defined by neurons turning on and off.
We hypothesize that adversarial attacks exploit the open space risk of classic monotonic activation functions. This paper introduces the tent activation function with bounded open space risk and shows that tents make deep learning models more robust to adversarial attacks. We demonstrate on the MNIST dataset that a classifier with tents yields an average accuracy of 91.8% against six white-box adversarial attacks, which is more than 15 percentage points above the state of the art.
Basically, causing all unbounded polytopes to have a zero-affine-transformation at extreme values improves adversarial robustness.
By the way, although the tent activation function prevents monotonic growth in the direction perpendicular to the decision hyperplane, I haven’t heard of any activation function that prevents the neuron from being active when the input goes too far out of distribution in a direction parallel to the hyperplane. It might be interesting to explore that angle.
This is a really useful way of looking at deep neural networks. An extension to non-ReLU activation functions would be interesting, although for most, I feel that the only difference would be to make the boundaries between polytopes smooth (e.g., with Swish or GELU) or to make it so affine transformations only occur near the boundaries (e.g., with sigmoid, tanh, or tent).
Notice that outside of the inner sphere of polytopes, there will inevitably be an infinite region of unbounded polytopes, where the “leftover” affine transformations continue on to more extreme values forever. This may be a source of adversarial examples, where the network reaches very high confidence in predicting the wrong class with even seemingly small perturbations. I imagine that a lot of adversarial training involves tamping down these unbounded polytopes. Maybe constraining all neurons to output 0 when the input goes too far out of distribution would solve this.
Thanks for your comment!
RE non-ReLU activation functions:
Extending the polytope lens to Swish or GELU activation functions is, fortunately, the subject of a paper by Baleistriero & Baraniuk (2018) https://arxiv.org/abs/1810.09274
We wrote a few sentence about this at the end of Appendix C:
“In summary—smooth activation functions must be represented with a probabilistic spline code rather than a one-hot binary code. The corresponding affine transformation at the input point is then a linear interpolation of the entire set of affine transformations, weighted by the input point’s probability of belonging to each region.”
RE adversarial examples:
It certainly seems possible that adversarial examples might arise from polytopes far from the origin. My intuition for this is that some small norm perturbations will happen to be in directions that cross lots of polytope boundaries, which means that later activations will be in quite different directions. This is somewhat tautological, though, given the definition of polytope boundaries is literally defined by neurons turning on and off.
Here is a paper that addresses using activation functions that bound the so-called “open space”:
Improved Adversarial Robustness by Reducing Open Space Risk via Tent Activations
According to the paper:
Basically, causing all unbounded polytopes to have a zero-affine-transformation at extreme values improves adversarial robustness.
By the way, although the tent activation function prevents monotonic growth in the direction perpendicular to the decision hyperplane, I haven’t heard of any activation function that prevents the neuron from being active when the input goes too far out of distribution in a direction parallel to the hyperplane. It might be interesting to explore that angle.