The polytope theory of neural networks (also known as the “spline theory of deep learning”) seeks to explain (ReLU) neural networks based on their piecewise linear regions. This gives a helpful intuition for how neural networks approximate functions, as well as a potential avenue for interpretability research. For anyone who wants to engage directly with the topic, I put together a small Colab notebook, with interactive 2D and 3D visuals.
Neural network polytopes (Colab notebook)
Link post
The polytope theory of neural networks (also known as the “spline theory of deep learning”) seeks to explain (ReLU) neural networks based on their piecewise linear regions. This gives a helpful intuition for how neural networks approximate functions, as well as a potential avenue for interpretability research. For anyone who wants to engage directly with the topic, I put together a small Colab notebook, with interactive 2D and 3D visuals.