This post spurred some interesting thoughts, which I kicked around with Scott Viteri.
First, those histograms of counts of near-zero eigenvalues per run over 150 runs sure do smell a lot like asymptotic equipartition visualizations:
(The post also mentions AEP briefly; I’m not sure what the right connection is.)
Second, trying to visualize what this implies about the Rashomon set… it suggests the Rashomon set is very spiky. Like, picture a 3D ball, and then there’s a bunch of spikes coming off the ball which quickly thin into roughly-2D flat spiky things, and then those also branch into a bunch of sub-spikes which quickly thin into roughly-1D needles. And there’s so many of those approximately-lower-dimensional spikes that they fill even more volume than the ball. Sort of like the Mathematica Spikey, but spikier:
Anyway, this visualization reminds me strongly of visuals of 2- or 3-dimensional shapes which have nearly all the volume very close to the surface, like high-dimensional shapes do. (For a not-very-fancy example, see page 6 here.) It makes me wonder if there’s some natural way of viewing the entire Rashomon set as a simple very-high-dimensional solid which has kinda been “crumpled up and flattened out” into the relatively-lower-dimensional parameter space, while approximately preserving some things (like nearly all the volume being very close to the surface).
This post spurred some interesting thoughts, which I kicked around with Scott Viteri.
First, those histograms of counts of near-zero eigenvalues per run over 150 runs sure do smell a lot like asymptotic equipartition visualizations:
(The post also mentions AEP briefly; I’m not sure what the right connection is.)
Second, trying to visualize what this implies about the Rashomon set… it suggests the Rashomon set is very spiky. Like, picture a 3D ball, and then there’s a bunch of spikes coming off the ball which quickly thin into roughly-2D flat spiky things, and then those also branch into a bunch of sub-spikes which quickly thin into roughly-1D needles. And there’s so many of those approximately-lower-dimensional spikes that they fill even more volume than the ball. Sort of like the Mathematica Spikey, but spikier:
Anyway, this visualization reminds me strongly of visuals of 2- or 3-dimensional shapes which have nearly all the volume very close to the surface, like high-dimensional shapes do. (For a not-very-fancy example, see page 6 here.) It makes me wonder if there’s some natural way of viewing the entire Rashomon set as a simple very-high-dimensional solid which has kinda been “crumpled up and flattened out” into the relatively-lower-dimensional parameter space, while approximately preserving some things (like nearly all the volume being very close to the surface).