Could you send a link to the “radial trace plots” work? I can’t seem to find anything.
I mostly agree with you about the loss landscape issue. It’s only looking at loss as a fn of a 2d subspace which is extremely limiting if you’re trying to make sense of a space with thousands or millions of dimensions.
in high dimensions it’s not clear that there are meaningful ways to visualize it
I think the question of whether or not it is possible to meaningfully visualize it is of prime importance to my work. My intuition is of course that it is possible. Very generally, my goal is to develop a “visual language” which serves as a memory aid, and set of tools for controlling “viewport rotation”, then in the same way that a 3d object can be understood as 3d by rotating it in your hands, a m<n dimensional subspace with properties of interest can be found in a region of a distribution, and the md distribution can be understood as md by rotating it and recalling it’s rotation dynamics with the help of interaction and visual hints.
But this seems like a difficult thing to communicate just with words. As I mention in this post, it is inspired by the work of Mingwei Li, particularly Grand Tour and UMAP Tour. If you haven’t glanced at those that might give you a much better intuition of the direction I’m thinking, but I have many additional ideas I need to document at some point, especially relating to a “visual language”.
I had not previously heard about SLT, but it looks like it is very relevant to my interests, so thank you very much for mentioning it. I will definitely work it into my study plans somewhere.
Could you send a link to the “radial trace plots” work? I can’t seem to find anything.
I mostly agree with you about the loss landscape issue. It’s only looking at loss as a fn of a 2d subspace which is extremely limiting if you’re trying to make sense of a space with thousands or millions of dimensions.
I think the question of whether or not it is possible to meaningfully visualize it is of prime importance to my work. My intuition is of course that it is possible. Very generally, my goal is to develop a “visual language” which serves as a memory aid, and set of tools for controlling “viewport rotation”, then in the same way that a 3d object can be understood as 3d by rotating it in your hands, a m<n dimensional subspace with properties of interest can be found in a region of a distribution, and the md distribution can be understood as md by rotating it and recalling it’s rotation dynamics with the help of interaction and visual hints.
But this seems like a difficult thing to communicate just with words. As I mention in this post, it is inspired by the work of Mingwei Li, particularly Grand Tour and UMAP Tour. If you haven’t glanced at those that might give you a much better intuition of the direction I’m thinking, but I have many additional ideas I need to document at some point, especially relating to a “visual language”.
I had not previously heard about SLT, but it looks like it is very relevant to my interests, so thank you very much for mentioning it. I will definitely work it into my study plans somewhere.
Sure, I sent you the file over discord.