My intuition is like… you get a topological circle by gluing the two ends of an interval together, but no subspace of the interval is homeomorphic to a circle. I’m not entirely sure that this sort of issue meaningfully impacts neural networks, but I don’t immediately see any reason why it wouldn’t?
I just opened up a topology textbook and found that I was using the word “subspace” while thinking about quotient topologies induced by a surjective function. (I wonder if there is a shorthand word for that like there is for induced subspace topologies? I think I’ll just say a “quotient space”.)
I’m getting the impression you are more familiar with this than me, but in case you want help recalling, or for the sake of other readers:
A subspace of topology X is a set S containing a subset of the elements of X that is “clipped”, so it does not containing the topological information of elements of X not found in S. Or more formally, the set of open sets of S is the set of each open set in X intersected with S.
A quotient topology Y from X wrt function f, where f is a surjective function from X to Y, is the topology I had been thinking of where f can be thought of as squishing, folding, or projecting points of a larger set into a smaller set. Formally, the open sets of Y is each set U for which the inverse image of U is an open set in X.
Thanks for catching that! I’m thinking I should change the article with some kind of note of correction.
( I’m not sure how embarrassed I should be about making this mistake. I think if I was a professor this would be quite embarassing. It’s less embarrassing as a recent BSc graduate who has only struggled through one course on topology, but is nevertheless very interested in it. Next time I’ll try to notice I should reference my textbook while writing the article. I think I got confused because I was thinking about both vector subspaces, which are topological subspaces of the larger vector space, but a different topology getting mapped into that vector subspace would be a quotient space not a subspace. )
Yeah, that’s probably part of it, although technically they are only the same with the quotient function being the very natural function of throwing away whatever component is not in the vector subspace to project straight down into that subspace, but this is not the only possible choice of function and so not the only possible space to get as a result.
I think all monotonic functions would give a homeomorphic space, but functions with discontinuities would not and I’m not sure about functions that are surjective but not injective. And functions that are not surjective fail the criteria for generating a quotient space.
Edit: I think maybe functions with discontinuities do still give a continuous space so long as they are surjective, which is required It would just break the vector properties between the two spaces, but that’s not required for a topological space. This is inspiring me to want to study topology more : )
Oh yeah, that makes sense. I wouldn’t want to make that assumption though, since activation functions are explicitly non-linear, otherwise the multiple layers can be multiplied together and a multi-layer perceptron would just be an indirect way of doing a single linear map.
My intuition is like… you get a topological circle by gluing the two ends of an interval together, but no subspace of the interval is homeomorphic to a circle. I’m not entirely sure that this sort of issue meaningfully impacts neural networks, but I don’t immediately see any reason why it wouldn’t?
You are of course correct!
I just opened up a topology textbook and found that I was using the word “subspace” while thinking about quotient topologies induced by a surjective function. (I wonder if there is a shorthand word for that like there is for induced subspace topologies? I think I’ll just say a “quotient space”.)
I’m getting the impression you are more familiar with this than me, but in case you want help recalling, or for the sake of other readers:
A subspace of topology X is a set S containing a subset of the elements of X that is “clipped”, so it does not containing the topological information of elements of X not found in S. Or more formally, the set of open sets of S is the set of each open set in X intersected with S.
A quotient topology Y from X wrt function f, where f is a surjective function from X to Y, is the topology I had been thinking of where f can be thought of as squishing, folding, or projecting points of a larger set into a smaller set. Formally, the open sets of Y is each set U for which the inverse image of U is an open set in X.
Thanks for catching that! I’m thinking I should change the article with some kind of note of correction.
( I’m not sure how embarrassed I should be about making this mistake. I think if I was a professor this would be quite embarassing. It’s less embarrassing as a recent BSc graduate who has only struggled through one course on topology, but is nevertheless very interested in it. Next time I’ll try to notice I should reference my textbook while writing the article. I think I got confused because I was thinking about both vector subspaces, which are topological subspaces of the larger vector space, but a different topology getting mapped into that vector subspace would be a quotient space not a subspace. )
I think maybe part of the confusion is that, when you’re working with vector spaces in particular, subspaces and quotient spaces are the same thing.
Yeah, that’s probably part of it, although technically they are only the same with the quotient function being the very natural function of throwing away whatever component is not in the vector subspace to project straight down into that subspace, but this is not the only possible choice of function and so not the only possible space to get as a result.
I think all monotonic functions would give a homeomorphic space, but functions with discontinuities would not and I’m not sure about functions that are surjective but not injective. And functions that are not surjective fail the criteria for generating a quotient space.
Edit: I think maybe functions with discontinuities do still give a continuous space so long as they are surjective, which is required It would just break the vector properties between the two spaces, but that’s not required for a topological space. This is inspiring me to want to study topology more : )
Oh, I meant in the category of (topological) vector spaces, which requires the quotient maps to be linear.
Oh yeah, that makes sense. I wouldn’t want to make that assumption though, since activation functions are explicitly non-linear, otherwise the multiple layers can be multiplied together and a multi-layer perceptron would just be an indirect way of doing a single linear map.