So in my research into machine learning algorithms that I can use to evaluate small block ciphers for cryptocurrency technologies, I have just stumbled upon a dimensionality reduction for tensors in tensor products of inner product spaces that according to my computer experiments exists, is unique, and which reduces a real tensor to another real tensor even when the underlying field is the field of complex numbers. I would not be too surprised if someone else came up with this tensor dimensionality reduction before since it has a rather simple description and it is in a sense a canonical tensor dimensionality reduction when we consider tensors as homogeneous non-commutative polynomials. But even if this tensor dimensionality reduction is not new, this dimensionality reduction algorithm belongs to a broader class of new algorithms that I have been studying recently such as LSRDRs.
Suppose that K is either the field of real numbers or the field of complex numbers. Let V1,…,Vn be finite dimensional inner product spaces over K with dimensions d1,…,dn respectively. Suppose that Vi has basis ei,1,…,ei,di. Given v∈V1⊗⋯⊗Vn, we would sometimes want to approximate the tensor v with a tensor that has less parameters. Suppose that (m0,…,mn) is a sequence of natural numbers with m0=mn=1. Suppose that Xi,j is a mi−1×mi matrix over the field K for 1≤i≤n and 1≤j≤di. From the system of matrices (Xi,j)i,j, we obtain a tensor T((Xi,j)i,j)=∑i1,…,inei1⊗⋯⊗ein⋅X1,i1…Xn,in. If the system of matrices (Xi,j)i,j locally minimizes the distance ∥v−T((Xi,j)i,j)∥, then the tensor T((Xi,j)i,j) is a dimensionality reduction of v which we shall denote by u.
Intuition: One can associate the tensor product V1⊗⋯⊗Vn with the set of all degree n homogeneous non-commutative polynomials that consist of linear combinations of the monomials of the form x1,i1⋯xn,in. Given, our matrices Xi,j, we can define a linear functional ϕ:V1⊗⋯⊗Vn→K by setting ϕ(p)=p((Xi,j)i,j). But by the Reisz representation theorem, the linear functional ϕ is dual to some tensor in V1⊗⋯⊗Vn. More specifically, ϕ is dual to T((Xi,j)i,j). The tensors of the form T((Xi,j)i,j) are therefore the
Advantages:
In my computer experiments, the reduced dimension tensor u is often (but not always) unique in the sense that if we calculate the tensor u twice, then we will get the same tensor. At least, the distribution of reduced dimension tensors u will have low Renyi entropy. I personally consider the partial uniqueness of the reduced dimension tensor to be advantageous over total uniqueness since this partial uniqueness signals whether one should use this tensor dimensionality reduction in the first place. If the reduced tensor is far from being unique, then one should not use this tensor dimensionality reduction algorithm. If the reduced tensor is unique or at least has low Renyi entropy, then this dimensionality reduction works well for the tensor v.
This dimensionality reduction does not depend on the choice of orthonormal basis ei,1,…,ei,di. If we chose a different basis for each Vi, then the resulting tensor u of reduced dimensionality will remain the same (the proof is given below).
If K is the field of complex numbers, but all the entries in the tensor v happen to be real numbers, then all the entries in the tensor u will also be real numbers.
This dimensionality reduction algorithm is intuitive when tensors are considered as homogeneous non-commutative polynomials.
Disadvantages:
This dimensionality reduction depends on a canonical cyclic ordering the inner product spaces V1,…,Vn.
Other notions of dimensionality reduction for tensors such as the CP tensor dimensionality reduction and the Tucker decompositions are more well-established, and they are obviously attempted generalizations of the singular value decomposition to higher dimensions, so they may be more intuitive to some.
The tensors of reduced dimensionality T((Xi,j)i,j) have a more complicated description than the tensors in the CP tensor dimensionality reduction.
Proposition: The set of tensors of the form ∑i1,…,ine1,i1⊗⋯⊗en,inX1,i1…Xn,in does not depend on the choice of bases (ei,1,…,ei,di)i.
Proof: For each i, let fi,1,…,fi,di be an alternative basis for Vi. Then suppose that ei,j=∑kui,j,kfi,k for each i,j. Then
A failed generalization: An astute reader may have observed that if we drop the requirement that mn=1, then we get a linear functional defined by letting
ϕ(p)=Tr(p((Xi,j)i,j)). This is indeed a linear functional, and we can try to approximate v using a the dual to ϕ, but this approach does not work as well.
So in my research into machine learning algorithms that I can use to evaluate small block ciphers for cryptocurrency technologies, I have just stumbled upon a dimensionality reduction for tensors in tensor products of inner product spaces that according to my computer experiments exists, is unique, and which reduces a real tensor to another real tensor even when the underlying field is the field of complex numbers. I would not be too surprised if someone else came up with this tensor dimensionality reduction before since it has a rather simple description and it is in a sense a canonical tensor dimensionality reduction when we consider tensors as homogeneous non-commutative polynomials. But even if this tensor dimensionality reduction is not new, this dimensionality reduction algorithm belongs to a broader class of new algorithms that I have been studying recently such as LSRDRs.
Suppose that K is either the field of real numbers or the field of complex numbers. Let V1,…,Vn be finite dimensional inner product spaces over K with dimensions d1,…,dn respectively. Suppose that Vi has basis ei,1,…,ei,di. Given v∈V1⊗⋯⊗Vn, we would sometimes want to approximate the tensor v with a tensor that has less parameters. Suppose that (m0,…,mn) is a sequence of natural numbers with m0=mn=1. Suppose that Xi,j is a mi−1×mi matrix over the field K for 1≤i≤n and 1≤j≤di. From the system of matrices (Xi,j)i,j, we obtain a tensor T((Xi,j)i,j)=∑i1,…,inei1⊗⋯⊗ein⋅X1,i1…Xn,in. If the system of matrices (Xi,j)i,j locally minimizes the distance ∥v−T((Xi,j)i,j)∥, then the tensor T((Xi,j)i,j) is a dimensionality reduction of v which we shall denote by u.
Intuition: One can associate the tensor product V1⊗⋯⊗Vn with the set of all degree n homogeneous non-commutative polynomials that consist of linear combinations of the monomials of the form x1,i1⋯xn,in. Given, our matrices Xi,j, we can define a linear functional ϕ:V1⊗⋯⊗Vn→K by setting ϕ(p)=p((Xi,j)i,j). But by the Reisz representation theorem, the linear functional ϕ is dual to some tensor in V1⊗⋯⊗Vn. More specifically, ϕ is dual to T((Xi,j)i,j). The tensors of the form T((Xi,j)i,j) are therefore the
Advantages:
In my computer experiments, the reduced dimension tensor u is often (but not always) unique in the sense that if we calculate the tensor u twice, then we will get the same tensor. At least, the distribution of reduced dimension tensors u will have low Renyi entropy. I personally consider the partial uniqueness of the reduced dimension tensor to be advantageous over total uniqueness since this partial uniqueness signals whether one should use this tensor dimensionality reduction in the first place. If the reduced tensor is far from being unique, then one should not use this tensor dimensionality reduction algorithm. If the reduced tensor is unique or at least has low Renyi entropy, then this dimensionality reduction works well for the tensor v.
This dimensionality reduction does not depend on the choice of orthonormal basis ei,1,…,ei,di. If we chose a different basis for each Vi, then the resulting tensor u of reduced dimensionality will remain the same (the proof is given below).
If K is the field of complex numbers, but all the entries in the tensor v happen to be real numbers, then all the entries in the tensor u will also be real numbers.
This dimensionality reduction algorithm is intuitive when tensors are considered as homogeneous non-commutative polynomials.
Disadvantages:
This dimensionality reduction depends on a canonical cyclic ordering the inner product spaces V1,…,Vn.
Other notions of dimensionality reduction for tensors such as the CP tensor dimensionality reduction and the Tucker decompositions are more well-established, and they are obviously attempted generalizations of the singular value decomposition to higher dimensions, so they may be more intuitive to some.
The tensors of reduced dimensionality T((Xi,j)i,j) have a more complicated description than the tensors in the CP tensor dimensionality reduction.
Proposition: The set of tensors of the form ∑i1,…,ine1,i1⊗⋯⊗en,inX1,i1…Xn,in does not depend on the choice of bases (ei,1,…,ei,di)i.
Proof: For each i, let fi,1,…,fi,di be an alternative basis for Vi. Then suppose that ei,j=∑kui,j,kfi,k for each i,j. Then
∑i1,…,ine1,i1⊗⋯⊗en,inX1,i1…Xn,in
=∑i1,…,in∑k1u1,i1,k1f1,i1⊗⋯⊗∑knun,in,knfn,inX1,i1…Xn,in
=∑k1,…,knf1,k1⊗⋯⊗fn,kn∑i1,…,inu1,i1,k1…un,in,knX1,i1…Xn,i,n
=∑k1,…,knf1,k1⊗⋯⊗fn,kn(∑i1u1,i1,k1X1,i1)…(∑inun,in,knXin). Q.E.D.
A failed generalization: An astute reader may have observed that if we drop the requirement that mn=1, then we get a linear functional defined by letting
ϕ(p)=Tr(p((Xi,j)i,j)). This is indeed a linear functional, and we can try to approximate v using a the dual to ϕ, but this approach does not work as well.