I think of tensors as homogeneous non-commutative polynomials. But I have found a way of reducing tensors that does not do anything for 2-tensors but which seems to work well for n-tensors where n≥3. We can consider tensors as homogeneous non-commutative polynomials in a couple of different ways depending on whether we have tensors in V1⊗V2⊗⋯⊗Vn or if we have V⊗⋯⊗V=V⊗n. Let K∈{R,C}. Given a homogeneous non-commutative polynomial p(x1,…,xr) over the field K, consider the fitness function Lp,d,K:Md(K)r→[0,∞) defined by setting Lp,d,K(X1,…,Xr)=ρ(p(X1,…,Xr))1/n/ρ(X1⊗(X∗1)T+⋯+Xr⊗(X∗r)T)1/2 (there are generalizations of this fitness function) where ρ(X) denotes the spectral radius of the matrix X. By locally maximizing Lp,d,K(X1,…,Xr), the matrices (X1,…,Xr) encode information about the tensor p as long as p has degree n≥3. This L2,d-spectral radius tensor dimensionality reduction seems to be well-behaved in the sense that if one runs this L2,d-spectral radius tensor dimensionality reduction multiple times, one will reach the same local maximum, so the notion of a L2,d-spectral radius tensor dimensionality reduction is pseudodeterministic. The L_{2,d}-spectral radius tensor dimensionality reduction seems to also be well-behaved in other aspects as well, and I hope that this dimensionality reduction becomes very useful for machine learning and AI safety.
I think of tensors as homogeneous non-commutative polynomials. But I have found a way of reducing tensors that does not do anything for 2-tensors but which seems to work well for n-tensors where n≥3. We can consider tensors as homogeneous non-commutative polynomials in a couple of different ways depending on whether we have tensors in V1⊗V2⊗⋯⊗Vn or if we have V⊗⋯⊗V=V⊗n. Let K∈{R,C}. Given a homogeneous non-commutative polynomial p(x1,…,xr) over the field K, consider the fitness function Lp,d,K:Md(K)r→[0,∞) defined by setting Lp,d,K(X1,…,Xr)=ρ(p(X1,…,Xr))1/n/ρ(X1⊗(X∗1)T+⋯+Xr⊗(X∗r)T)1/2 (there are generalizations of this fitness function) where ρ(X) denotes the spectral radius of the matrix X. By locally maximizing Lp,d,K(X1,…,Xr), the matrices (X1,…,Xr) encode information about the tensor p as long as p has degree n≥3. This L2,d-spectral radius tensor dimensionality reduction seems to be well-behaved in the sense that if one runs this L2,d-spectral radius tensor dimensionality reduction multiple times, one will reach the same local maximum, so the notion of a L2,d-spectral radius tensor dimensionality reduction is pseudodeterministic. The L_{2,d}-spectral radius tensor dimensionality reduction seems to also be well-behaved in other aspects as well, and I hope that this dimensionality reduction becomes very useful for machine learning and AI safety.