This post will be about my machine learning algorithm where quadratic algebraic numbers including the golden ratio appear in the trained models. This demonstrates that these machine learning models behave mathematically which is exactly the kind of thing that we want for AI interpretability and AI safety.
This post will be about particular examples of -spectral radius dimensionality reductions (LSRDRs). I originally developed the notion of an LSRDR to evaluate the cryptographic security of block ciphers for cryptocurrency mining, but let’s talk about machine learning instead of cryptocurrency technologies here.
Also, the results that I have obtained in this proof have been obtained experimentally. I have not proven these results rigorously.
Dimensionality reduction: Let denote either the field of real or complex numbers. Suppose that are -matrices over and are -matrices over . Then define the operation by setting . Define the operator .
Define the -spectral radius similarity by setting
.
Here, the spectral radius is analogous to a dot product, and is analogous to the cosine similarity.
If are fixed matrices and , then we say that is an -SRDR if the similarity is locally maximized. Informally, the LSRDR is a collection of smaller matrices that approximates the collection of bigger matrices.
Lie algebras: A Lie algebra is a vector space over a field together with a bilinear operation that satisfies the identities:
for all
for all .
For example, if is an associative bilinear operation, then one can check that the commutator operation defined by is a Lie-bracket, and a Lie algebra should be thought of as a vector space with an abstract commutator operation.
Let denote the Lie algebra of -anti-symmetric matrices over where the Lie algebra operation is just the commutator For the rest of this post, we shall set . Then is a Lie algebra of dimension
Set and let be an orthonormal basis for . Use the standard orthonormal basis if you want, but it does not matter which basis you choose.
An observation about the spectrum: Let be the linear operators defined by setting for each . Let be an -SRDR of . It turns out that the spectrum eventually stabilizes in the sense that if we keep constant and set greater than around or so, then does not depend on whenever Therefore, let denote the multiset for sufficiently large Then is the multi-set
multiplied by a constant scaling factor. Here, the notion means that the eigenvalue has multiplicity .
The general pattern:
So if we want to get interesting experimental results about LSRDRs, then we just need to the following. We first select a finite dimensional inner product space with an interesting bilinear operation , but make sure that is not associative. We then select an orthonormal basis of and define linear operators by . Then take an LSRDR of and then the operators will have interesting spectra.
Testing if a number is quadratic:
After evaluating the spectra, I needed to first normalize the spectrum and then try to figure out exact values of the eigenvalues from their floating point approximation. This is easy to do for quadratic algebraic numbers. You just take the continued fraction representation of your number that you want to test. If the continued fraction representation terminates, then you have a rational number. And your continued fraction of a positive irrational repeats if and only if it is a solution to a quadratic equation with integer coefficients, and it is easy to find those coefficients from the continued fraction representation.
Are LSRDRs relevant to deep learning?
LSRDRs are linear models without all the layers that deep neural networks have. But I have been generalizing LSRDRs to deeper machine learning models that retain some but not all of the interesting mathematical properties of LSRDRs. I would therefore consider these investigations into LSRDRs as relevant to deep learning.
This post will be about my machine learning algorithm where quadratic algebraic numbers including the golden ratio appear in the trained models. This demonstrates that these machine learning models behave mathematically which is exactly the kind of thing that we want for AI interpretability and AI safety.
This post will be about particular examples of
Also, the results that I have obtained in this proof have been obtained experimentally. I have not proven these results rigorously.
Dimensionality reduction: Let
Define the
Here, the spectral radius
If
Lie algebras: A Lie algebra is a vector space
For example, if
Let
Set
An observation about the spectrum: Let
The general pattern:
So if we want to get interesting experimental results about LSRDRs, then we just need to the following. We first select a finite dimensional inner product space
Testing if a number is quadratic:
After evaluating the spectra, I needed to first normalize the spectrum and then try to figure out exact values of the eigenvalues from their floating point approximation. This is easy to do for quadratic algebraic numbers. You just take the continued fraction representation of your number that you want to test. If the continued fraction representation terminates, then you have a rational number. And your continued fraction of a positive irrational repeats if and only if it is a solution to a quadratic equation with integer coefficients, and it is easy to find those coefficients from the continued fraction representation.
Are LSRDRs relevant to deep learning?
LSRDRs are linear models without all the layers that deep neural networks have. But I have been generalizing LSRDRs to deeper machine learning models that retain some but not all of the interesting mathematical properties of LSRDRs. I would therefore consider these investigations into LSRDRs as relevant to deep learning.