Joseph Van Name

• Joseph Van Name 12 Oct 2025 20:03 UTC

  1 point

  0

  in reply to: joseph_c’s comment on: Us­ing com­plex polyno­mi­als to ap­prox­i­mate ar­bi­trary con­tin­u­ous functions

  I have not heard about the IBM paper until now. This is inspired by my personal experiments training (obviously classical) machine learning models.

  Suppose that $V_0,\dots ,V_n,W_0,\dots ,W_n$ are real or complex finite dimensional inner product spaces. Suppose that the training data consists of tuples of the form $(v_0,\dots ,v_n)$ where $v_0\in V_0,\dots ,v_n\in V_n$ are vectors. Let $W_0=V_0$ and let $B_j:V_{j+1}\times W_j\to W_{j+1}$ be bilinear for all $j$. Then let

  $L_v\left(w\right)=B_j(v,w)$ whenever $v\in V_{j+1},w\in W_j$. Then we define our polynomial by setting $p(v_0,\dots ,v_n)=L_{v_n}\dots L_{v_1}\left(v_0\right)$. In other words, my machine learning models are just compositions of bilinear mappings. In addition to wanting ${\text{Tr}}_U\left(\right[p(v_0,\dots ,v_n)\left]\right)$ to approximate the label, we also include regularization that makes the machine learning model pseudodeterministically trained so that if we train it twice with different initializations, we end up with the same trained model. Here, the machine learning model has $n$ layers, but the addition of extra layers gives us diminishing returns since bilinearity is close to linearity, so I still want to figure out how to improve the performance of such a machine learning model to match deep neural networks (if that is even feasible).

  I use quantum information theory for my experiments mainly because quantum information theory behaves well unlike neural networks.

• Joseph Van Name 12 Oct 2025 1:50 UTC

  1 point

  0

  in reply to: ChristianKl’s comment on: Joseph Van Name’s Shortform

  Proof-of-stake is still wasteful since it promotes pump and dump scams and causes people to waste their money on scam projects. If the creators are able to get their reward at the very beginning of a project, they will be more interested in short-term gains rather than a long-term token that will last. Humans are not psychologically/​socially equipped to invest in proof-of-stake cryptocurrencies since they tend to get scammed.

• Joseph Van Name 11 Oct 2025 5:35 UTC

  7 points

  1

  on: Joseph Van Name’s Shortform

  Bitcoin mining is a real-world example of a goal that people spend an enormous amount of resources to attain, but this goal is useless or at least horribly inefficient.

  Recall that the orthogonality thesis states that it is possible for an intelligent entity to have bad or dumb goals and that it is also possible for a not-so-intelligent entity to have good goals. I would therefore consider Bitcoin mining to be a real-world prominent example of the orthogonality thesis as it in a sense a dumb goal attained intelligently (though, this example is imperfect).

  Bitcoin’s mining algorithm consists of computing many SHA-256 hashes relentlessly. The Bitcoin miners are rewarded whenever they compute a suitable SHA-256 hash that is lower than the target. These SHA-256 hashes establish decentralized consensus about the state of the blockchain, and they distribute newly minted bitcoins. But besides this, computing so many SHA-256 hashes is nearly useless. Computing so many SHA-256 hashes consumes large quantities of energy and creates electronic waste.

  So what are some of the possible alternatives to Bitcoin mining? It seems like the best alternative that does not significantly change the nature of Bitcoin mining would be to replace SHA-256 mining with some other mining algorithm that serves some scientific purpose.

  This is more difficult than it seems because Bitcoin mining must satisfy a list of cryptographic properties. If the mining algorithm did not satisfy these cryptographic problems, then it might not be feasible for newly minted bitcoins to be dispersed every 10 minutes, and we may enter a scenario where a single entity with a secret algorithm or slightly faster hardware were to put all the blocks on the blockchain.

  Since Bitcoin mining must satisfy a list of cryptographic properties, it is difficult to come up with a more scientifically useful mining algorithm that satisfies these cryptographic properties. But in science, if there is a difficult problem, people should perform research on this scientific problem. While finding a useful cryptocurrency mining algorithm has its challenges, cryptocurrency mining algorithms are easy to produce since they can be made from cryptographic hash functions without requiring public key encryption or other advanced cryptographic algorithms, so difficulty seems more like an excuse rather than a legitimate reason not to investigate useful cryptocurrency mining algorithms. The cryptocurrency sector does not want to perform this research. I can think of several reasons why people refuse to support this sort of endeavor despite the great effort that people put into Bitcoin mining, but none of these reasons justify the lack of interest in useful cryptocurrency mining.

  The diminishing quality of cryptocurrency users:

  It seems like when altcoins were first being developed around 2014, people were much more interested in developing scientifically useful mining algorithms. But around 2017 when cryptocurrency really started to become popular, people simply wanted to make money from cryptocurrencies, yet they were not very interested in understanding how cryptocurrencies work or how to improve them.

  Mining algorithms with questionable scientific use:

  Some cryptocurrencies and proposals such as Primecoin and Gapcoin have more scientific mining algorithms, but these mining algorithms still have questionable usefulness. For example, the objective in Primecoin mining is to find suitable Cunningham chains. A Cunningham chain of the first kind is a sequence of prime numbers $(p_1,\dots ,p_n)$ where $p_{j+1}=2p_j+1$ whenever $1\le j<n$. The most interesting thing about Cunningham chains is that they can be used in cryptocurrency mining algorithms, but they are otherwise of minor importance to mathematics.

  These questionable mining algorithms are supposed to steer the cryptocurrency community into a more scientific direction, but in reality, they have just steered the cryptocurrency community towards using mining to perform mathematical calculations that not even mathematicians care that much about.

  Alternative solutions to the energy waste problem:

  Many people just want to do away with cryptocurrency mining in an altcoin by replacing it with proof-of-stake or some other consensus mechanism. This solution is attractive to the cryptocurrency creators since they want complete control over all the coins at the beginning of the project, and they just use the energy usage of cryptocurrency as a marketing strategy to get people interested in their project. But this solution should not be appealing to anyone who wants to use the cryptocurrency even if a cryptocurrency is better funded without much mining (of course, if mining is replaced with another consensus mechanism after all the coins have been created, then this objection does not stand). After all, Satoshi Nakamoto did not fund Bitcoin by selling bitcoins. There are other ways to fund a cryptocurrency project without alternate consensus mechanisms.

  Hostility against cryptocurrency technologies:

  It seems like many members of society are hostile against cryptocurrency technologies and hate people who own or are in any way interested in cryptocurrency. This sort of hostility is a very good reason to conduct as many transactions using just cryptocurrency since I do not want to deal with all of those Karens. But this hostility may have turned people away from researching useful cryptocurrency mining algorithms even though the usefulness would probably not benefit the cryptocurrency directly.

  Hardcore Bitcoiners:

  If Bitcoin mining were magically replaced with a useful mining algorithm, barely anything about Bitcoin would change. But in my experience, Bitcoiners do not see it this way. They are so stuck in their ways that they reject all altcoins.

  Conclusion:

  While cryptocurrencies have a lot of monetary value, they are not exactly powerhouses of innovation, nor do I find them extremely interesting on their own. But a good scientific mining algorithm would make them much more innovative and interesting.

Us­ing com­plex polyno­mi­als to ap­prox­i­mate ar­bi­trary con­tin­u­ous functions

• Joseph Van Name 22 Sep 2025 7:29 UTC

  2 points

  0

  on: Joseph Van Name’s Shortform

  In this post, we shall go over a way to produce mostly linear machine learning classification models that output probabilities for each possible label. These mostly linear models are pseudodeterministically trained (or pseudodeterministic for short) in the sense that if we train them multiple times with different initializations, we will typically get the same trained model (up-to-symmetry and miniscule floating point differences).

  The algorithms that I am mentioning in this post generalize to more complicated multi-layered algorithms in the sense that the multi-layered algorithms remain pseudodeterministic, but for simplicity, we shall stick to just linear operators here.

  Let $K$ denote either the field of real numbers, the field of complex numbers, or the division ring of quaternions. Let $U$ be a finite dimensional inner product space over $K$. The training data is a set $D$ of pairs $(u,v)$ where $u\in U$ and $v\in \{1,\dots ,n\}$ where $u$ is the machine learning model input and $v$ is the label. The machine learning model is trained to predict the label $v$ when given the input $u$. The trained model is a function $f$ that maps $U$ to the set of all probability vectors of length $n$, so the trained model actually gives the probabilities for each possible label.

  Suppose that $V_i$ is a finite dimensional inner product space over $K$ for each $i\in \{1,\dots ,n\}$. Then the domain of the fitness function consists of tuples $(A_1,\dots ,A_n)$ where each $A_i$ is a linear operator from $U$ to $V_i$. Let $p\in (0,1),r\in (0,\infty ),q\in (1,\infty )$, and let $\lambda \ge 0$. The parameter $p$ is the exponent while $\lambda$ is the regularization parameter. Define (almost total) functions $G,R,F:L(U,V_1)\times \cdots \times L(U,V_n)\to R$ by setting

  $G(A_1,\dots ,A_n)={\sum }_{(u,v)\in D}(\frac{\parallel A_{v}u{\parallel }^r}{\parallel A_{1}u{\parallel }^r+\cdots +\parallel A_{n}u{\parallel }^r}{)}^p/|D|$

  $R(A_1,\dots ,A_n)=({\sum }_{(u,v)\in D}\lambda \cdot \log (\parallel A_{v}u\parallel )/|D|)$

  $-\lambda \cdot \left(\log \right(\parallel A_1{\parallel }_q)+\cdots +\log (\parallel A_n{\parallel }_q\left)\right)/n$.

  Here, $\parallel \ast {\parallel }_q$ denotes the Schatten $q$-norm which can be defined by setting

  $\parallel A{\parallel }_q=\text{Tr}\left(\right(A{A}^{\ast }{)}^{q/2})$.

  Set $F=G+R$. Here, $F$ denotes our fitness function. The function $G$ what we really want to maximize, but unfortunately, $G$ is typically non-pseudodeterministic, so we need to add the regularization term $R$ to obtain pseudodeterminism. The regularization term $R$ also has the added effect of making $\parallel A_{v}u\parallel$ relatively large compared to the norm $\parallel A{\parallel }_q$ for training data points $(u,v)$. This may be useful in determining whether a pair should belong to either the training or test data in the first place.

  We observe that $F$ is $0$-homogeneous in the sense that$F(A_1,\dots ,A_n)=F(c{A}_1,\dots ,c{A}_n)$ for each non-zero scalar $c$ (in the quaternionic case, the scalars are just the real numbers).

  Suppose now that we have obtained a tuple $(A_1,\dots ,A_n)$ that maximizes the fitness $F(A_1,\dots ,A_n)$. Let $PV\left(n\right)$ denote the set of all probability vectors of length $n$. Then define an almost total function $f:U\to PV\left(n\right)$ by setting

  $f\left(u\right)=\frac{(\parallel A_{1}u{\parallel }^{r(1-p)},\dots ,\parallel A_{n}u{\parallel }^{r(1-p)})}{\parallel A_{1}u{\parallel }^{r(1-p)}+\cdots +\parallel A_{n}u{\parallel }^{r(1-p)}}.$

  If $(u,v)$ belongs to the training data set, then the $i$-th entry of $f\left(u\right)$ is the machine learning model’s estimate of the probability that $i=v$. I will let the reader justify this calculation of the probabilities.

  We can generalize the function $f$ to pseudodeterministically trained machine learning models with multiple layers by replacing the linear operators $A_1,\dots ,A_n$with some non-linear or multi-linear operators. Actually, there are quite a few ways of generalizing the fitness function $F$, and I have taken some liberty in the exact formulation for $F$.

  In addition to being pseudodeterministic, the fitness function $F$ has other notable desirable properties. For example, when maximizing $F$ using gradient ascent, one tends to converge to the local maximum at an exponential rate without needing to decay the learning rate.

• Joseph Van Name 7 Aug 2025 5:08 UTC

  1 point

  0

  in reply to: AnthonyC’s comment on: Con­sider showering

  Whether one takes or should take a cold shower or not depends on a lot of factors including whether one exercises, one’s health, one’s personal preferences, the air temperature, the cold water temperature, the humidity level, and the hardness of the shower water. But it seems like most people can’t fathom taking a cold shower simply because they are cold intolerant even though cold showers have many benefits.

  In addition to the practical benefits of cold showers, cold showers also may offer health benefits.

  Cold showers could improve one’s immune system (though we should).

  The Effect of Cold Showering on Health and Work: A Randomized Controlled Trial—PMC

  Cold showers may boost mood or alleviate depression.

  Scientific Evidence-Based Effects of Hydrotherapy on Various Systems of the Body—PMC

  Adapted cold shower as a potential treatment for depression—ScienceDirect

  Cold showers could also improve circulation and metabolism.

  Cold showers also offer other benefits.

  I always use the exhaust fan. It is never powerful enough to reduce the humidity faster than a warm shower increases the humidity. I also lock the door when taking a shower, and I do not know why anyone would take a shower without locking the door. Opening the door while showering just makes the rest of the home humid as well, and we can’t have that.

  I exercise daily, so out of habit, I always take a shower after I exercise, and most of my showers are after exercise. Even if I spend a few minutes cooling down after exercise, I need the shower to cool down even more, and by taking a warm shower, I cannot cool down as effectively, so I end up sweating after taking the shower. And I sometimes take my temperature after exercise and the shower and even after the shower, I tend to have a mouth temperature of 99.0 to 99.5 degrees Fahrenheit. I doubt that people who barely need to take a shower after exercising are doing much exercise or perhaps they are doing weights instead of cardio which produces less sweat, but in any case, I have never exercised and thought that I do not need a shower regardless of whether I am doing cardio, weights, or whatever.

  Soap scum left over after taking a cold shower seems to be a problem for you and for you only.

  Added 8/​20/​2025: And taking a hot shower produces all the condensation that helps all that mirror bacteria grow. Biological risk from the mirror world — LessWrong

• Joseph Van Name 4 Aug 2025 0:59 UTC

  −1 points

  0

  in reply to: AnthonyC’s comment on: Con­sider showering

  Instead of not taking showers, we should all take cold showers for many reasons.

  1. You already mentioned the energy usage which is a problem.

  2. Hot showers increase the relative humidity of the bathroom to 100 percent which is way too high. And that humidity means that you get a lot of condensation in the bathroom too. That is good only if you want the bathroom covered in mold.

  3. If you take a hot shower that fogs up all the mirrors, you are censoring your own nakeyness. Please don’t do that.

  4. I do not care if people shower daily. But people need to exercise daily. And after exercising, people need to shower. As a corollary, most of the time that people shower should be right after exercising. But after exercising, you are already warm, so the goal is to cool down. This means that everyone needs to take a cold shower.

  5. Cold intolerance is a major problem. People need to get over it. People who can’t tolerate a little bit of cold probably are intolerant in other areas as well. They cannot go mountain climbing because the mountains have snow on them. They can’t tolerate hot peppers. And they are afraid of spiders too.

• Joseph Van Name 12 Jun 2025 6:36 UTC

  3 points

  0

  on: Joseph Van Name’s Shortform

  I am going to share an algorithm that I came up with that tends to produce the same result when we run it multiple times with a different initialization. The iteration is not even guaranteed convergence since we are not using gradient ascent, but it typically converges as long as the algorithm is given a reasonable input. This suggests that the algorithm behaves mathematically and may be useful for things such as quantum error correction. After analyzing the algorithm, I shall use the algorithm to solve a computational problem.

  We say that an algorithm is pseudodeterministic if it tends to return the same output even if the computation leading to that output is non-deterministic (due to a random initialization). I believe that we should focus a lot more on pseudodetermistic machine learning algorithms for AI safety and interpretability since pseudodeterministic algorithms are inherently interpretable.

  Define $f\left(z\right)=3z^2-2z^3$ for all complex numbers $z$. Then $f\left(0\right)=0,f\left(1\right)=1,f^{\prime }\left(0\right)=f^{\prime }\left(1\right)=0$, and there are neighborhoods $U,V$ of $0,1$ respectively where if $x\in U$, then $f^N\left(x\right)\to 0$ quickly and if $y\in V$, then $f^N\left(y\right)\to 1$ quickly. Set $f^{\infty }={lim}_{N\to \infty }{f}^N$. The function $f^{\infty }$ serves as error correction for projection matrices since if $Q$ is nearly a projection matrix, then $f^{\infty }\left(Q\right)$ will be a projection matrix.

  Suppose that $K$ is either the field of real numbers, complex numbers or quaternions. Let $Z\left(K\right)$ denote the center of $K$. In particular, $Z\left(R\right)=R,Z\left(C\right)=C,Z\left(H\right)=R$.

  If $A_1,\dots ,A_r$ are $m\times n$-matrices, then define $\Phi (A_1,\dots ,A_r):M_n\left(K\right)\to M_m\left(K\right)$ by setting $\Phi (A_1,\dots ,A_r)={\sum }_{k=1}^r{A}_{k}X{A}_k^{\ast }$. Then we say that an operator of the form $\Phi (A_1,\dots ,A_r)$ is completely positive. We say that a $Z\left(K\right)$-linear operator $E:M_n\left(K\right)\to M_m\left(K\right)$ is Hermitian preserving if $E\left(X\right)$ is Hermitian whenever $X$ is Hermitian. Every completely positive operator is Hermitian preserving.

  Suppose that $E:M_n\left(K\right)\to M_n\left(K\right)$ is $Z\left(K\right)$-linear. Let $t>0$. Let $P_0\in M_n\left(K\right)$ be a random orthogonal projection matrix of rank $d$. Set $P_{N+1}=f^{\infty }(P_N+t\cdot E(P_N\left)\right)$ for all $N$. Then if everything goes well, the sequence $(P_N{)}_N$ will converge to a projection matrix $P$ of rank $d$, and the projection matrix $P$ will typically be unique in the sense that if we run the experiment again, we will typically obtain the exact same projection matrix $P$. If $E$ is Hermitian preserving, then the projection matrix $P$ will typically be an orthogonal projection. This experiment performs well especially when $E$ is completely positive or at least Hermitian preserving or nearly so. The projection matrix $P$ will satisfy the equation $P\cdot E\left(P\right)=E\left(P\right)\cdot P=P\cdot E\left(P\right)\cdot P$.

  In the case when $E$ is a quantum channel, we can easily explain what the projection $P$ does. The operator $P$ is a projection onto a subspace of complex Euclidean space that is particularly well preserved by the channel $E$. In particular, the image $\text{Im}\left(P\right)$ is spanned by the top $d$ eigenvectors of $E\left(P\right)$. This means that if we send the completely mixed state $P/d$ through the quantum channel $E$ and we measure the state $E\left(P/d\right)$ with respect to the projective measurement $(P,I-P)$, then there is an unusually high probability that this measurement will land on $P$ instead of $I-P$.

  Let us now use the algorithm that obtains $P$ from $E$ to solve a problem in many cases.

  If $x$ is a vector, then let $\text{Diag}\left(x\right)$ denote the diagonal matrix where $x$ is the vector of diagonal entries, and if $X$ is a square matrix, then let $\text{Diag}\left(X\right)$ denote the diagonal of $X$. If $x$ is a length $n$ vector, then $\text{Diag}\left(x\right)$ is an $n\times n$-matrix, and if $X$ is an $n\times n$-matrix, then $\text{Diag}\left(X\right)$ is a length $n$ vector.

  Problem Input: An $n\times n$-square matrix $A$ with non-negative real entries and a natural number $d$ with $1\le d<n$.

  Objective: Find a subset $B\subseteq \{1,\dots ,n\}$ with $|B|=d$ and where if $x=A\cdot {\chi }_B$, then the $d$ largest entries in $x$ are the values $x\left[b\right]$ for $b\in B$.

  Algorithm: Let $E$ be the completely positive operator defined by setting $E\left(X\right)=\text{Diag}(A\cdot \text{Diag}(X\left)\right)$. Then we run the iteration using $E$ to produce an orthogonal projection $P$ with rank $d$. In this case, the projection $P$ will be a diagonal projection matrix with rank $d$ where $\text{diag}\left(P\right)={\chi }_B$ and where $B$ is our desired subset of $\{1,\dots ,n\}$.

  While the operator $P$ is just a linear operator, the pseudodeterminism of the algorithm that produces the operator $P$ generalizes to other pseudodeterministic algorithms that return models that are more like deep neural networks.

• Joseph Van Name 4 Jun 2025 21:01 UTC

  1 point

  0

  in reply to: Logan Riggs’s comment on: Spec­tral radii di­men­sion­al­ity re­duc­tion com­puted with­out gra­di­ent calculations

  I would have thought that a fitness function that is maximized using something other than gradient ascent and which can solve NP-complete problems at least in the average case would be worth reading since that means that it can perform well on some tasks but it also behaves mathematically in a way that is needed for interpretability. The quality of the content is inversely proportional to the number of views since people don’t think the same way as I do.

  Wheels on the Bus | @CoComelon Nursery Rhymes & Kids Songs

  Stuff that is popular is usually garbage.

  But here is my post about the word embedding.

  Interpreting a matrix-valued word embedding with a mathematically proven characterization of all optima — LessWrong

  And I really do not want to collaborate with people who are not willing to read the post. This is especially true of people in academia since universities promote violence and refuse to acknowledge any wrongdoing. Universities are the absolute worst.

  Instead of engaging with the actual topic, people tend to just criticize stupid stuff simply because they only want to read about what they already know or what is recommended by their buddies; that is a very good way not to learn anything new or insightful. For this reason, even the simplest concepts are lost on most people.

• Joseph Van Name 28 May 2025 19:22 UTC

  8 points

  0

  in reply to: Logan Riggs’s comment on: Spec­tral radii di­men­sion­al­ity re­duc­tion com­puted with­out gra­di­ent calculations

  In this post, the existence of a non-gradient based algorithm for computing LSRDRs is a sign that LSRDRs behave mathematically and are quite interpretable. Gradient ascent is a general purpose optimization algorithm that works in the case when there is no other way to solve the optimization problem, but when there are multiple ways of obtaining a solution to an optimization problem, the optimization problem is behaving in a way that should be appealing to mathematicians.

  LSRDRs and similar algorithms are pseudodeterministic in the sense that if we train the model multiple times on the same data, we typically get identical models. Pseudodeterminism is a signal of interpretability for several reasons that I will go into more detail in a future post:

  1. Pseudodeterministic models do not contain any extra random or even pseudorandom information that is not contained in the training data already. This means that when interpreting these models, one does not have to interpret random information.

  2. Pseudodeterministic models inherit the symmetry of their training data. For example, if we train a real LSRDR using real symmetric matrices, then the projection $P$ will itself by a symmetric matrix.

  3. In mathematics, a well-posed problem is a problem where there exists a unique solution to the problem. Well-posed problems behave better than ill-posed problems in the sense that it is easier to prove results about well-posed problems than it is to prove results about ill-posed problems.

  In addition to pseudodeterminism, in my experience, LSRDRs are quite interpretable since I have interpreted LSRDRs already in a few posts:

  Interpreting a dimensionality reduction of a collection of matrices as two positive semidefinite block diagonal matrices — LessWrong

  When performing a dimensionality reduction on tensors, the trace is often zero. — LessWrong

  I have Generalized LSRDRs so that they are starting to behave like deeper neural networks. I am trying to expand the capabilities of generalized LSRDRs so they behave more like deep neural networks, but I still have some work to expand their capabilities while retaining pseudodeterminism. In the meantime, generalized LSRDRs may still function as narrow AI for specific problems and also as layers in AI.

  Of course, if we want to compare capabilities, we should also compare NNs to LSRDRs at tasks such as evaluating the cryptographic security of block ciphers, solving NP-complete problems in the average case, etc.

  As for the difficulty of this post, it seems like that is the result of the post being mathematical. But going through this kind of mathematics so that we obtain inherently interpretable AI should be the easier portion of AI interpretability. I would much rather communicate about the actual mathematics than about how difficult the mathematics is.

Spec­tral radii di­men­sion­al­ity re­duc­tion com­puted with­out gra­di­ent calculations

• Joseph Van Name 14 May 2025 5:42 UTC

  1 point

  0

  on: Joseph Van Name’s Shortform

  In this post, we shall describe 3 related fitness functions with discrete domains where the process of maximizing these functions is pseudodeterministic in the sense that if we locally maximize the fitness function multiple times, then we typically attain the same local maximum; this appears to be an important aspect of AI safety. These fitness functions are my own. While these functions are far from deep neural networks, I think they are still related to AI safety since they are closely related to other fitness functions that are locally maximized pseudodeterministically that more closely resemble deep neural networks.

  Let $K$ denote a finite dimensional algebra over the field of real numbers together with an adjoint operation $\ast$ (the operation $\ast$ is a linear involution with $(xy{)}^{\ast }=y^{\ast }{x}^{\ast }$). For example, $K$ could be the field of real numbers, complex numbers, quaternions, or a matrix ring over the reals, complex, or quaternions. We can extend the adjoint $\ast$ to the matrix ring $M_r\left(K\right)$ by setting $(x_{i,j}{)}_{i,j}^{\ast }=(x_{j,i}^{\ast }{)}_{i,j}$.

  Let $n$ be a natural number. If $A_1,\dots ,A_r\in M_n\left(K\right),X_1,\dots ,X_r\in M_d\left(K\right)$, then define

  $\Gamma (A_1,\dots ,A_r;X_1,\dots ,X_r):M_{n,d}\left(K\right)\to M_{n,d}\left(K\right)$ by setting $\Gamma (A_1,\dots ,A_r;X_1,\dots ,X_r)\left(X\right)=A_{1}X{X}_1^{\ast }+\cdots +A_{r}X{X}_r^{\ast }$.

  Suppose now that $1\le d<n$. Then let $S_d\subseteq M_{n,n}\left(K\right)$ be the set of all $0,1$-diagonal matrices with $d$ many $1$’s on the diagonal. We observe that each element in $S_d$ is an orthogonal projection. Define fitness functions $F_d,G_d,H_d:S_d\to R$ by setting

  $F_d\left(P\right)=\rho \left(\Gamma \right(A_1,\dots ,A_r;P{A}_{1}P,\dots ,P{A}_{r}P\left)\right)$,

  $G_d\left(P\right)=\rho \left(\Gamma \right(P{A}_{1}P,\dots ,P{A}_{r}P;P{A}_{1}P,\dots ,P{A}_{r}P\left)\right)$, and

  $H_d\left(P\right)=\frac{F_d(P{)}^2}{G_d\left(P\right)}$. Here, $\rho$ denotes the spectral radius.

  $F_d\left(P\right)$ is typically slightly larger than $G_d\left(P\right)$, so these three fitness functions are closely related.

  If $P,Q\in S_d$, then we say that $Q$ is in the neighborhood of $P$ if $Q$ differs from $P$ by at most 2 entries. If $F$ is a fitness function with domain $S_d$, then we say that $(P,F(P\left)\right)$ is a local maximum of the function $F$ if $F\left(P\right)\ge F\left(Q\right)$ whenever $Q$ is in the neighborhood of $P$.

  The path from initialization to a local maximum $(P_s,F(P_s\left)\right)$ for will be a sequence $(P_0,\dots ,P_s)$ where $P_j$ is always in the neighborhood of $P_{j-1}$ and where $F\left(P_j\right)\ge F\left(P_{j-1}\right)$ for all $j$ and the length of the path will be $s$ and where $P_0$ is generated uniformly randomly.

  Empirical observation: Suppose that $F\in \{F_d,G_d,H_d\}$. If we compute a path from initialization to local maximum for $F$, then such a path will typically have length less than $n$. Furthermore, if we locally maximize $F$ multiple times, we will typically obtain the same local maximum each time. Moreover, if $P_F,P_G,P_H$ are the computed local maxima of $F_d,G_d,H_d$ respectively, then $P_F,P_G,P_H$ will either be identical or differ by relatively few diagonal entries.

  I have not done the experiments yet, but one should be able to generalize the above empirical observation to matroids. Suppose that $M$ is a basis matroid with underlying set $\{1,\dots ,n\}$ and where $|A|=d$ for each $A\in M$. Then one should be able to make the same observation about the fitness functions $F_d{|}_M,G_d{|}_M,H_d{|}_M$ as well.

  We observe that the problems of maximizing $F_d,G_d,H_d$ are all NP-complete problems since the clique problems can be reduced to special cases of maximizing $F_d,G_d,H_d$. This means that the problems of maximizing $F_d,G_d,H_d$ can be sophisticated problems, but this also means that we should not expect it to be easy to find the global maxima for $F_d,G_d,H_d$ in some cases.

• Joseph Van Name 10 May 2025 5:57 UTC

  1 point

  0

  on: Joseph Van Name’s Shortform

  This is a post about some of the machine learning algorithms that I have been doing experiments with. These machine learning models behave quite mathematically which seems to be very helpful for AI interpretability and AI safety.

  Sequences of matrices generally cannot be approximated by sequences of Hermitian matrices.

  Suppose that $A_1,\dots ,A_r$ are $n\times n$-complex matrices and $X_1,\dots ,X_r$ are $d\times d$-complex matrices. Then define a mapping $\Gamma (A_1,\dots ,A_r;X_1,\dots ,X_r):M_{n,d}\left(C\right)\to M_{n,d}\left(C\right)$ by $\Gamma (A_1,\dots ,A_r;X_1,\dots ,X_r)\left(X\right)=A_{1}X{X}_1^{\ast }+\cdots +A_{r}X{X}_r^{\ast }$ for all $X$. Define

  $\Phi (A_1,\dots ,A_r)=\Gamma (A_1,\dots ,A_r;A_1,\dots ,A_r)$. Define the $L_2$

  -spectral radius by setting ${\rho }_2(A_1,\dots ,A_r)=\rho \left(\Phi \right(A_1,\dots ,A_r){)}^{1/2}$. Define the $L_2$-spectral radius similarity between $(A_1,\dots ,A_r)$ and $(X_1,\dots ,X_r)$ by

  $\parallel (A_1,\dots ,A_r)\simeq (X_1,\dots ,X_r){\parallel }_2$

  $=\frac{\rho \left(\Gamma \right(A_1,\dots ,A_r;X_1,\dots ,X_r\left)\right)}{{\rho }_2(A_1,\dots ,A_r){\rho }_2(X_1,\dots ,X_r)}$.

  The $L_2$-spectral radius similarity is always in the interval $[0,1]$. if $n=d$, $A_1,\dots ,A_r$ generates the algebra of $n\times n$-complex matrices, and $X_1,\dots ,X_r$ also generates the algebra of $n\times n$-complex matrices, then $\parallel (A_1,\dots ,A_r)\simeq (X_1,\dots ,X_r){\parallel }_2=1$ if and only if there are $C,\lambda$ with $A_j=\lambda C{X}_j{C}^{-1}$ for all $j$.

  Define ${\rho }_{2,d}^H(A_1,\dots ,A_r)$ to be the supremum of

  ${\rho }_2(A_1,\dots ,A_r)\cdot \parallel (A_1,\dots ,A_r)\simeq (X_1,\dots ,X_r){\parallel }_2$

  where $X_1,\dots ,X_r$ are $d\times d$-Hermitian matrices.

  One can get lower bounds for ${\rho }_{2,d}^H(A_1,\dots ,A_r)$ simply by locally maximizing $\parallel (A_1,\dots ,A_r)\simeq (X_1,\dots ,X_r){\parallel }_2$ using gradient ascent, but if one locally maximizes this quantity twice, one typically gets the same fitness level.

  Empirical observation/​conjecture: If $(A_1,\dots ,A_r)$ are $n\times n$-complex matrices, then ${\rho }_{2,n}^H(A_1,\dots ,A_r)={\rho }_{2,d}^H(A_1,\dots ,A_r)$ whenever $d\ge n$.

  The above observation means that sequences of $n\times n$-matrices $(A_1,\dots ,A_r)$ are fundamentally non-Hermitian. In this case, we cannot get better models of $(A_1,\dots ,A_r)$ using Hermitian matrices larger than the matrices $(A_1,\dots ,A_r)$ themselves; I kind of want the behavior to be more complex instead of doing the same thing whenever $d\ge n$

  , but the purpose of modeling $(A_1,\dots ,A_r)$ as Hermitian matrices is generally to use smaller matrices and not larger matrices.

  This means that the function ${\rho }_{2,d}^H$ behaves mathematically.

  Now, the model $(X_1,\dots ,X_r)$ is a linear model of $(A_1,\dots ,A_r)$ since the mapping $A_j\mapsto X_j$ is the restriction of a linear mapping, so such a linear model should be good for a limited number of tasks, but the mathematical behavior of the model $(X_1,\dots ,X_r)$ generalizes to multi-layered machine learning models.

• Joseph Van Name 3 May 2025 9:38 UTC

  9 points

  0

  on: Joseph Van Name’s Shortform

  In this post, I will post some observations that I have made about the octonions that demonstrate that the machine learning algorithms that I have been looking at recently behave mathematically and such machine learning algorithms seem to be highly interpretable. The good behavior of these machine learning algorithms is in part due to the mathematical nature of the octonions and also the compatibility with the octonions and the machine learning algorithm. To be specific, one should think of the octonions as encoding a mixed unitary quantum channel that looks very close to the completely depolarizing channel, but my machine learning algorithms work well with those sorts of quantum channels and similar objects.

  Suppose that $K$ is either the field of real numbers, complex numbers, or quaternions.

  If $A_1,\dots ,A_r\in M_m\left(K\right),B_1,\dots ,B_r\in M_n\left(K\right)$ are matrices, then define an superoperator $\Gamma (A_1,\dots ,A_r;B_1,\dots ,B_r):M_{m,n}\left(K\right)\to M_{m,n}\left(K\right)$

  by setting$\Gamma (A_1,\dots ,A_r;B_1,\dots ,B_r)\left(X\right)=A_{1}X{B}_1^{\ast }+\cdots +A_{r}X{B}_r^{\ast }$

  (the domain and range of )and define $\Phi (A_1,\dots ,A_r)=\Gamma (A_1,\dots ,A_r;A_1,\dots ,A_r)$. Define the L_2-spectral radius similarity $\parallel (A_1,\dots ,A_r)\simeq (B_1,\dots ,B_r){\parallel }_2$ by setting

  $\parallel (A_1,\dots ,A_r)\simeq (B_1,\dots ,B_r){\parallel }_2$

  $=\frac{\rho \left(\Gamma \right(A_1,\dots ,A_r;B_1,\dots ,B_r\left)\right)}{\rho \left(\Phi \right(A_1,\dots ,A_r\left){)}^{1/2}\rho \right(\Phi (B_1,\dots ,B_r){)}^{1/2}}$ where $\rho$ denotes the spectral radius.

  Recall that the octonions are the unique (up-to-isomorphism) 8 dimensional real inner product space $V$ together with a bilinear binary operation $\ast$ such that$\parallel x\ast y\parallel =\parallel x\parallel \cdot \parallel y\parallel$ and $1\ast x=x\ast 1=x$ for all $x,y\in V$.

  Suppose that $e_1,\dots ,e_8$ is an orthonormal basis for $V$. Define operators $(A_1,\dots ,A_8)$ by setting $A_{i}v=e_j\ast v$. Now, define operators $(B_1,\dots ,B_{64})$ up to reordering by setting $\{B_1,\dots ,B_{64}\}=\{A_i\otimes A_j:i,j\in \{1,\dots ,8\left\}\right\}$.

  Let $d$ be a positive integer. Then the goal is to find complex symmetric $d\times d$-matrices $(X_1,\dots ,X_{64})$ where $\parallel (A_1,\dots ,A_{64})\simeq (X_1,\dots ,X_{64}){\parallel }_2$ is locally maximized. We achieve this goal through gradient ascent optimization. Since we are using gradient ascent, I consider this to be a machine learning algorithm, but the function mapping $A_j$ to $X_j$ is a linear transformation, so we are training linear models here (we can generalize this fitness function to one where we train non-linear models though, but that takes a lot of work if we want the generalized fitness functions to still behave mathematically).

  Experimental Observation: If $1\le d\le 8$, then we can easily find complex symmetric matrices $(X_1,\dots ,X_{64})$ where $\parallel (A_1,\dots ,A_{64})\simeq (X_1,\dots ,X_{64}){\parallel }_2$ is locally maximized and where $\parallel (A_1,\dots ,A_{64})\simeq (X_1,\dots ,X_{64}){\parallel }_2^2=(2d+6)/64=(d+3)/32.$

  If $7\le d\le 16$, then we can easily find complex symmetric matrices $(X_1,\dots ,X_{64})$ where $\parallel (A_1,\dots ,A_{64})\simeq (X_1,\dots ,X_{64}){\parallel }_2$ is locally maximized and where$\parallel (A_1,\dots ,A_{64})\simeq (X_1,\dots ,X_{64}){\parallel }_2^2=(2d+4)/64=(d+2)/32.$

  .

• Joseph Van Name 28 Apr 2025 7:28 UTC

  1 point

  0

  on: Joseph Van Name’s Shortform

  Here are some observations about the kind of fitness functions that I have been running experiments on for AI interpretability. The phenomena that I state in this post are determined experimentally without a rigorous mathematical proof and they only occur some of the time.

  Suppose that $F:X\to [-\infty ,\infty )$ is a continuous fitness function. In an ideal universe, we would like for the function $F$ to have just one local maximum. If $F$ has just one local maximum, we say that $F$ is maximized pseudodeterministically (or simply pseudodeterministic). At the very least, we would like for there to be just one real number of the form $F\left(x\right)$ for local maximum $(x,F(x\left)\right)$. In this case, all local maxima will typically be related by some sort of symmetry. Pseudodeterministic fitness function seem to be quite interpretable to me. If there are many local maximum values and the local maximum value that we attain after training depends on things such as the initialization, then the local maximum will contain random/​pseudorandom information independent of the training data, and the local maximum will be difficult to interpret. A fitness function with a single local maximum value behaves more mathematically than a fitness function with many local maximum values, and such mathematical behavior should help with interpretability; the only reason I have been able to interpret pseudodeterminisitic fitness functions before is that they behave mathematically and have a unique local maximum value.

  Set $O=F^{-1}\left[\right(-\infty ,\infty \left)\right]=X\setminus F^{-1}\left[\right\{-\infty \left\}\right]$. If the set $O$ is disconnected (in a topological sense) and if $L$ behaves differently on each of the components of $L$, then we have literally shattered the possibility of having a unique local maximum, but in this post, we shall explore a case where each component of $O$ still has a unique local maximum value.

  Let $m_0,\dots ,m_n$ be positive integers with $m_0=m_n=1$ and where $m_1\ge 1,\dots ,m_{n-1}\ge 1$. Let $r_0,\dots ,r_{n-1}$ be other natural numbers. The set $X$ is the collection of all tuples $A=(A_{i,j}{)}_{i,j}$ where each $A_{i,j}$ is a real $m_{i+1}\times m_i$-matrix and where the indices range from $i\in \{0,\dots ,n-1\},j\in \{1,\dots ,r_i\}$ and where $(A_{i,j}{)}_j$ is not identically zero for all $i$.

  The training data is a set $\Sigma$ that consists of input/​label pairs $(u,v)$ where $v\in \{-1,1\}$ and where $u=(u_0,\dots ,u_{n-1})$ such that each $u_i$ is a subset of $\{1,\dots ,r_i\}$ for all $i$ (i.e.$\Sigma$ is a binary classifier where $u$ is the encoded network input and $v$ is the label).

  Define $W(u,A)=\left({\sum }_{j\in u_{n-1}}{A}_{n-1,j}\right)\dots \left({\sum }_{j\in u_0}{A}_{0,j}\right)$. Now, we define our fitness level by setting

  $F\left(A\right)={\sum }_{(u,v)\in \Sigma }\log \left(|W\right(u,A\left)|\right)/|\Sigma |-{\sum }_i\log (\parallel {\sum }_j{A}_{i,j}{A}_{i,j}^{\ast }{\parallel }_p)/2$

  $=E\left(\log \right(|W(u,A)|\left)\right)-{\sum }_i\log (\parallel {\sum }_j{A}_{i,j}{A}_{i,j}^{\ast }{\parallel }_p)/2$ where the expected value is with respect to selecting an element $(u,v)\in \Sigma$ uniformly at random. Here, $\parallel \ast {\parallel }_p$ is a Schatten $p$-norm which is just the ${\ell }_p$-norm of the singular values of the matrix. Observe that the fitness function $F$ only depends on the list $(u:(u,v)\in \Sigma )$, so $F$ does not depend on the training data labels.

  Observe that $O=X\setminus {\bigcup }_{u\in \Sigma }\{A\in X:W(u,A)=0\}$ which is a disconnected open set. Define a function $f:O\to \{-1,1{\}}^{\Sigma }$ by setting $f\left(A\right)=\left(W\right(u,A\left)/|W\right(u,A)|{)}_{(u,v)\in \Sigma }$. Observe that if $x,y$ belong to the same component of $O$, then $f\left(x\right)=f\left(y\right)$.

  While the fitness function $F$ has many local maximum values, the function $F$ seems to typically have at most one local maximum value per component. More specifically, for each $({\alpha }_i{)}_{i\in \Sigma }$, the set $f^{-1}\left[\right\{\left({\alpha }_i{)}_{i\in \Sigma }\right\}]$ seems to typically be a connected open set where $F$ has just one local maximum value (maybe the other local maxima are hard to find, but if thye are hard to find, they are irrelevant).

  Let $\Omega =f^{-1}\left[\right\{\left(v{)}_{(u,v)\in \Sigma }\right]$. Then $\Omega$ is a (possibly empty) open subset of $O$, and there tends to be a unique (up-to-symmetry) $A_0\in \Omega$ where $F\left(A_0\right)$ is locally maximized. This unique $A_0$ is the machine learning model that we obtain when training on the data set $\Sigma$. To obtain $A_0$, we first perform an optimization that works well enough to get inside the open set $\Omega$. For example, to get inside $\Omega$, we could try to maximize the fitness function ${\sum }_{(u,v)\in \Sigma }\arctan (v\cdot W(u,A\left)\right)$. We then maximize $F$ inside the open set $\Omega$ to obtain our local maximum.

  After training, we obtain a function $f$ defined by $f\left(u\right)=W(u,A_0)$. Observe that the function $f$ is a multi-linear function. The function $f$ is highly regularized, so if we want better performance, we should tone down the amount of regularization, but this can be done without compromising pseudodeterminism. The function $f$ has been trained so that $f\left(u\right)/|f\left(u\right)|=v$ for each $(u,v)\in \Sigma$ but also so that $|f\left(u\right)|$ is large compared to what we might expect whenever $(u,v)\in \Sigma$. In other words, $f$ is helpful in determining whether $(u,v)$ belongs to $\Sigma$ or not since one can examine the magnitude and sign of $f\left(u\right)$.

  In order to maximize AI safety, I want to produce inherently interpretable AI algorithms that perform well on difficult tasks. Right now, the function $f$ (and other functions that I have designed) can do some machine learning tasks, but they are not ready to replace neural networks, but I have a few ideas about how to improve my AI algorithms performance without compromising pseudodeterminism. I do not believe that pseudodeterministic machine learning will increase AI risks too much because when designing these pseudodeterministic algorithms, we are trading some (but hopefully not too much) performance for increased interpretability, but this tradeoff is good for safety by increasing interpretability without increasing performance.

• Joseph Van Name 24 Apr 2025 10:08 UTC

  3 points

  0

  on: Joseph Van Name’s Shortform

  This post gives an example of some calculations that I did using my own machine learning algorithm. These calculations work out nicely which indicates that the machine learning algorithm I am using is interpretable (and it is much more interpretable than any neural network would be). These calculations show that one can begin with old mathematical structures and produce new mathematical structures, and it seems feasible to completely automate this process to continue to produce more mathematical structures. The machine learning models that I use are linear, but it seems like we can get highly non-trivial results simply by iterating the procedure of obtaining new structures from old using machine learning.

  I made a similar post to this one about 7 months ago, but I decided to revisit this experiment with more general algorithms and I have obtained experimental results which I think look nice.

  To illustrate how this works, we start off with the octonions. The octonions consists of an 8-dimensional inner product space $V$ together with a bilinear operation $\ast$ and a unit $1\in V$ where $1\ast v=v\ast 1=v$ for all $v\in V$ and where $\parallel u\ast v\parallel =\parallel u\parallel \cdot \parallel v\parallel$ for all $u,v\in V$. The octonions are uniquely determined up to isomorphism from these properties. The operation $\ast$ is non-associative, but the $\ast$ is closely related to the quaternions and complex numbers. If we take a single element in $v\in V\setminus \text{Span}\left(1\right)$, then $\{1,v\}$ generates a subalgebra of $(V,\ast )$ isomorphic to the field of complex numbers, and if $u,v\in V$ and $\{1,u,v\}$ are linearly independent, then $\{1,u,v,u\ast v\}$ spans a subalgebra of $V$ isomorphic to the division ring of quaternions. For this reason, one commonly thinks of the octonions as the best way to extend the division ring of quaternions to a larger algebraic structure in the same way that the quaternions extend the field of complex numbers. But since the octonions are non-associative, they cannot be used to construct matrices, so they are not as well-known as the quaternions (and the construction of the octonions is more complicated too)

  Suppose now that $\{e_0,e_1,\dots ,e_7\}$ is an orthonormal basis for the division ring of octonions with $e_0=1$. Then define matrices $A_0,\dots ,A_7:V\to V$ by setting $A_{j}v=e_j\ast v$ for all $j$. Our goal is to transform $(A_0,\dots ,A_7)$ into other tuples of matrices that satisfy similar properties.

  If $(A_1,\dots ,A_r),(B_1,\dots ,B_r)$ are matrices, then define the $L_2$

  -spectral radius similarity between $(A_1,\dots ,A_r)$ and $(B_1,\dots ,B_r)$ as

  $\parallel (A_1,\dots ,A_r)\simeq (B_1,\dots ,B_r){\parallel }_2=$

  $\frac{\rho (A_1\otimes \overline{B_1}+\cdots +A_r\otimes \overline{B_r})}{\rho (A_1\otimes \overline{A_1}+\cdots +A_r\otimes \overline{A_r}{)}^{1/2}\cdot \rho (B_1\otimes \overline{B_1}+\cdots +B_r\otimes \overline{B_r}{)}^{1/2}}$

  where $\rho$ denotes the spectral radius, $\otimes$ is the tensor product, and $\overline{X}$ is the complex conjugate of $X$ applied elementwise.

  Let $d\in \{1,\dots ,8\}$, and let $F_d,G_d,H_d$ denote the maximum value of the fitness level $8\cdot \parallel (A_0,\dots ,A_7)\simeq (X_0,\dots ,X_7){\parallel }^2$ such that each $X_j$ is a complex $d\times d$ anti-symmetric matrix ($X=-X^T$), a complex $d\times d$ symmetric matrix ($X=X^T$), and a complex $d\times d$-Hermitian matrix ($X=X^{\ast }$) respectively.

  The following calculations were obtained through gradient ascent, so I have no mathematical proof that the values obtained are actually correct.

  $G_1=2$, $H_1=1$

  $G_2=3$, $H_2=3$

  $F_3=1+\sqrt{3}$ , $G_3=3.5$, $H_3=1+2\sqrt{2}$

  $F_4=4$ , $G_4=4$, $H_4=1+3\sqrt{2}$

  $F_5=(5+\sqrt{13})/2$, $G_5=4.5$, $H_5\approx 5.27155841$

  $F_6=5$ , $G_6=5$, $H_6=3+2\sqrt{2}$

  $F_7=6$ , $G_7=2+2\sqrt{3}\approx 5.4641$, $H_7=1+2\sqrt{7}$

  $F_8=7$ , $G_8=6$, $H_8=7$

  Observe that with at most one exception, all of these values $F_d,G_d,H_d$ are algebraic half integers. This indicates that the fitness function that we maximize to produce $F_d,G_d,H_d$ behaves mathematically and can be used to produce new tuples $(X_1,\dots ,X_r)$ from old ones $(A_1,\dots ,A_r)$. Furthermore, an AI can determine whether something notable is going on with the new tuple $(X_1,\dots ,X_r)$ in several ways. For example, if $\parallel (A_1,\dots ,A_r)\simeq (X_1,\dots ,X_r){\parallel }^2$ has low algebraic degree at the local maximum, then $(X_1,\dots ,X_r)$ is likely notable and likely behaves mathematically (and is probably quite interpretable too).

  The good behavior of $F_d,G_d,H_d$ demonstrates that the octonions are compatible with the $L_2$-spectral radius similarity. The operators $(A_0,\dots ,A_7)$ are all orthogonal, and one can take the tuple $(A_0,\dots ,A_7)$ as a mixed unitary quantum channel that is very similar to the completely depolarizing channel. The completely depolarizing channel completely mixes every quantum state while the mixture of orthogonal mappings $(A_0,\dots ,A_7)$ completely mixes every real state. The $L_2$-spectral radius similarity works very well with the completely depolarizing channel, so one should expect for the $L_2$-spectral radius similarity to also behave well with the octonions.