joseph_c

Karma: 180

joseph_c 4 Sep 2025 3:18 UTC
3 points
0
on: Remap your caps lock key
For Linux users on US Keyboards, you might want to try making Caps Lock the multi key (also called the compose key). On Cinnamon this can be done by going to Keyboard > Layouts > Options… > Position of Compose key, and other desktop environments probably have similar settings.

This lets me type umlauts (ä, ü, ö), foreign currencies (£, €, ¥,), copyright/trademark (©, ™), and a bunch of other stuff. For example, “ü” is made by typing Compose, u, and ” in sequence. I also added the line
```
<Multi_key> <backslash> : "λ"
```
to my ~/.XCompose file so that I can type λ efficiently; this is useful when writing Lisp code.

joseph_c 1 Sep 2025 4:44 UTC
4 points
1
on: My AI Predictions for 2027
Why the Architecture of LLMs Makes Them Bad at Deep Thinking: They’re Too Wide
GPT-3 is 96 layers deep (where each layer is only a few “operations”), but 49,152 “neurons” wide at the widest. This is an insanely wide, very shallow network. This is for good reasons: wide networks are easier to run efficiently on GPUs, and apparently deep networks are hard to train.
I don’t find this argument compelling, because the human brain is much wider and possibly shallower than GPT-3. Humans have a conscious reaction time of about 200 milliseconds, while neurons take about 1ms to influence their neighbors, meaning an upper bound on the depth of a conscious reaction is 200 neurons.

joseph_c 29 Aug 2025 22:06 UTC
3 points
0
on: The Gabian History of Mathematics

Thanks to Hilbert’s list, a lot of progress was made toward formalising proofs, logic, consistency and other similar concepts.

Hmmm… I don’t think this accurately describes the era of mathematics starting around the 1920s. In fact, I would argue that the correct era would be about 1910-1937, starting with Russell and Whitehead’s Principia Mathematica and ending with the proof that the lambda calculus is exactly as powerful as the Turing machine.

This era was focused on applying logic to computation. It saw the development of type theory and the foundations of computation. Some aspects, like the halting problem, were related to logical consistency, but I think the more important breakthroughs had to do with formalizing computation.

joseph_c 24 Aug 2025 19:16 UTC
20 points
−1
on: Shorter Tokens Are More Likely
So, I had a hypothesis last night that training on a different scoring rule might solve this problem (because it could encourage uncertain probabilities to be lower, and thus it would be easier to filter them out without making short tokens undeservedly more likely).

I ended up forking your code, and this morning trained an LLM on the Shakespeare dataset using the $α$ -ReLU loss (from Speeding Up Entmax). The $α$ -ReLU loss is a proper scoring rule based off of the Tsallis entropy.

My results were the following:

Letter Score Rule Top-K Control Test T=1.0 Test T=0.8 Test T=0.1

c Cross-entropy 200 3.09% 5.29% 6.64% 7.26%

c $α$ -ReLU 200 3.41% 4.66% 4.75% 4.33%

c $α$ -ReLU $\infty$ 3.41% 3.87% 4.00% 3.49%

All models were trained for 1500 iterations. The control was trained without the special C-tokenizer, while the tests were. In the paper, $α$ -ReLU is parameterized by $α$ which controls the Tsallis exponent, and $τ$ , which is a constant shift in the ReLU. I chose to set $α = 1.5$ and $τ = 0$ for all the experiments (which is the default for the $α$ -ReLU python library).

The $α$ -ReLU trained neural network does NOT seem to exhibit the same trend of lower temperatures leading to higher probability of words starting with “c”. And, when the top-k is set to $\infty$ , most of the difference between it and the control also disappears.

So, perhaps we should just be training with a different scoring rule!

You can find my fork at https://github.com/cooljoseph1/nanoGPT-tokenizer-experiment/.

EDIT: I realized I was calculating temperature unfairly for the $α$ -ReLU sampling. After fixing it, it is actually worse than the cross-entropy trained network:

Letter Score Rule Top-K Control Test T=1.0 Test T=0.8 Test T=0.1

c Cross-entropy 200 3.09% 5.29% 6.64% 7.26%

c $α$ -ReLU 200 3.41% 4.66% 7.54% 31.66%

c $α$ -ReLU $\infty$ 3.41% 3.87% 5.94% 31.66%

Letter	Score Rule	Top-K	Control	Test T=1.0	Test T=0.8	Test T=0.1
c	Cross-entropy	200	3.09%	5.29%	6.64%	7.26%
c	$α$ -ReLU	200	3.41%	4.66%	4.75%	4.33%
c	$α$ -ReLU	$\infty$	3.41%	3.87%	4.00%	3.49%

Letter	Score Rule	Top-K	Control	Test T=1.0	Test T=0.8	Test T=0.1
c	Cross-entropy	200	3.09%	5.29%	6.64%	7.26%
c	$α$ -ReLU	200	3.41%	4.66%	7.54%	31.66%
c	$α$ -ReLU	$\infty$	3.41%	3.87%	5.94%	31.66%

joseph_c 24 Aug 2025 1:36 UTC
4 points
0
on: Shorter Tokens Are More Likely
I remember reading a paper about how aiming for a certain entropy per token made LLMs sound more human. I think it might have been this paper? This marginalization of later tokens might be the reason why—aiming for a certain entropy would encourage lower probability tokens more often than a fixed temperature would while still avoiding “noisy” tokens.

joseph_c 11 Aug 2025 0:53 UTC
4 points
0
on: Run-time Steering Can Surpass Post-Training: Reasoning Task Performance
This seems especially easy to do with RWKV. Perhaps I will try that out sometime this week, but probably not. I did something like that (but not specifically for reasoning) two years ago for MIT’s Splash.

My code from Splash is on Github if anyone else wants to give it a try before I (maybe) get to it. The code is very bad, though. It’s just something I wrote for myself for the most part (so it doesn’t have any kind of documentation, and it has lots of dead code lying around). So, it might be easier to just write your own code.

joseph_c 21 Jul 2025 5:12 UTC
1 point
0
in reply to: Brendan Long’s comment on: LLMs Can’t See Pixels or Characters
It worked for me on the second attempt (also using ChatGPT).

Attempt 1:

Spell out “riedenheit”, i.e. peace, letter by letter with spaces separating them

Sure! Here’s “Zufriedenheit” (German for peace or contentment) spelled out with spaces:
Z U F R I E D E N H E I T

Attempt 2:

Spell out riedenheit, i.e. peace, letter by letter with spaces separating them

Sure! Here’s “riedenheit” spelled out letter by letter with spaces:
R I E D E N H E I T

joseph_c 14 Jul 2025 15:47 UTC
1 point
0
on: Arrow theorem is an artifact of ordinal preferences
Suppose every citizen truthfully provides a normalized cardinal valuation for each alternative: that is, valuations are positive, and for each citizen the sum of the valuations for all alternatives is one. Then, the sum of the valuations by alternative is a consistent measure of the social value for every alternative (Dhillon, Bouveret and Lemaître, 1999).
Why not have the sum of the squares of the valuations sum to one? By the Cauchy-Schwartz inequality, one’s utility is maximized by reporting a vector of valuations parallel to one’s true preferences, so you can get rid of the requirement for honesty.

joseph_c 2 Mar 2025 7:54 UTC
1 point
0
on: [PAPER] Jacobian Sparse Autoencoders: Sparsify Computations, Not Just Activations
I’m a little confused about which computation you’re trying to sparsify. The paper seems to be written in the context of the technique where one uses sparse autoencoders to extract “features” from the embedding space of large language models which are hopefully interpretable. (Please correct me if I’m wrong about that!)
The goal would seem to be, then, to sparsify the computation of the language model. However, the method in your paper seems to sparsify the computation of the autoencoders themselves, not the language model. Shouldn’t the goal be to sparsify the language model’s computation? If so, why not use weight pruning? What is JSAE better at?

joseph_c 28 Dec 2024 21:36 UTC
2 points
0
in reply to: James Camacho’s comment on: If all trade is voluntary, then what is “exploitation?”
Isn’t $\beta$ proportional to the inverse temperature, and so should be smaller now (with easier, more frequent trading)?

joseph_c 28 Oct 2024 17:45 UTC
8 points
0
on: On Shifgrethor
A math textbook leaving certain results as an exercise for the reader?
I think this is usually actually one of (1) the author not wanting to write out the proof (because it’s boring/tedious) or (2) a proof that would make a good exercise because it is easy enough if you understand the big ideas (and coming up with good exercises is not always easy).

joseph_c 3 Oct 2024 22:10 UTC
1 point
1
in reply to: Raemon’s comment on: “Slow” takeoff is a terrible term for “maybe even faster takeoff, actually”
Gradual/Sharp

joseph_c 23 Aug 2024 0:34 UTC
4 points
0
on: joseph_c’s Shortform
I recently came across Backpack Language Models and wanted to share it in case any AI interpretability people have not seen it. (I have yet to see this posted on LessWrong.)

The main difference between a backpack model and an LLM is that it enforces a much stricter rule to map inputs’ embeddings to output logits. Most LLMs allow the output logits to be an arbitrary function of the inputs’ embeddings; a backpack model requires the output logits to be a linear transformation of a linear combination of the input embeddings. The weights for this linear combination are parameterized by a transformer.

The nice thing about backpack models is that they are somewhat easier to interpret/edit/control: The output logits are a linear combination of the inputs’ embeddings, so you can directly observe how changing the embeddings changes the outputs.

joseph_c’s Shortform

joseph_c23 Aug 2024 0:34 UTC

3 points

1 comment LW link

joseph_c 27 Jun 2024 3:13 UTC
2 points
−1
on: Childhood and Education Roundup #6: College Edition
Most students, 48 percent, claimed to be Native American on their application....
According to Intelligent.com Managing Editor Kristen Scatton, the prevalence of applicants who claim Native American ancestry is possibly due to the popular narrative that for many Americans, a small percentage of their DNA comes from a Native American tribe.
Maybe these students are purposely misinterpreting “Native American” to be someone who was born and raised in the United States, perhaps with ancestors born and raised in the US as well. This is actually the older sense of the term “Native American”, found, for example, in the name of the Native American Party back in the mid-1800s.

joseph_c 31 May 2024 16:53 UTC
3 points
1
on: Less Anti-Dakka
includeIt is written in More Dakka:
If something is a good idea, you need a reason to not try doing more of it.
Taken at face value, it implies the contrapositive:
If something is a bad idea, you need a reason to not try doing less of it.
This is not the contrapositive. It is not even the opposite.

joseph_c 30 Apr 2024 4:17 UTC
1 point
0
on: UDT1.01: The Story So Far (1/10)
Parfit’s Hitchhiker: You’re stranded in the desert and Omega comes up. It will give you a ride out of the desert iff it predicts you’d give it 10,000 dollars upon reaching civilization again. You get a ride. When in civilization again, do you go over to the bank and withdraw some money? Well, policies which pay up in this specific situation get (value of a life − 10,000 dollars) more than policies which don’t pay in this specific situation, which just die.
Why is this called Parfit’s Hitchhiker? Who is the Parfit it is referring to? Where was this scenario first written up? (I’m trying to dig up the original reference.)

joseph_c 21 Mar 2024 4:00 UTC
1 point
0
in reply to: the gears to ascension’s comment on: I was raised by devout Mormons, AMA &| Offer Advice
Not for Mormons. They don’t believe in an omnipresent God.

joseph_c 19 Mar 2024 21:46 UTC
64 points
52
on: Increasing IQ by 10 Points is Possible
Well, what are your actual steps? Or is this just advertisement?

joseph_c 13 Mar 2024 17:43 UTC
2 points
0
on: I was raised by devout Mormons, AMA &| Offer Advice
Do you still live in Utah?
Did your family cut you off?
Do you know about [r/exmormon](https://old.reddit.com/r/exmormon/)?

joseph_c

Why the Architecture of LLMs Makes Them Bad at Deep Thinking: They’re Too Wide

joseph_c’s Shortform