joseph_c

Karma: 252

joseph_c 31 Dec 2025 2:36 UTC
2 points
0
on: The origin of rot
You say that “Perhaps for several millennia, [rot didn’t exist]” and then provide a date about 3300 years ago about when rot might potentially have began. You also say that several millennia is a “very, very long time”. A millennium is 1,000 years. Do you think that life has only existed on Earth for a few thousand years?

joseph_c 14 Dec 2025 21:37 UTC
2 points
0
on: The Axiom of Choice is Not Controversial
Now, where did the weirdness come from here. Well, to me it seems clear that really it came from the fact that the reals can be built out of a bunch of shifted rational numbers, right?

I think the weirdness comes from trying to assign a real number measure, instead of allowing infinitesimals. I’ve never understood why infinite sets are readily accepted, but infinitesimal/infinite measures are not.

EDIT: To explain my reasoning more, suppose you were Pythagoras and your student came to you and drew a geometric diagram with lengths not in a ratio of whole numbers. You have two options here:
1. You can declare that not all lengths are commensurate. Not every ratio of lengths results in a number.
2. You can extend your number system.
Finding the right extension is not an easy problem. Should we extend the numbers to allow square roots (including nesting), but nothing else? This suffices for geometry. But it’s actually more useful to use something like a Cauchy sequence completion: Let any sequence of rational numbers that gets closer and closer together “converge” to a real number. Historically, extending your system of numbers has been what has worked.

When we come across an “immeasurable” set, this to me feels like the same kind of problem. Perhaps we don’t yet have a general consensus on what the “right” extension is to infinitesimals/infinities. However, there clearly are some sets with infinitesimal measure, like the set you constructed. We should figure out a way to give that set infinitesimal measure, not just call it immeasurable.

joseph_c 9 Dec 2025 1:32 UTC
2 points
1
in reply to: GenericModel’s comment on: Gödel’s Ontological Proof
To try to clarify why it felt self-referential. I think there’s a self-reference regardless of whether you talk about classes or not, but it’s more obvious if you talk about classes.

I think the correct mathematical term is “Impredicativity”, not “self-referential”, but I’m no logician.

joseph_c 9 Dec 2025 1:26 UTC
1 point
0
in reply to: GenericModel’s comment on: Gödel’s Ontological Proof
If I were to try to translate this into classes instead of properties, it would look like, “The class of perfect properties contains the property of being every perfect property in this class”. That seems self-referential to me.

joseph_c 9 Dec 2025 1:19 UTC
1 point
0
in reply to: GenericModel’s comment on: Gödel’s Ontological Proof
An object is defined to be G if it has every perfect property, and then G is assumed (by axiom) to be a perfect property, hence being G requires being G. Now that I think about it a bit more, though, this seems more like a greatest-lower-bound situation than a Russell’s paradox situation.

joseph_c 9 Dec 2025 1:15 UTC
1 point
0
in reply to: JBlack’s comment on: Gödel’s Ontological Proof
I didn’t know that coherent logic was actually a term logicians used! I’m not a logician myself—I’m a programmer. Thanks for letting me know!

joseph_c 9 Dec 2025 1:11 UTC
1 point
0
in reply to: GenericModel’s comment on: Gödel’s Ontological Proof
The reason I was saying it looked like a type error was because of the self reference. I’m extremely wary of self-referential definitions because you can quickly run into problems like Russell’s paradox. It seems like sometimes it’s okay to have self-referential definitions (like the greatest lower bound), but I’m not confident that Axiom 4 actually avoids those problems.

joseph_c 9 Dec 2025 0:19 UTC
1 point
0
in reply to: GenericModel’s comment on: Gödel’s Ontological Proof
Could you please link me to that formalization?

joseph_c 9 Dec 2025 0:16 UTC
18 points
11
on: Gödel’s Ontological Proof
I think if you replace the word “God” with “top” and “perfect” with “highest”, it would be much more clear what the proof actually implies about the real world: Very little.

Definition 0: Say that ψ is higher than φ if □ ∀x φ(x) → ψ(x).

Axiom 1: A property higher than a highest property is also highest.
Axiom 2: The negation of a highest property is not highest.
Axiom 3: If a property is highest, then in some world there exists an object with that property.
Definition 1: An object is top if it is every highest property.
Axiom 4: The property “top” is highest. Note: This looks like a a type error to me.
Definition 2: A property φ is highest-generating for an object x if it is true of x, and every highest property of x follows from φ in every possible world.
Definition 3: Superior object. (I changed your name for this as well.) An object x is “superior” if for every highest-generating property of x, every world has an object with that property (not necessarily x).
Axiom 5: A highest property is highest in every world.
Axiom 6: The property “superior” is highest. Probably also a type error.

Sheesh, there sure are a lot of axioms. At this point, I’m not even sure we have a ~~coherent logic~~ sane logic anymore! Especially with those potential type errors.

The end result we prove is that in every world, there exists a top object.

When you divorce the “God” and “perfect” language from the axioms, we don’t really get anything that implies much about the real world or Christianity, do we?

joseph_c 9 Dec 2025 0:15 UTC
1 point
0
on: Gödel’s Ontological Proof
Axioms 4 and 6 look like type errors to me. Could you please explain how they are not?

joseph_c 30 Nov 2025 2:57 UTC
3 points
0
in reply to: lsusr’s comment on: Traditional Food
Thank you for the more detailed recipe! I’m not going to switch to only eating this meal (don’t worry about my vitamin intake!). It’s just something I plan to add to my rotation because it’s easy to make and healthy.

joseph_c 30 Nov 2025 0:42 UTC
3 points
0
on: Traditional Food
Is the bean dish supposed to be more like soup or chili? I tried it out, and it was rather soupy. Maybe I added too much broth. What portions did you use for the canned tomatoes, beans, and broth?

joseph_c 23 Nov 2025 23:59 UTC
3 points
0
in reply to: Kaj_Sotala’s comment on: The Enemy Gets The Last Hit
What’s going on is something like adverse selection in an auction. In reality, chess is a solvable game, so that in perfect play win/loss/draw probabilities are all 0 or 1 for every board state. However, you don’t know what these probabilities are, so you use a model. A naive player might just want to play the action which takes them to the board state they model as having the highest probability of winning. However, this fails to take into account the fact that one’s model can be wrong, and so one will tend to pick actions which lead to board states which one mispredicts as being better than they actually are.

If one can accurately model how inaccurate one’s model tends to be at certain board states, then one can do fine without ending on one’s opponent’s move by discounting board states one models as modelling poorly. This is nontrivial for humans to do correctly, however.

Instead, a heuristic one can use is to just let one’s opponent make the last move in one’s search tree. This gives one a lower bound on how good each board state is (an optimistic guess for one’s opponent is a pessimistic guess for one’s self), so one’s choices of board states will not be catastrophically biased. By catastrophically biased, I mean that in chess it’s much easier to wreck your game than to make a stunningly clever move which causes you to win, so that being too optimistic is much worse than being too pessimistic.

joseph_c 15 Nov 2025 7:26 UTC
1 point
0
on: Lambda Calculus Prior
This is not at all my specialty, but might the problem go away if instead of directly passing in the next term into your lambda calculus machine, you first quote it? By “quoting”, I mean converting it to a representation that the lambda calculus machine can inspect, like the QUOTE operator in Lisp.

joseph_c 27 Oct 2025 0:35 UTC
2 points
0
in reply to: abstractapplic’s comment on: Results of “Experiment on Bernoulli processes”
I don’t have any immediate plans to do something like this again, but I’ll make use of your offer if I end up doing another challenge. Thanks!

Results of “Experiment on Bernoulli processes”

joseph_c26 Oct 2025 21:47 UTC

9 points

2 comments4 min readLW link

joseph_c 15 Oct 2025 16:34 UTC
14 points
2
on: It will cost you nothing to “bribe” a Utilitarian
I think the problem here is the assumption that there is only one AI company. If there are multiple AI companies and they don’t form a trust, then they need to bid against each other to acquire safety researchers, right? This is like in economics where if you are the only person selling bread, you can sell it for $ε$ less than its value to any given customer, but if there are multiple people selling bread you need to sell it for $ε$ minus your competitors’ prices.

joseph_c 14 Oct 2025 22:34 UTC
2 points
0
on: Discrete Generative Models
When generating, we will sample uniformly, which requires
$K L (p_{batch mean} ∥ Uniform) = constant - H (p_{batch mean})$
bits to describe. This gives the loss
$∥ final image - target ∥^{2} - iters \sum iter = 1 H (p_{batch mean}^{iter}) .$
You should be using a MSE between uniform and $p_{batch mean}$ instead of a KL divergence in the loss. The batch mean is only an estimate for what you truly want, which is the mean over the entire dataset (or perhaps, all possible images, but there’s not much you can do about that). If you directly substitute it in for the dataset mean in the KL divergence, the resulting gradients are not unbiased estimators of the correct gradients. On the other hand, if you use a MSE loss instead, the gradients are unbiased estimators of the correct gradients for the MSE. In the limit as the dataset marginals approach the uniform distribution, the gradients of the KL divergence will be parallel to the gradients of the MSE gradients, so it’s okay to use a MSE instead.

joseph_c 14 Oct 2025 4:19 UTC
2 points
0
on: Discrete Generative Models
Right now in your code, you only calculate reconstruction error gradients for the very last step.
```
if random.random() > delta:
    loss = loss + (probs * err).sum(dim=1).mean()
    break
```
Pragmatically, it is more efficient to calculate reconstruction error gradients at every step and just weight by the probability of being the final image:
```
loss = loss + (1 - delta) * (probs * err).sum(dim=1).mean()
if random.random() > delta:
    break
```

joseph_c 14 Oct 2025 2:46 UTC
2 points
1
on: Discrete Generative Models

Although not mentioned in Yang’s paper, we can instead select images proportional to $p \propto e^{- β \cdot error (x, target)}$ …

This gives the loss $∥ final image - target ∥^{2} - iters \sum iter = 1 H (p_{batch mean}^{iter}) .$ If we want an infinite-depth model, we can choose to sometimes halt, but usually sample another image with probability $δ$ (for ‘discount factor’). Also, as the depth increases, the images should become more similar to each other, so $β$ should increase exponentially to compensate. Empirically, I found $β = δ^{- iter}$ as $δ \to 1$ to give decent results.

I think you should choose $β$ so that ${(\frac{β}{2})}^{- 1} = \frac{\sum_{i = 1}^{B} ∥ best x_{i} - {target}_{i} ∥^{2}}{B - 1},$ the sample variance over the batch between the closest choice and the target. This is because a good model should match both the mean and the variance of the ground truth. The ground truth is that, when you encode an image, you choose the $x_{i}$ that has the least reconstruction error. The probabilities $p_{i} \propto e^{- β error (x_{i}, target)}$ can be interpreted as conditional probabilities that you chose the right $x_{i}$ for the encoding, where each $x_{i}$ has a Gaussian prior for being the “right” encoding with mean $x_{i}$ and variance $2 / β$ . The variance of the prior for the $x_{i}$ that is actually chosen should match the variance it sees in the real world. Hence, my recommendation for $β$ .

(You should weight the MSE loss by $β$ as well.)

joseph_c

Re­sults of “Ex­per­i­ment on Bernoulli pro­cesses”

Results of “Experiment on Bernoulli processes”