I should’ve specified: no communication allowed between drawing cards and the (simultaneous) guesses.
Adam Scherlis
I meant this specific conjecture, not all conjectures. More generally it applies to all conjectures of the form “there is no number n such that Q(n)” where Q is straightforward to check for a particular n.
I think you’re right that that’s the interesting part, and I did somehow fail to mention it—except in passing, since it’s the easiest way to prove that every successful strategy has exactly one winning guess.
I’m not so sure. I think a lot of physicists get better at this through practice, maybe especially in undergrad. I have a PhD in physics, and at this point I think I’m really good at figuring out the appropriate level of abstraction to use on something (something I’d put in the same category as the things mentioned in the OP.) I don’t totally trust my own recollection, but I think I was worse at this freshman year, and much more likely to pick e.g. continuum vs. discrete models of things in mechanics inappropriately and make life hard for myself.
(This is partly a response to the comment above, but I got kind of carried away.)
The Standard Model of particle physics accounts for everyday life (except gravity) in ridiculous detail, including all the “natural messiness” you have in mind (except gravity). It consists of some simple and unique (but mathematically tricky) assumptions called “quantum field theory” and “relativity”, plus the following details, which completely specify the theory:
* the gauge group is SU(3) x SU(2) x U(1) (or “the product of the three simplest things you could write down”)
* the matter particles break parity symmetry, using the simplest set of charges that works
* there are three copies of each matter particle
* there is also a scalar doublet
* the 20ish real-valued parameters implied by the above list have values which you can find by doing 20ish experiments.I dare anybody to give a specification of, say, all of known organic chemistry or geology with a list that short. You don’t need to spell out any mathematical details, so long as a mathematician could plausibly have invented it without being inspired by physical reality (which are the rules I’m playing by in this comment—I think QFT, relativity, and concepts like “gauge group” and “parity symmetry” that I assume knowledge of are all things math could/would have produced eventually).
In some sense I’m handwaving past the hard part, but I think the remarkable thing about physics is that the hard part is entirely math; if you did enough math in a cave without observing anything about the physical world, you would emerge with the kind of perspective from which the known laws of physics (except gravity) seem extremely parsimonious. (Gravity is also parsimonious but sort of stands alone for now.) On the other hand, if you go do a lot of experiments instead, the laws of physics will seem bizarre and complicated. Which I admit is kind of a strange fact! It’s not clear that “math parsimony” is the same concept as, say, Turing-machine-based Kolmogorov complexity, and it definitely isn’t anybody’s intuitive notion of “simplicity”.
And of course, quite a lot of the “natural messiness” of the world is captured by even simpler Newtonian-mechanics models, although chemistry becomes a kind of nasty black box from a Newtonian perspective.
It is definitely not a TOE, but it is a successful EFT that accounts for everything except gravity/cosmology.
You said “extremely simplified and idealised situations … frictionless planes, free fall in a vacuum, and so on”. That’s a pretty different ballpark than, say, every phenomenon any human before the 1990s had any knowledge of, in more detail than you can see under any microscope (except gravity).
Do you consider everything you’ve experienced in your entire life to have happened in “extremely simplified and idealised situations”?
Interestingly, I have better algebra intuition than analysis intuition, within math, and my physics intuition almost feels more closely related to algebra (especially “how to split stuff into parts”) than analysis to me.
Although there’s another thing which is sort of both algebra and analysis, which is looking at a structure in some limit and figuring out what other structure it looks like. (Lie groups/algebras come to mind.)
How do you install a TAP like that? It seems hard to do 10x in a row when you’re currently in “deciding I should have a TAP for this” mode and not already feeling angry/defensive.
(Actually, same question about “when my fridge is empty looking”.)
EDIT: From conversation with other people it sounds like it’s usually sufficient to imagine the trigger and action, in detail, several times.
Hmm, these posts feel like they’re skipping over the step I’m asking about, though.
Like, I don’t know how to do either of these things:
http://agentyduck.blogspot.com/2015/06/the-art-of-noticing.html “Whenever you notice your trigger, make a precise physical gesture.”
http://agentyduck.blogspot.com/2014/12/how-to-train-noticing.html “I keep a search going on in the background for anything in the neighborhood of the experience I predicted. Odds are good I’ll miss several instances of weak contrary evidence, but as soon as I realize I’ve encountered one, I go into reflective attention so I’m aware of as many details of my immediate subjective experience as possible.”
The thing I’m having trouble with is: at some point in the future, the trigger I’m thinking about will happen. How will I notice/remember that that was a trigger? What can I do now, when the trigger has not yet happened, to make me remember later? I don’t think I know how to keep searches going in the background, or anything like that.
Partial solutions, to gesture at the gap:
* Phone alarms for triggers that happen at a specific time (for the first few triggers, before the habit is formed)
* Assign someone to watch me 24⁄7 and poke me in the side whenever a desired trigger happens
* When I come up with an idea for a TAP, [my single-point-of-reference productivity Google Doc, under the “TAP” section, tells me to] close my eyes and imagine the trigger happening for 5 minutes
I’m confused by the number of posts on TAPs (excluding OP) that gloss over this part.
Language model parameter counts were growing much faster than 2x/18mo for a while.
(Computational chemistry may be intractable, even with approximations, on classical hardware—but what about if one has a quantum computer with a few hundred qubits, enough that one can do quantum simulation?) The importance of constant factors is one of the major traps in practical use of complexity classes: a fancy algorithm with a superior complexity class may easily be defeated by a simpler algorithm with worse complexity but faster implementation.
This example contradicts the point he’s trying to make here. Quantum computers are thought to be asymptomatically faster at some tasks (including quantum simulation) but have much, much worse constant factors for individual operations.
Coming from you that’s high praise!
I haven’t read all of this yet. I like it so far. One nitpick: I would really try to avoid referring to individual microstates as having “entropy” assigned to them. I would call (or things playing a similar role) something else like “surprisal” or “information”, and reserve entropy (rather than “average entropy”) for things that look like or .
Of course, for macrostates/distributions with uniform probability, this works out to be equal to for every state in the macrostate, but I think the conceptual distinction is important.
(I’m as guilty as anyone of calling simple microstates “low-entropy”, but I think it’s healthier to reserve that for macrostates or distributions.)
Footnote 17 sounds confusing and probably wrong to me, but I haven’t thought it through. Macrostates should have some constraint that makes their total probability 1; you can’t have a macrostate containing a single very unlikely microstate.
(Edit: “wrong” seems a bit harsh on reflection but I dislike the vagueness about “cheating” a lot. The single-improbable-thing macrostate should just not typecheck instead of somehow being against the spirit of things.)
I’m fine with choosing some other name, but I think all of the different “entropies” (in stat mech, information theory, etc) refer to weighted averages over a set of states, whose probability-or-whatever adds up to 1. To me that suggests that this should also be true of the abstract version.
So I stand by the claim that the negative logarithm of probability-or-whatever should have some different name, so that people don’t get confused by the ([other thing], entropy) → (entropy, average entropy) terminology switch.
I think “average entropy” is also (slightly) misleading because it suggests that the -log(p)’s of individual states are independent of the choice of which microstates are in your macrostate, which I think is maybe the root problem I have with footnote 17. (See new comment in that subthread)
I would maybe say that your “average entropy” (what I’d call entropy) is always the average over every state, every single time, and (uniform) macrostates are just a handy conceptual shorthand for saying “I want all of these states to have equal p (equal -log p) and all of these to have zero p (infinite -log p)” without getting bogged down in why 0 log 0 is 0. A state is “in” a macrostate if it’s one of the states with nonzero p for that macrostate, but the sum is always over everything.
I almost agree, but I really do stand by my claim that Alex has nicely identified the correct abstract thing and then named the wrong part of it entropy.
[EDIT: I now think the abstract thing I describe below—statistical entropy—is not the full thing Alex is going for. A more precise claim is: Alex is describing some general thing, and calling part of it “entropy”. When I map that thing onto domains like statmech or information theory, his “entropy” doesn’t map onto the thing called “entropy” in those domains, even though the things called “entropy” in those domains do map onto each other. This might be because he wants it to map onto “algorithmic entropy” in the K-complexity setting, but I think this doesn’t justify the mismatch.]
The abstract thing [EDIT: “statistical entropy”] is shaped something like: there are many things (call ’em microstates).
Each thing has a “weight”, p.
(Let’s not call it “probability” because that has too much baggage.)We care a lot about the negative log of p. However, in none of the manifestations of this abstract concept is that called “entropy”.
We also care about the average of -log(p) over every possible microstate, weighted by p. That’s called “entropy” in every manifestation of this pattern (if the word is used at all), never “average entropy”.
I don’t see why it helps intuition to give these things the same name, and especially not why you would want to replace the various specific “entropy”s with an abstract “average entropy”.
Also, to state the obvious, noticing that the concept wants a short name (if we are to tie a bunch of other things together and organize them properly) feels to me like a unit of conceptual progress regardless of whether I personally like the proposed pun
Agreed!
schwa and lug
Yeah, shorthand for this seems handy. I like these a lot, especially schwa, although I’m a little worried about ambiguous handwriting. My contest entry is
nl
(for “negative logarithm” or ”ln
but flipped”).
If it can’t be proven false, then it definitely isn’t false!
Equivalently: If it’s false, then it can be proven false.
Why do I say that? Well, if it’s false, then there exists a power of two that is the reverse of a power of five. But that has a very short proof: just write down the smallest example.
As to the case where it can’t be proven either way: I would say that it has to be true in that case, but this might be one of those things that sufficiently diehard constructivists would agree with you on.