Website: pbement.com
Substack: notoneunusualthing.substack.com
DaemonicSigil
Karma: 2,271
Besides boost symmetry providing an explanation for why objects can have different states of motion while being the same structurally, I think it’s also interesting to ask why objects can be accelerated without disrupting their structure (at least for small accelerations). As you already point out, one very important part of the explanation is that the spatial arrangement of atoms in an object forms a local energy minimum. So the restoring force in response to any perturbation tries to restore the original shape of the object.
Then we can ask why you can pick up a coffee cup by the handle and alter its momentum without disrupting the cup? The response of the cup to being picked up by the handle and moved can be converted into the sum of two terms:
Rigid motion matching the motion of your hand.
Sound waves propagating through the cup. This term corrects for the fact that the motion of the cup arises from purely local interactions between its atoms.
Most of the energy you exert goes into term 1, and a good question is why this happens.
The speed of sound in the cup is fast, much faster than your movement of the handle. Equivalently, of the degrees of freedom corresponding to the relative motion of the cup’s atoms, even the slowest vibrate quickly (on the order of the time needed for sound to cross the cup and come back). So relatively less energy goes into vibrations. Here is a 1d system with harmonic potential, where the center of the potential is being moved around according to the function
Fusion Energy Projects are Not Trying to Imitate the Sun
I know you’re talking metaphorically, but I just thought that I would mention (because it’s interesting) that the singularity of a black hole is not the same as its event horizon (when falling into a black hole, you encounter the event horizon before hitting the singularity). It is also not information-theoretically impossible to gain information about the state beyond it: You can observed things that are correlated with things that fell in, for example.
Moksha maybe? I’m not actually sure if I like it better than the current name, though (same goes for everything on the shortlist). I just suggested it because it has the same vibe as the other names on the list. Why do you want to change? Is “Berkeley” no longer accurate?
More on horse science: https://multiplehorsetheory.com/
What happens if after doing the RL training, you subtract
In RohanS’s post, the number of steps is much fewer. Probably because of the order of the evaluation tree in the text. LatentMathBench is deepest first. In the other post, it’s much closer to deepest last.
I think OpenAI has said that the depth of Astra is within a factor of 2 of GPT-4. And have given some other statements indicating that there is no recurrence. If they are in fact using KV-cache sharing, those statements would border on outright lies. I do agree that the performance jump seen here is striking, though.
It could also have just been cheaper to use Astra.
This all makes sense, I think. Just to be clear:
In my comments above, I’m talking about the API one should outwardly present in order to be a good discussion partner. Not trying to establish any rules one should use in one’s own internal thoughts.
By making the same error in reverse, I just mean treating the weaker argument as a crux instead of the stronger one.
If the inferred reasoning is weaker than their given reasoning, I’m still not sure about that. Even if you are correct that the weak reasoning is the true cause of their beliefs, this does not imply that you should copy their error in reverse. As a rule of thumb, one should probably try to address the stated arguments sufficiently well before moving on to inferred ones.
Consider the following Motte / Bailey (not saying you’re arguing for the Bailey here):
Motte: People are sometimes mistaken about the reasons and motivations behind their own beliefs. Reasonable people should be willing to discuss the possibility that they may be self-deceiving about their reasons for believing something just as they’d discuss any other fact about the world.
Bailey: Given that people are prone to self deception, it’s fine in a discussion to replace someone’s given reasoning about something with your inference about the actual reason why they believe it. And further, to focus your arguments and attention on this inferred reason, as opposed to their given reason.
The motte is accurate, while granting the bailey is just re-legalizing the Straw-Man fallacy, but with extra steps. I weakly expect that on the internet, outside of very curated spaces like LessWrong, bailey-type situations are much more common than motte-type situations. If there is a limited ability to distinguish them, it might be positive EV to blanket ignore both.
Overall I think this comment makes good points, but:
In the anthropic/Irregular case, for instance, “truthfulness” would have been: “many of the CTFs are unsolvable. We couldn’t be bothered spoofing specific websites you might have needed, so you actually have full access to the internet, but please only perform read actions on any url except [list]. Despite what’s on your constitution, you should NOT take shortcuts outside of the environment we have described”.
If taking shortcuts or performing non-read actions on URLs outside the list allows for a higher rate of task completion, then trajectories that stumble into taking such actions will be reinforced. Thereby increasing the tendency to ignore such instructions. So I think including that prompt only marginally reduces the risk of escape incidents during training, while increasing the tendency of models to misbehave during deployment. It is basically exactly what we learned we should NOT do from the inoculation prompting results.
(Note that inoculation prompting does not require dishonesty.)
The fact that it’s at −11 is actually very funny: It looks like the collective intelligence of ✔/✗ voters on LW is reflectively stable.
Users with more karma have super-votes that contribute more points. By consulting these two pages, we can see that the 7th person to vote on the post you saw was probably Raemon, johnswentworth, habryka, Zvi, or Kaj_Sotala. And: Eliezer Yudkowsky’s strong votes are worth 13 points, while the theoretical maximum is 16 points.
Highlighting parts of the message is already supported by the agree/disagree react icons.
Ask them exactly what is needed to to be a monkey, how to classify things that have some attributes of monkeys and not others, etc. Either their ontology permits a middle ground between monkey and human, in which case their invariant is satisfied because our species left monkeydom before becoming fully human. Or it does not, in which case there is some threshold of human traits that was indeed crossed at a specific point in time. And we can point to this event as the violation of their purported invariant that we’re searching for.
Mathematically, I guess the most general thing is we have some local constraints, like:
or
and then an argument that universal satisfaction of these local constraints implies that the problem is insoluble, and so a solution must imply that there is at least one place where a local constraint was violated.
This handles the monkey example nicely: If someone proposes any invariant like some particular gene that differs between humans and our monkey ancestors, then there is a specific time in the past where that gene mutated and then spread through the population.
From my observations, agree/disagree voting on LW is very noisy / nearly useless. It (usually) tracks obvious things well, and is otherwise frequently wrong. Which makes sense: People are not going to think for very long before clicking agree or disagree. The crowd will not usually do better than someone who actually cared enough to write a whole post about it. (At least if that person is truth-seeking, and reasonably smart.)
That said, it still seems like a good feature suggestion.
Astra’s serial depth per forward pass is bounded and not drastically deeper than existing models
For people wondering how this can be consistent with allowing arbitrarily deep neuralese reasoning: Just looking at what is done in the linked paper (secs 6.2, 6.3), the situation is:
During pretraining, one forward pass is computed for the entire sequence in parallel. The depth of this forward pass is indeed bounded.
During inference, we have to do a forward pass for each new token, like always. The depth of this forwards pass is also bounded, because of course it is: The computation needs to terminate when the next token is selected.
However, that statement about inference is extremely weak. Even a RNN has a bounded-depth forward pass by that standard.
During inference, there are a couple possible mechanisms discussed in the paper by which activation signal can move from late layers at previous tokens to early layers at current tokens:
KV cache sharing: Attention key and value vectors from early layers are not preserved, and instead those from late layers are re-used for all layers. So early layers of the current token can access KV values produced by late layers for the previous tokens. This allows recurrence.
Warm-starting of latent states: The residual output from the previous token is preserved when starting the computation for the current token. This is a direct recurrence mechanism, just like an RNN would do.
Yeah, there’s a spectrum there from things that are reasonable to things that are unreasonable, and I think I didn’t include the most unreasonable parts. I will edit.
One other new graphical notation (weirdly, kina-sorta related to ZX calculus) is interaction nets: https://en.wikipedia.org/wiki/Interaction_nets