Muireall

Karma: 398

Muireall 30 Mar 2024 21:05 UTC
2 points
0
in reply to: Said Achmiz’s comment on: My Interview With Cade Metz on His Reporting About Slate Star Codex
Yeah, plus all the other stuff Alexander and Metz wrote about it, I guess.

Muireall 30 Mar 2024 20:28 UTC
3 points
0
in reply to: Said Achmiz’s comment on: My Interview With Cade Metz on His Reporting About Slate Star Codex
It’s just a figure of speech for the sorts of thing Alexander describes in Kolmogorov Complicity. More or less the same idea as “Safe Space” in the NYT piece’s title—a venue or network where people can have the conversations they want about those ideas without getting yelled at or worse.
Mathematician Andrey Kolmogorov lived in the Soviet Union at a time when true freedom of thought was impossible. He reacted by saying whatever the Soviets wanted him to say about politics, while honorably pursuing truth in everything else. As a result, he not only made great discoveries, but gained enough status to protect other scientists, and to make occasional very careful forays into defending people who needed defending. He used his power to build an academic bubble where science could be done right and where minorities persecuted by the communist authorities (like Jews) could do their work in peace...
But politically-savvy Kolmogorov types can’t just build a bubble. They have to build a whisper network...
They have to serve as psychological support. People who disagree with an orthodoxy can start hating themselves – the classic example is the atheist raised religious who worries they’re an evil person or bound for Hell – and the faster they can be connected with other people, the more likely they are to get through.
They have to help people get through their edgelord phase as quickly as possible. “No, you’re not allowed to say this. Yes, it could be true. No, you’re not allowed to say this one either. Yes, that one also could be true as best we can tell. This thing here you actually are allowed to say still, and it’s pretty useful, so do try to push back on that and maybe we can defend some of the space we’ve still got left.”
They have to find at-risk thinkers who had started to identify holes in the orthodoxy, communicate that they might be right but that it could be dangerous to go public, fill in whatever gaps are necessary to make their worldview consistent again, prevent overcorrection, and communicate some intuitions about exactly which areas to avoid. For this purpose, they might occasionally let themselves be seen associating with slightly heretical positions, so that they stand out to proto-heretics as a good source of information. They might very occasionally make calculated strikes against orthodox overreach in order to relieve some of their own burdens. The rest of the time, they would just stay quiet and do good work in their own fields.

Muireall 30 Mar 2024 18:38 UTC
9 points
7
in reply to: Jiro’s comment on: My Interview With Cade Metz on His Reporting About Slate Star Codex
That section is framed with
Part of the appeal of Slate Star Codex, faithful readers said, was Mr. Siskind’s willingness to step outside acceptable topics. But he wrote in a wordy, often roundabout way that left many wondering what he really believed.
More broadly, part of the piece’s thesis is that the SSC community is the epicenter of a creative and influential intellectual movement, some of whose strengths come from a high tolerance for entertaining weird or disreputable ideas.
Metz is trying to convey how Alexander makes space for these ideas without staking his own credibility on them. This is, for example, what Kolmogorov Complicity is about; it’s also what Alexander says he’s doing with the neoreactionaries in his leaked email. It seems clear that Metz did enough reporting to understand this.
The juxtaposition of “Scott aligns himself with Murray [on something]” and “Murray has deplorable beliefs” specifically serves that thesis. It also pattern-matches to a very clumsy smear, which I get the impression is triggering readers before they manage to appreciate how it relates to the thesis. That’s unfortunate, because the “vague insinuation” is much less interesting and less defensible than the inference that Alexander is being strategic in bringing up Murray on a subject where it seems safe to agree with him.

Muireall 27 Mar 2024 1:31 UTC
1 point
−8
in reply to: Tenoke’s comment on: My Interview With Cade Metz on His Reporting About Slate Star Codex
In 2021, I was following these events and already less fond of Scott Alexander than most people here, and I still came away with the impression that Metz’s main modes were bumbling and pattern-matching. At least that’s the impression I’ve been carrying around until today. I find his answers here clear, thoughtful, and occasionally cutting, although I get the impression he leaves more forceful versions on the table for the sake of geniality. I’m wondering whether I absorbed some of the community’s preconceptions or instinctive judgments about him or journalists in general.
I do get the stubbornness, but I read that mostly as his having been basically proven right (and having put in the work at the time to be so confident).

Muireall 25 Jan 2024 18:40 UTC
24 points
5
on: Is a random box of gas predictable after 20 seconds?
In the 2D case, there’s no escaping exponential decay of the autocorrelation function for any observable satisfying certain regularity properties. (I’m not sure if this is known to be true in higher dimensions. If it’s not, then there could conceivably be traps with sub-exponential escape times or even attractors, but I’d be surprised if that’s relevant here—I think it’s just hard to prove.) Sticking to 2D, the question is just how the time constant in that exponent for the observable in question compares to 20 seconds.
The presence of persistent collective behavior is a decent intuition but I’m not sure it saves you. I’d start by noting that for any analysis of large-scale structure—like a spectral analysis where you’re looking at superpositions of harmonic sound waves—the perturbation to a single particle’s initial position is a perturbation to the initial condition for every component in the spectral basis, all of which perturbations will then see exponential growth.
In this case you can effectively decompose the system into “Lyapunov modes” each with their own exponent for the growth rate of perturbations, and, in fact, because the system is close to linear in the harmonic basis, the modes with the smallest exponents will look like the low-wave-vector harmonic modes. One of these, conveniently, looks like a “left-right density” mode. So the lifetime (or Q factor) of that first harmonic is somewhat relevant, but the actual left-right density difference still involves the sum of many harmonics (for example, with more nodes in the up-down dimension) that have larger exponents. These individually contribute less (given equipartition of initial energy, these modes spend relatively more of their energy in up-down motion and so affect left-right density less), but collectively it should be enough to scramble the left-right density observable in 20 seconds even with a long-lived first harmonic.
On the other hand, 1 mol in 1 m^3 is not very dense, which should tend to make modes longer-lived in general. So I’m not totally confident on this one without doing any calculations. Edit: Wait, no, I think it’s the other way around. Speed of sound and viscosity are roughly constant with gas density and attenuation otherwise scales inversely with density. But I think it’s still plausible that you have a 300 Hz mode with Q in the thousands.

Muireall 22 Jan 2024 0:43 UTC
1 point
0
in reply to: Muireall’s comment on: Muireall’s Shortform
Related would be some refactoring of Deception Chess.
When I think about what I’d expect to see in experiments like that, I get curious about a sort of “baseline” set of experiments without deception or even verbal explanations. When can I distinguish the better of two chess engines more efficiently than playing them against each other and looking at the win/loss record? How much does it help to see the engines’ analyses over just observing moves?
How is this related? Well, how deep is Chess? Ratings range between, say, 800 and 3500, with 300 points being enough to distinguish players (human or computer) reasonably well. So we might say there are about 10 “levels” in practice, or that it has a rating depth of 10.
If Chess were Best-Of-30 ChessMove as described above, then ChessMove would have a rating depth a bit below 2 (just dividing by $\sqrt{30}$ ). In other words, we’d expect it to be very hard to ever distinguish any pair of engines off a single recommended move—and difficult with any number of isolated observations, given our own error-prone human evaluation. If it’s closer to Best-Of-30 Don’tBlunder, it’s a little more complicated—usually you can’t tell the difference because there basically is none, but on rare pivotal moves it will be nearly as easy to tell as when looking at a whole game.
The solo version of the experiment looks like this:
1. I find a chess engine with a rating around mine, and use it to analyze positions in games against other engines. Play a bunch of games to get a baseline “hybrid” rating for myself with that advisor.
2. I do the same thing with a series of stronger chess engines, ideally each within a “level” of the last.
3. I do the same thing with access to the output of two engines, and I’m blinded to which is which. (The blinding might require some care around, for example, timing, as well as openings.) In sub-experiment A, I only get top moves and their scores. In sub-experiment B, I can look at lines from the current position up to some depth. In sub-experiment C, I can use these engines however I want. For example, I can basically play them against each other if I want to run down my own clock doing it. (Because pairs might be within a level of one another, I can’t be sure which is stronger from a single win/loss outcome. I’d hope to find more efficient ways of distinguishing them.)
4. I repeat #3 with different random pairs of advisors.
What I’d expect is that my ratings with pairs of advisors should be somewhere between my rating with the bad advisor and my rating with the good advisor. If I can successfully distinguish them, it’s close to the latter. If I’m just guessing, it’s close to the former (in the Don’tBlunder world) or to the midpoint (in the ChessMove world). I should have an easier time in sub-experiments B and C. Having a worse engine in the mix weighs me down relatively more (a) the closer the engines are to each other, and (b) the stronger both engines are compared to me.
The main question I’d hope might be answerable this way would be something like, “How do (a) and (b) trade off?” Which is easier to distinguish—1800 and 2100, or, say, 2700 and 3300? Will there be a ceiling beyond which I’m always just guessing? Might I tend to side with worse advisors because, being closer to my level, they agree with me?
It seems like we’d want some handle on these questions before asking how much worse outright deception can be.
(There’s some trouble here because higher-ranked players are more likely to draw given a fixed rating difference. This itself is relatively Don’tBlunder-like, and it makes me wonder if it’s possible to project how far our best engines are likely to be from perfect play. But it makes it harder to disentangle inability to draw distinctions in play above my level from “natural” indistinguishability. There are also more general issues in doing these experiments with computers—for example, weak engines tend to be weak in ways humans wouldn’t be, and it’s hard to calibrate ratings for superhuman play.)
(It might also be interesting to automate myself out of this experiment by choosing between recommendations using some simple scripted logic and evaluation by a relatively weak engine.)
Along the lines of what I wrote in the parent, even though I think there’s potentially a related and fairly deep “worldview”-type crux (crux generator?) nearby when it comes to AI risk—are we in a ChessMove world or a Don’tBlunder world?—[sorry, these are terrible names, because actual Chess moves are more like Don’tBlunder, which is itself horribly ugly]—I’m not particularly motivated to do this experiment, because I don’t think any possible answer on this level of metaphor would be informative enough to shift anyone on more important questions.

Muireall 15 Jan 2024 0:29 UTC
1 point
0
on: Muireall’s Shortform
I sometimes wonder how much we could learn from toy models of superhuman performance, in terms of what to expect from AI progress. I suspect the answer is “not much”, but I figured I’d toss some thoughts out here, as much to discharge any need I feel to think about them further as to see if anyone has any good pointers here.
Like—when is performance about making super-smart moves, and when is it about consistently not blundering for as long as possible? My impression is that in Chess, something like “average centipawn loss” (according to some analysis engine) doesn’t explain outcomes as well as “worst move per game”. (I don’t know the keywords to search for, but I relatedly found this neat paper which finds a power law for the difference between best and second-best moves in a position.) What does Go look like, in comparison?
How deep are games? What’s the longest chain of players such that each consistently beats the next? How much comes from the game itself being “deep” versus the game being made up of many repeated small contests? (E.g., the longest chain for best-of-9 Chess is going to be about 3 times longer than that for Chess, if the assumptions behind the rating system hold. Or, another example, is Chess better thought of as Best-Of-30 ChessMove with Elo-like performance and rating per move, or perhaps as Best-Of-30 Don’tBlunder with binary performance per move?)
Where do ceilings come from? Are there diminishing returns on driving down blunder probabilities given fixed deep uncertainties or external randomness? Is there such a thing as “perfect play”, and when can we tell if we’re approaching it? (Like—maybe there’s some theoretically-motivated power law that matches a rating distribution until some cutoff at the extreme tail?)
What do real-world “games” and “rating distributions” look like in this light?

Muireall 22 Dec 2023 23:01 UTC
14 points
0
on: A problem with infohazards as a concept [Linkpost]

Many times have I heard people talk about ideas they thought up that are ‘super infohazardous’ and ‘may substantially advance capabilities’ and then later when I have been made privy to the idea, realized that they had, in fact, reinvented an idea that had been publicly available in the ML literature for several years with very mixed evidence for its success – hence why it was not widely used and known to the person coming up with the idea.

I’d be very interested if anyone has specific examples of ideas like this they could share (that are by now widely known or obviously not hazardous). I’m sympathetic to the sorts of things the article says, but I don’t actually have any picture of the class of ideas it’s talking about.

Muireall 17 Nov 2023 14:45 UTC
2 points
0
in reply to: Bezzi’s comment on: Social Dark Matter
It sounds like you’re saying that you can tell once someone’s started transitioning, not that you can recognize trans people who haven’t (or who haven’t come out, at least not to a circle including you), right? Whether or not you’re right, the spirit of this post includes the latter, too.

Muireall 8 Nov 2023 23:26 UTC
7 points
0
in reply to: DaemonicSigil’s comment on: Muireall’s Shortform
This reasoning is basically right, but the answer ends up being 5 for a relatively mundane reason.
If the time-averaged potential energy is k_B T / 2, so is the kinetic energy. Because damping is low, at some point in a cycle, you’ll deterministically have the sum of the two in potential energy and nothing in kinetic energy. So you do have some variation getting averaged away.
More generally, while the relaxation timescale is the relevant timescale here, I also wanted to introduce an idea about very fast measurement events like the closing of the electrical circuit. If you have observables correlated on short timescales, then measurements faster than that won’t necessarily follow expectations from naive equilibrium thinking.

Muireall 30 Oct 2023 0:44 UTC
9 points
1
on: A new intro to Quantum Physics, with the math fixed
Setting aside most problems with the original, I’ve always found this interferometer example an unsatisfying introduction because it’s surprisingly ambiguous exactly what’s quantum mechanical here or what’s special about quantum mechanics.
You have superposition and interference in classical electromagnetism. That’s enough for everything until you get to the two-photon experiment (that is, for everything in “Configurations and Amplitude”). Single photons and photon counters are posited, but these are taken as given where I would sooner take as given the idea that a solution to a wave equation can be associated with a complex amplitude. Otherwise, up to that point one might as well have been talking about electromagnetic pulses and intensity detectors.
So is the interference between many-photon states the key? (“Joint Configurations”?) Not exactly. If you have classical light sources, then both quantum and classical theories give the same answer. Worse than that, really—for the quantum description, you have to posit that photons from different sources can be indistinguishable for the purposes of interference between many-photon states (“Distinct Configurations”, although it’s extra unclear about that), whereas that comes naturally if you’re just talking about electromagnetic fields as usual.
It feels somehow unfashionable in an age of quantum information to talk about wave-particle duality as the central surprise in quantum mechanics, but I think it’s right to zero in on the idea that you get superposition and interference in systems where otherwise-successful analogies from everyday experience don’t allow that.
Maybe it’s because my perspective is “electromagnetism-first”. From that direction, you’d introduce quantum mechanics by establishing the need for photons with things like the photoelectric effect. I suppose if you’re coming from the perspective that photons are only real, discrete, individual particles, then all this build-up for interferometry might make sense—did you know light can act like a wave, too? But then I think electron diffraction or spin polarization is more straightforward and doesn’t risk hammering on things that are totally fine classically.
(A marginally related suggestion—the diagrams of MZIs with lasers are going to be a little misleading for talking about experiments with photon number states, because lasers are not single photon sources. Maybe I’d be clearer about the difference between the diagram and situation in the text or just modify the figure.)

Muireall 28 Oct 2023 15:30 UTC
1 point
0
in reply to: Raemon’s comment on: Announcing Dialogues
I’d be up for a dialogue mostly in the sense of the first bullet about “making sense of technical debates as a non-expert”. As far as it’s “my domain” it’s in the context of making strategic decisions in R&D, but for example I’d also consider things like understanding claims about LK-99 to fall in the domain.
I think on and off about how one might practice (here’s one example) and always come away ambivalent. Case studies and retrospectives are valuable, but lately, I tend to lean more pragmatic—once you have a basically reasonable approach, it’s often best to come to a question with a specific decision in mind that you want to inform, rather than try to become generically stronger through practice. (Not just because transfer is hard, but also for example because the best you can do often isn’t that good, anyway—an obstacle for both pragmatic returns and for feedback loops.) And then I tend to think that the real problem is social—the most important information is often tacit or embedded in a community’s network (as in these examples)—and while that’s also something you can learn to navigate, it makes systematic deliberate practice difficult.

Muireall 9 Oct 2023 13:14 UTC
1 point
0
in reply to: Raemon’s comment on: Announcing Dialogues
The first! These are things I think about sometimes, but I suspect I’d be better at asking interesting questions than at saying interesting things myself.

Muireall 9 Oct 2023 1:34 UTC
1 point
0
in reply to: Ben Pace’s comment on: Announcing Dialogues
I’m interested in being a dialogue partner for any of the things I tend to post about, but maybe especially:
- Explosive economic growth—I’m skeptical of models like Tom Davidson’s but curious to probe others’ intuitions. For example, what would speeding up hardware research by a factor of 10 or more look like?
- Molecular nanotechnology, for example clarifying my critique of the use of equilibrium statistical mechanics in Nanosystems
- Maybe in more of an interviewer/rubber-duck role: Forecasting and its limits, professional ethics in EA, making sense of scientific controversies
- My own skepticism about AI x-risk as a priority

Muireall 7 Oct 2023 22:13 UTC
13 points
8
in reply to: tailcalled’s comment on: Sam Altman’s sister, Annie Altman, says Sam has (severely) abused her
My understanding is that perpetrator sexuality has little to do with the gender of chosen victims in child sexual abuse. If Annie was four years old and Sam thirteen at the time, I don’t think attraction to women played much of a role either way.

Muireall 3 Oct 2023 23:12 UTC
1 point
0
in reply to: Thomas Kwa’s comment on: Thomas Kwa’s MIRI research experience
Interesting, thanks!

Muireall 3 Oct 2023 22:38 UTC
1 point
0
in reply to: Raemon’s comment on: Thomas Kwa’s MIRI research experience
Sure. I only meant to use Thomas’s frame, where it sounds like Thomas did originally accept Nate’s model on some evidence, but now feels it wasn’t enough evidence. What was originally persuasive enough to opt in? I haven’t followed all Nate’s or Eliezer’s public writing, so I’d be plenty interested in an answer that draws only from what someone can detect from their public writing. I don’t mean to demand evidence from behind the confidentiality screen, even if that’s the main kind of evidence that exists.
Separately, I am skeptical and a little confused as to what this could even look like, but that’s not what I meant to express in my comment.

Muireall 3 Oct 2023 18:31 UTC
22 points
26
on: Thomas Kwa’s MIRI research experience
The model was something like: Nate and Eliezer have a mindset that’s good for both capabilities and alignment, and so if we talk to other alignment researchers about our work, the mindset will diffuse into the alignment community, and thence to OpenAI, where it would speed up capabilities. I think we didn’t have enough evidence to believe this, and should have shared more.
What evidence were you working off of? This is an extraordinary thing to believe.

Muireall 2 Oct 2023 23:16 UTC
7 points
0
in reply to: Muireall’s comment on: Muireall’s Shortform
Measuring noise and measurement noise
You’re using an oscilloscope to measure the thermal noise voltage across a resistance $R$ . Internally, the oscilloscope has a parallel input resistance $R_{i n}$ and capacitance $C$ , where the voltage on the capacitor is used to deflect electrons in a cathode ray tube to continuously draw a line on the screen proportional to the voltage over time.
The resistor and oscilloscope are at the same temperature. Is it possible to determine $R$ from the amplitude of the fluctuating voltage shown on the oscilloscope?
1. Yes, if $R_{i n} ≪ R$
2. Yes, if $R_{i n} \sim R$
3. Yes, if $R_{i n} ≫ R$
4. No
What links here?
- 5 Physics Problems by DaemonicSigil (18 Mar 2024 8:05 UTC; 60 points)

Muireall 28 Sep 2023 0:18 UTC
4 points
0
in reply to: Muireall’s comment on: Muireall’s Shortform
Molecular electromechanical switch
You’ve attached one end of a conductive molecule to an electrode. If the molecule bends by a certain distance $d$ at the other end, it touches another electrode, closing an electrical circuit. (You also have a third electrode where you can apply a voltage to actuate the switch.)
You’re worried about the thermal bending motion of the molecule accidentally closing the circuit, causing an error. You calculate, using the Boltzmann distribution over the elastic potential energy in the molecule, that the probability of a thermal deformation of at least $d$ is $10^{- 9}$ (a single-tailed six-sigma deformation in a normal distribution where expected potential energy is $k_{B} T / 2$ ), but you don’t know how to use this information. You know that the bending motion has a natural frequency of 100 GHz with an energy decay timescale of 0.1 nanosecond, and that it behaves as an ideal harmonic oscillator in a thermal bath.
You’re considering integrating this switch into a 1 GHz processor. What is the probability $p$ of an error in a 1 nanosecond clock cycle?
1. $p < 10^{- 9}$ — the Boltzmann distribution is a long-time limit, so you have sub-Boltzmann probability in finite time.
2. $p = 10^{- 9}$ — the probability is determined by the Boltzmann distribution.
3. $p \approx 10^{- 8}$ — the 0.1 nanosecond damping timescale means, roughly, it gets 10 draws from the Boltzmann distribution.
4. $p \approx 10^{- 7}$ — the 100 GHz natural frequency means it gets 100 tries to cause an error.
5. $p > 10^{- 7}$ — the Boltzmann distribution is over long-time averages, so you expect larger deviations on short timescales that otherwise get averaged away.

Muireall

Measuring noise and measurement noise

Molecular electromechanical switch