orthonormal

Karma: 14,849

orthonormal 13 Jan 2021 4:18 UTC
LW: 13 AF: 9
AF
on: Understanding “Deep Double Descent”
If this post is selected, I’d like to see the followup made into an addendum—I think it adds a very important piece, and it should have been nominated itself.

orthonormal 13 Jan 2021 4:07 UTC
LW: 4 AF: 2
AF
on: More realistic tales of doom
I think this post (and similarly, Evan’s summary of Chris Olah’s views) are essential both in their own right and as mutual foils to MIRI’s research agenda. We see related concepts (mesa-optimization originally came out of Paul’s talk of daemons in Solomonoff induction, if I remember right) but very different strategies for achieving both inner and outer alignment. (The crux of the disagreement seems to be the probability of success from adapting current methods.)
Strongly recommended for inclusion.

orthonormal 13 Jan 2021 3:57 UTC
LW: 6 AF: 3
AF
on: Soft takeoff can still lead to decisive strategic advantage
It’s hard to know how to judge a post that deems itself superseded by a post from a later year, but I lean toward taking Daniel at his word and hoping we survive until the 2021 Review comes around.

orthonormal 13 Jan 2021 3:52 UTC
4 points
on: Rationality and Levels of Intervention
I can’t think of a question on which this post narrows my probability distribution.
Not recommended.

orthonormal 13 Jan 2021 3:19 UTC
LW: 6 AF: 4
AF
on: Chris Olah’s views on AGI safety
The content here is very valuable, even if the genre of “I talked a lot with X and here’s my articulation of X’s model” comes across to me as a weird intellectual ghostwriting. I can’t think of a way around that, though.

orthonormal 12 Jan 2021 17:40 UTC
2 points
in reply to: orthonormal’s comment on: AlphaStar: Impressive for RL progress, not for AGI progress
That being said, I’m not very confident this piece (or any piece on the current state of AI) will still be timely a year from now, so maybe I shouldn’t recommend it for inclusion after all.

orthonormal 12 Jan 2021 15:09 UTC
6 points
in reply to: Matt Goldenberg’s comment on: Relevance Norms; Or, Gricean Implicature Queers the Decoupling/Contextualizing Binary
Ironically enough for Zack’s preferred modality, you’re asserting that even though this post is reasonable when decoupled from the rest of the sequence, it’s worrisome when contextualized.

orthonormal 10 Jan 2021 22:07 UTC
4 points
in reply to: Ben Pace’s comment on: The AI Timelines Scam
I agree about the effects of deep learning hype on deep learning funding, though I think very little of it has been AGI hype; people at the top level had been heavily conditioned to believe we were/are still in the AI winter of specialized ML algorithms to solve individual tasks. (The MIRI-sphere had to work very hard, before OpenAI and DeepMind started doing externally impressive things, to get serious discussion on within-lifetime timelines from anyone besides the Kurzweil camp.)
Maybe Demis was strategically overselling DeepMind, but I expect most people were genuinely over-optimistic (and funding-seeking) in the way everyone in ML always is.

orthonormal 10 Jan 2021 5:51 UTC
4 points
on: You Have About Five Words
This is a retroactively obvious concept that I’d never seen so clearly stated before, which makes it a fantastic contribution to our repertoire of ideas. I’ve even used it to sanity-check my statements on social media. Well, I’ve tried.
Recommended, obviously.

orthonormal 10 Jan 2021 5:49 UTC
LW: 2 AF: 1
AF
on: The Parable of Predict-O-Matic
This reminds me of That Alien Message, but as a parable about mesa-alignment rather than outer alignment. It reads well, and helps make the concepts more salient. Recommended.

orthonormal 10 Jan 2021 5:30 UTC
4 points
on: Rule Thinkers In, Not Out
This makes a simple and valuable point. As discussed in and below Anna’s comment, it’s very different when applied to a person who can interact with you directly versus a person whose works you read. But the usefulness in the latter context, and the way I expect new readers to assume that context, leads me to recommend it.

orthonormal 10 Jan 2021 5:17 UTC
4 points
on: The AI Timelines Scam
I liked the comments on this post more than I liked the post itself. As Paul commented, there’s as much criticism of short AGI timelines as there is of long AGI timelines; and as Scott pointed out, this was an uncharitable take on AI proponents’ motives.
Without the context of those comments, I don’t recommend this post for inclusion.

orthonormal 8 Jan 2021 6:36 UTC
3 points
on: Yes Requires the Possibility of No
I’ve referred and linked to this post in discussions outside the rationalist community; that’s how important the principle is. (Many people understand the idea in the domain of consent, but have never thought about it in the domain of epistemology.)
Recommended.

orthonormal 8 Jan 2021 6:30 UTC
12 points
on: Book summary: Unlocking the Emotional Brain
As mentioned in my comment, this book review overcame some skepticism from me and explained a new mental model about how inner conflict works. Plus, it was written with Kaj’s usual clarity and humility. Recommended.

orthonormal 8 Jan 2021 6:18 UTC
5 points
on: AlphaStar: Impressive for RL progress, not for AGI progress
I stand by this piece, and I now think it makes a nice complement to discussions of GPT-3. In both cases, we have significant improvements in chunking of concepts into latent spaces, but we don’t appear to have anything like a causal model in either. And I’ve believed for several years that causal reasoning is the thing that puts us in the endgame.
(That’s not to say either system would still be safe if scaled up massively; mesa-optimization would be a reason to worry.)

orthonormal 8 Jan 2021 6:01 UTC
4 points
in reply to: Kaj_Sotala’s comment on: Book summary: Unlocking the Emotional Brain
I never found a Coherence Therapy practitioner, but I found a really excellent IFS practitioner who’s helped me break down many of my perpetual hangups in ways compatible with this post.
In particular, one difference between the self-IFS I’d attempted before is that I’d previously tried to destroy some parts as irrational or hypocritical, where the therapist was very good at being non-judgmental towards them. That approach paid better dividends.

orthonormal 27 Dec 2020 0:53 UTC
7 points
in reply to: Oskar Mathiasen’s comment on: What evidence will tell us about the new strain? How are you updating?
Can’t update on #4. Of course a rapidly growing new strain will have a negligible impact on total numbers early on; it’s a question of whether it will dominate the total numbers in a few months.

orthonormal 20 Dec 2020 18:34 UTC
LW: 12 AF: 8
AF
on: Radical Probabilism
Remind me which bookies count and which don’t, in the context of the proofs of properties?
If any computable bookie is allowed, a non-Bayesian is in trouble against a much larger bookie who can just (maybe through its own logical induction) discover who the bettor is and how to exploit them.
[EDIT: First version of this comment included “why do convergence bettors count if they don’t know the bettor will oscillate”, but then I realized the answer while Abram was composing his response, so I edited that part out. Editing it back in so that Abram’s reply has context.]

orthonormal 16 Dec 2020 17:30 UTC
8 points
on: Covid 12/10: Vaccine Approval Day in America
Not the most important thing, but Adler and Colbert’s situations feel rather different to me.
Colbert is bubbled with a small team in order to provide mass entertainment to the nation… just like sports teams, which you endorse.
Adler is partying for his own benefit.

orthonormal 25 Oct 2020 20:37 UTC
0 points
on: PredictIt: Presidential Market is Increasingly Wrong
To steelman the odds’ consistency (though I agree with you that the market isn’t really reflecting careful thinking from enough people), Biden is farther ahead in the 538 projection now than he was before, but on the other hand, Trump has completely gotten away with refusing to commit to a peaceful transfer of power. Even if that’s not the most surprising thing in the world (how far indeed we have fallen), it wasn’t at 100% two months ago.