Jisk, formerly Jacob. (And when Jacobs are locally scarce, still Jacob.)
LW has gone downhill a lot from its early days and I disapprove of most of the moderation choices but I’m still, sometimes, here.
It should be possible to easily find me from the username I use here, though not vice versa, for interview reasons.
In particular, it looks like we’re close enough to being able to implement corrigibility that the largest obstacle involves being able to observe how corrigible an AI is.
That’s a wild claim to make without reference to specific papers or milestones. I’m not fully up on ‘superalignment’ progress but last I looked no one on the modern paradigm side was seriously attempting to study corrigibility, let alone making this kind of progress. And results like Golden Gate Claude and the ‘buggy code → evil’ transformation indicated it was probably just as hard and unnatural vs in the MIRI paradigm.
In the scenario where he knows that prophecy foretells an outcome incompatible with his success here, his major decision point is long-past; he has reason to do it anyway (presumably prophetic reason). I see nothing he could do which is obviously better, and the conversation may itself be part of the keyhole future path.
If he doesn’t, this is still far from an “approximately-worst option.” It’s still a really good trap unless Quirrell knows the Mirror is going to be the trap, knows Harry’s Cloak is the genuine article that will still hide him from the Mirror, and can trick and coerce Harry into coming with him, which is three different things Dumbledore has good reason to think he probably doesn’t know. The latter two are both achieved only through adventures Dumbledore doesn’t know about—Azkaban, and Harry using up his time-loop password on the first day. As Lucius told Draco—any plan that relies on three things going right for you is at the limit of possible plans, and the real limit is two.
I still do not know what you think he should have done, either in the scenario where he knows he will fail due to prophecy, or the one where he does not.
I have two questions that come to mind, both about Chapter 86:
What else did the Dark Mark do? Did you work any of it out?
As many people have probably wondered since they read
“But as you can see, the Dark Lord was quite cunning.” His gaze grew more distant. “Oh,” Severus breathed, “he was very cunning indeed...”
and
If Harry hadn’t been interrupted by Mad-Eye arriving, would he have conceded enough of the point, and explained enough about how dangerous he would be fighting a war, to shock McGonagall and Snape into believing him? Would this have been catastrophic to the prophecy, probably via convincing Snape that Voldemort hadn’t ever been fighting to win?
Because I reread this scene a couple weeks ago, and it seemed like there were a bunch of reasons for him to find a way to elaborate on why he thought a true Dark Rationalist fighting a war against Dumbledore would win in hours rather than years, and his internal debate on this seems to be trending solidly in that direction, but then when he’s going to reach for paper, the meeting is interrupted and he never gets back to it. Which made me think that Dumbledore had arranged things for prophecy’s sake to prevent him, and consider why.
What is Dumbledore doing, that carries the Idiot Ball in the Mirror scene?
It begins with him having Quirrell hostage in an inescapable trap, proceeds through a conversation in which nothing of consequence occurs (nor is anything of consequence concealed), and ends with his assumption about the trap being invalidated by a cause nothing he could have done would have affected.
Or, with foreknowledge, it begins with the appearance of a trap, which Dumbledore knows will fail because he will fall very soon and Voldemort will not be beaten by his own hand. It proceeds through a conversation that is for show for both of them, possibly directed by prophecy because Harry is a hidden audience. And it ends where Dumbledore knows it must, without any action he could have taken to affect it.
Why does nobody ever ask about the Project Lawful / Planecrash epilogue? I still have to finish that one too!
Because we assume that you care more about HPMoR’s epilogue, I suspect. Also some amount of ‘nobody ever finishes promised epilogues for rationalfics’, which has not proven to be a law of nature but is certainly a strong prior.
So much was given to you, and that’s all you managed?
The principal problem here is the expectations. I expect those to essentially vanish in a state of abundance.
Also, the description of a school as a utopia seems (a) false and (b) totally irrelevant to the interesting claims made about the effect of abundance.
I don’t know that I’ve ever actually seen a show of any kind that actually felt over-rehearsed to me, but that may be a personal perception thing, and this does seem like a good strategy if you can swing it. Spreading practice out over three months probably 80/20s it.
The whole strategy of “write speech, practice doing that exact speech, then deliver it exactly as practiced” leaves no room to match the audience’s energy on the fly.
I don’t think this is true. Yes, you can’t ad-lib freely this way, but a lot of why I’m strongly in favor of getting off-book is that you can respond to the audience on the fly; mostly not with your words, but you can emote with body language, move, maybe shift the emphasis. You don’t want to be stuck to your script—that’s the point. This is probably a skill and not one I know how to teach because this part I think I just got right naturally, but I think it does come pretty naturally to most people, when they don’t feel stuck to checking their notes.
Separately, I expect for most people, even ones who are pretty good, heavy practice and polish on a prewritten script will have higher EV than ad-lib. Ad-libbing is higher variance, and you’re almost certainly right that it has a higher ceiling, but the downside is larger than the upside relative to a strategy of preparation and polish. (As well as requiring more skill.)
Claim that the stance in question is fairly canonical or standard for rationalists-as-a-group, modulo disclaimers about rationalists never agreeing on anything.
I don’t think this claim is correct. I have not noticed this being particularly common among rationalists relative to other similar populations, nor normative.
I think it’s probably unusually common among postrationalists, but those are a very different culture from rationalists, grounded primarily in not sharing any of the main assumptions common to rationalists.
Said had a habit of responding to posts that were reasonably well-reasoned but sparsely justified with one-word comments like “Sources?”, or very short coments along the lines of “X point is insufficeiently justified.” without any throat-clearing or praise, which is a good example of it done well.
In general, “Your premises are treated as obvious when they are actually bizarre, and your argument is therefore irrelevant.” is maybe the central example of when this is both highly confrontational but also highly necessary.
I can’t make out anything but word salad from this comment.
[Originally regarding Said Achmiz and myself ca. 2023]
I feel like you both favor a more aggressive flavor of discourse than I tend to like.
The aggressiveness is, I think, a symptom of the underlying trait, which is being disagreeable about taking people’s frames as valid
Most people, when given a weird framing of a situation which feels vaguely off but comes from someone who seems well-intentioned and cooperative, will go along with it and argue within that frame rather than contest it.
But this is very exploitable, and you don’t have to actually be consciously trying to exploit it to do so. And so people who do this a lot (e.g. Duncan, but also numerous other people I respect more, including Eliezer except when he’s explicitly being careful about it, which he usually is) can warp the whole field of discourse around them
Obviously most people who are disagreeable about this are disagreeable in general, and therefore usually aggressive about arguments and discourse. This isn’t necessary in principle but if anyone knows how to teach it I’ve never met them
Doesn’t this imply that having a theory of the domain you’re experimenting in is of low to no value? I find that hard to believe, and therefore doubt your assumptions are correct and applicable.
https://philpapers.org/rec/ARVIAA
This paper uses famous problems from philosophy of science and philosophical psychology—underdetermination of theory by evidence, Nelson Goodman’s new riddle of induction, theory-ladenness of observation, and “Kripkenstein’s” rule-following paradox—to show that it is empirically impossible to reliably interpret which functions a large language model (LLM) AI has learned, and thus, that reliably aligning LLM behavior with human values is provably impossible.
So, this seems provisionally to be bullshit because it doesn’t admit of thinking probabilistically or simplicity priors. But I’m not totally sure it’s worthless. Anyone read it in detail?
The older deck sucks. It contains the entirety of the essay without regard to what’s important. This deck is still messy—including too much focusing on the ordering and numbering of the virtues—but it’s significantly superior, and contains concise hearts of the matter. If you’re trying to create a memory aid for the Twelve Virtues, this deck was absolutely an improvement.
If there are a lot of people for the very-low-context NY meetup, possibly at least one very-low-context meetup per quarter is worth doing, to see if that gets people in/back more?
Like others, apparently “think like a mathematician” is enough to get it to work.
Not only is there not a standard name for this set of numbers, but it’s not clear what that set of numbers is. I consulted a better mathematician in the past, and he said that if you allow multiplication it becomes an known unsolved problem whether its representations are unique and whether it can construct all algebraic numbers.
CAST is a great idea and seems like the most promising way forward with architectures similar to the ones we have, but I do not see any reason to believe we could, if we had a corrigibility meter, build an AI that implemented corrigibility with reasonable robustness within a year. Five years would probably be enough but at that point you’re looking for at least one, and maybe 2-3, major insights.