Jisk, formerly Jacob. (And when Jacobs are locally scarce, still Jacob.)
LW has gone downhill a lot from its early days and I disapprove of most of the moderation choices but I’m still, sometimes, here.
It should be possible to easily find me from the username I use here, though not vice versa, for interview reasons.
C.S. Lewis is the usual author to cite for the other direction, and I’ve met both. More common among rationalist types. The Moon Is Harsh Mistress is possibly more effective than Rand.
if we needed correct internal computations for qualia (and not just correct behavior) would mean the overall system would falsely believe to have a quale (like being in pain), it would act, in all ways, like it was in pain, but actually, it wouldn’t be in pain.
To all appearances LLMs already do that and have for several years now. So, yes, that is clearly possible for a non-conscious thing to do.
Your definition of qualia is nonstandard, and defines it out of meaningfulness. More standard definitions generally include at least one synonym for ‘ineffable’ and I believe them to be entirely mysterious answers to mysterious questions.
This seems kind of silly, as a complaint. Many people would be disappointed by this; therefore, a good future won’t have it more than necessary as a tradeoff with other good things. The important puzzles where solving them is better on net than leaving them to be solved, will be solved; puzzles where humanity will get more out of never being told hints toward the answer than we lose from having the problems still exist, will not be solved. It’s possible that the latter category ends up empty and all that’s left is play, that solving Fun Theory doesn’t leave ‘real’ problems, but in a good future that will only happen if it’s worth it.
Oh, my mistake, technically you just made sweeping claims without attempting to justify them in the slightest. That is not literally equivalent to claiming they’re obvious. However, that is the same thing in practice. If you want to say “Ontologically speaking, any physical system exhibiting the same input-output pattern as a conscious being has identical conscious states.” and then never explain why you believe this to be true or defend it in any way, even when challenged—which you did—then you are, in every way that matters, claiming that it is obvious to every possible interlocutor. That no interlocutor’s doubts make it worth your time to explain yourself or defend your position. Let alone make an attempt to convince someone who has different priors, or different experiences.
(This is, of course, what people claiming something is obvious mean. That no one, or no one who counts, could possibly deny them. This is why good teachers of philosophy, mathematics, and science strongly discourage their students from getting in the habit of saying things are obvious; because that is almost never true.)
Also, I reread the parts of the Sequences about the zombie argument and I stand by what I said—they’re basically with me, that qualia are irrelevant. No useful definition of consciousness relies on qualia. If your definition of consciousness relies on qualia it is not useful, because it necessarily makes no empirical predictions. It is not quite as ridiculous as full epiphenomenal zombieism, but it is bad for the same reason.
As I recall the Sequences, they are very negative on qualia as a concept. Belief in quales is a belief which does not pay rent. I am generally unconvinced humans have qualia. I don’t see that I do. There appears to be no means of demonstrating them by experiment, and so presumptively they have no predictive power and so might as well not exist; if they exist, they’re irrelevant, and therefore unparsimonious.
It does seem plausible that an abruptly-stopping mind upload is conscious. It does not seem obvious; there’s a boundary condition and consciousness is very plausibly sensitive to boundary conditions and abrupt jumps according to many varieties of the theory. Most of your claims are of this nature; if you stop making the arrogant and unjustified claims that they’re obvious, there would be no reason to make further objections, because they’re perfectly plausible.
Then nothing can be obvious.
Indeed, that’s probably true in most contexts. “Obvious” rarely if ever has explanatory or didactic power and most people’s vocabularies would be better served by dropping it. I make use of ‘seems obvious’ much more than ‘is obvious’ because it is much more useful as a statement about my mind (conveys information about my reaction) than about the world (makes a universal claim which is enormously difficult to justify).
Different truths are obvious to different people.
And if it’s not obvious to everyone, it isn’t obvious. That’s what it means to claim something is obvious.
That’s not possible in principle. No matter what you empirically observe in a system, there is a possibility it’s not a quale (because perhaps you were mistaken about what constitutes a quale).
Then it’s not possible in principle for it to become obvious and you should stop trying to convince people it is.
I gave an example where it is not necessary.
No, you gave examples where it was still necessary. In none of your thought experiments is it clear that the variant emulation is conscious. In the ‘chocolate ice cream’ example I’d say it is very likely it is not conscious, because you can’t just make a small change like that and not have it propagate to larger ones, and making an arbitrary spot change without that will disrupt what’s going on in a way that probably, at least temporarily, disrupts consciousness. Compare to a concussion with loss of memory, or blackout drunkenness, during which most people will agree no consciousness is taking place.
It is not a necessary condition for the claim that this is true. It is very much necessary for the claim that it is obvious.
This is a matter of philosophy. No research results can help here, nor they are needed.
If no empirical results will make it clear—and your thought experiments certainly wouldn’t! - then it will never, ever be obvious.
It’s still not been common until much more recently than that. Five years at most, which is not a new normal. It hasn’t been working out that well for anyone except Trump himself and there’s a decent chance it explodes on his death (which will almost certainly be before ’32).
You don’t seem to understand the distinction between ‘mislead’ and ‘lie to the face.’ Or, you know, you’re lying to our face. Which would be a stupid plan, but maybe you’re doing it on purpose.
Even politicians lying to their audience’s faces is quite rare, in the USA. That it’s become meaningfully popular in the last 10 years is considered extremely notable and a sign of the decay and apocalypse of the United States, and to lesser degrees other Western countries. Bill Clinton got caught lying to Congress’s face about a trivial but embarrassing matter and “I did not have sexual relations with that woman” might as well be the words on his tombstone even though he suffered no real material consequences for it, because it’s very rare and no one will forget it.
Lying is hard. Being misinformed, or interpreting true fair information in a biased way to reach a biased conclusion, or writing the bottom line first, or quoting true statements in such a way as to give a deliberately misleading impression, are easy. Politicians do them all the time, and this is expected. Journalists did it a little until the Web revolution and now do it more than that, and this is, again, considered very notable and a sign of the decay of the industry. Scientists and experts do it a little, and almost always the first two types, but when they do even a little of the latter types this makes people riot and consider it very notable and a sign of the decay of the field.
Oh, fantastic, I just went looking. Thanks for investigating this!
It may not be a necessary condition, but if you want to present it as obvious, it is necessary. Anything short of an exact match is only allegedly the same until you have some research results that don’t currently exist to demonstrate strong reason to believe your lesser conditions are sufficient.
CAST is a great idea and seems like the most promising way forward with architectures similar to the ones we have, but I do not see any reason to believe we could, if we had a corrigibility meter, build an AI that implemented corrigibility with reasonable robustness within a year. Five years would probably be enough but at that point you’re looking for at least one, and maybe 2-3, major insights.
In particular, it looks like we’re close enough to being able to implement corrigibility that the largest obstacle involves being able to observe how corrigible an AI is.
That’s a wild claim to make without reference to specific papers or milestones. I’m not fully up on ‘superalignment’ progress but last I looked no one on the modern paradigm side was seriously attempting to study corrigibility, let alone making this kind of progress. And results like Golden Gate Claude and the ‘buggy code → evil’ transformation indicated it was probably just as hard and unnatural vs in the MIRI paradigm.
In the scenario where he knows that prophecy foretells an outcome incompatible with his success here, his major decision point is long-past; he has reason to do it anyway (presumably prophetic reason). I see nothing he could do which is obviously better, and the conversation may itself be part of the keyhole future path.
If he doesn’t, this is still far from an “approximately-worst option.” It’s still a really good trap unless Quirrell knows the Mirror is going to be the trap, knows Harry’s Cloak is the genuine article that will still hide him from the Mirror, and can trick and coerce Harry into coming with him, which is three different things Dumbledore has good reason to think he probably doesn’t know. The latter two are both achieved only through adventures Dumbledore doesn’t know about—Azkaban, and Harry using up his time-loop password on the first day. As Lucius told Draco—any plan that relies on three things going right for you is at the limit of possible plans, and the real limit is two.
I still do not know what you think he should have done, either in the scenario where he knows he will fail due to prophecy, or the one where he does not.
I have two questions that come to mind, both about Chapter 86:
What else did the Dark Mark do? Did you work any of it out?
As many people have probably wondered since they read
“But as you can see, the Dark Lord was quite cunning.” His gaze grew more distant. “Oh,” Severus breathed, “he was very cunning indeed...”
and
If Harry hadn’t been interrupted by Mad-Eye arriving, would he have conceded enough of the point, and explained enough about how dangerous he would be fighting a war, to shock McGonagall and Snape into believing him? Would this have been catastrophic to the prophecy, probably via convincing Snape that Voldemort hadn’t ever been fighting to win?
Because I reread this scene a couple weeks ago, and it seemed like there were a bunch of reasons for him to find a way to elaborate on why he thought a true Dark Rationalist fighting a war against Dumbledore would win in hours rather than years, and his internal debate on this seems to be trending solidly in that direction, but then when he’s going to reach for paper, the meeting is interrupted and he never gets back to it. Which made me think that Dumbledore had arranged things for prophecy’s sake to prevent him, and consider why.
What is Dumbledore doing, that carries the Idiot Ball in the Mirror scene?
It begins with him having Quirrell hostage in an inescapable trap, proceeds through a conversation in which nothing of consequence occurs (nor is anything of consequence concealed), and ends with his assumption about the trap being invalidated by a cause nothing he could have done would have affected.
Or, with foreknowledge, it begins with the appearance of a trap, which Dumbledore knows will fail because he will fall very soon and Voldemort will not be beaten by his own hand. It proceeds through a conversation that is for show for both of them, possibly directed by prophecy because Harry is a hidden audience. And it ends where Dumbledore knows it must, without any action he could have taken to affect it.
Why does nobody ever ask about the Project Lawful / Planecrash epilogue? I still have to finish that one too!
Because we assume that you care more about HPMoR’s epilogue, I suspect. Also some amount of ‘nobody ever finishes promised epilogues for rationalfics’, which has not proven to be a law of nature but is certainly a strong prior.
So much was given to you, and that’s all you managed?
The principal problem here is the expectations. I expect those to essentially vanish in a state of abundance.
Also, the description of a school as a utopia seems (a) false and (b) totally irrelevant to the interesting claims made about the effect of abundance.
Is any subculture not a gift economy? Smuggling in ‘small’ connotations to exclude things like ‘the gay community’ (though ca. 1960 maybe them too) or ‘goths,’ but I think it’s the default and it’s notable when a subculture deviates from it.
Possible places to find exceptions, highly monetizing and/or systematizing, many of which I suspect are as much gift economies as us:
Anglophone quant traders
Pro and semi-pro poker players
Crypto coin boosters
/r/WallStreetBets