Jisk, formerly Jacob. (And when Jacobs are locally scarce, still Jacob.)
LW has gone downhill a lot from its early days and I disapprove of most of the moderation choices but I’m still, sometimes, here.
It should be possible to easily find me from the username I use here, though not vice versa, for interview reasons.
I don’t think so but it might, and it does push toward symmetry.
It’s not broad cynicism, it’s a particular view of how corrupted hardware works, borne out by small-group politics everywhere. This says that it is strongly favored to believe yourself to be type 2 while actually being type 3, sufficiently so that to a first approximation anyone who believes themselves to be type 2 is incorrect, and that no one can ever verify that they, let alone anyone else, are genuinely type 2, and so must assume that if they appear to be, they are actually type 3.
It does not say that about type 4. The strong incentives/forces there don’t apply, because it’s a more complicated move much harder to do implicitly. It is very difficult to be making a move specifically to grab status by pretending to be something else and not notice you’re doing it, which type 4 requires and type 2 does not. Type 4 is possible in a way type 3 is not; there is no strong force pushing away from it. But there is a moderate force pushing away from that to ‘type 5’: people able to disentangle their reasoning from status concerns and favoring status concerns anyway.
Even if #2 is extremely rare it’s still the best of both worlds and something that we all should aspire too, isn’t it?
No, I don’t think it is. Even if everyone could do #2 reliably, that would be a local optimum but globally wasteful.
Person #2 effectively doesn’t exist. That’s what it feels like to be person #3; you think you’re person #2, and for 95% of people, you’re wrong. I’m skeptical of #4, too; that’s not what that kind of social-move-maker typically feels like internally either.
Basically all the ways LessWrong is different from Reddit are structured, very explicitly, to move it away from the pole of blunt honesty and not taking offense back toward the general population. I think the only feature LW has introduced since LW 2.0 that pushes in favor of either of those things is the introduction of agreement karma distinct from overall karma, and that’s a fairly weak push.
A good culture is somewhere in between, but closer to A than B. Forum B encourages people to equivocate and utter half-truths and forget what it means to tell the truth or to lie. Much better to have confrontations, edit older comments when you’ve changed your mind, and have your faced rubbed into your current internal contradictions; then you will at least notice them, and that’s the necessary prerequisite to correcting them, or if you won’t/can’t, observing they make you unreliable at reasoning in whatever matter presents itself.
Ironic, considering Duncan’s one of the first three names that come to mind as perpetrators. Fortunately his attempts to push boundaries resulted in him being pushed off LW rather than successfully destroying its norms.
Though my perception is that they did considerable damage to norms I care about that were already very vulnerable. I was only paying close attention to the dispute with Said_Achmiz but I observed a larger pattern of decreasing honesty and forthrightness beyond that, with Duncan as one of the main vocal champions of the change. For reasons that sounded plausibly good which I think he believes in. But which I have never been confident are his true rejection, or that of others.
Absolutely not. From my experience, not at all. I have turned down sex with someone I’d been on a date with, found attractive, and was generally interested in because they were, that day, boring me enough that the attraction couldn’t stick.
Other things like kinks and fantasies can get more flexible far easier than the appeal of a partner. Conversely, an intellectually appealing partner makes everything else more flexible, including beauty standards and what seems hot.
I guess that disqualifies him as an intellectual? But when he does the same thing on purpose that the people he’s accusing do by accident—ignore any subtleties or higher-order effects and just go with the simple idea—I submit that it makes him part of the category he’s described. If you don’t want to be grouped with the people you’re insulting, stay well clear of behaving like them.
The greatest idiot of an intellectual I’ve ever read or heard a word from is Taleb himself. As with much of his writing, the concept has immense memetic value but no predictive value.
As someone with plenty of experience being full of contempt for just about everyone and (mostly) not being a bastard about it, I kind of think the most helpful lens is less philosophical per se and more practical and ~political. Namely, libertarianism and the knowledge problem.
The world is full of idiots. The country is full of idiots. The government is full of idiots. The corporations are full of idiots. The charities, the social media feeds, the buses and trains, they’re all full of idiots. Some of those are more true than they were twenty years ago but they were all true then too.
But they still know their own lives better than us. They do things that are objectively kind of terrible like get payday loans and buy tons of lottery tickets, but almost always it’s because the better options are blocked off to them or not as good as they look, even if it’s not possible to explain why to an outsider like us even if we were to poke our nose in and ask. (And they wanted to answer for some ungodly reason.)
Libertarianism mostly likes to talk about how the government should butt out and let people live their lives. And this is true, it should. But the same argument also applies to elitists. Yes, they’re doing a bunch of dumb shit, and probably if they were smarter they would change some of it and make their lives better, but you can’t just drop yourself into their lives and ‘fix’ things and expect it to work out. Some of it would make things worse, some of it would theoretically make things better but only if you changed their entire social sphere at once too, and some of it you wouldn’t notice you’d fucked up for years. And mostly that’s still true if “drop yourself into their lives” is replaced with “give insistent well-meaning advice” or “ruminate on how they’re idiots and if they just listened to you, the superior person, they’d be better”. (Okay, the practical issues prohibiting getting any positive impact from ruminating are mostly more obvious than this. But this too.)
Calibrating appropriately is hard. You do, sometimes, want to give advice once or twice. If you figure out when to do that and when to back off, tell me, please. What advice I could try to give there is neither specific nor wonderfully effective so I won’t bother.
But I think this is generally pretty effective at being, technically, contemptuous of most strangers you interact with but mostly keeping it from being something they notice or one of the top three things you notice when having a conversation with them, and that’s pretty good at not encouraging behaving contemptuously, and that’s pretty good at avoiding steeping in contempt until your standards for lack of contempt rise, and then apply to precisely seven people in the world none of whom is you. (Which I think is a risk here.) And that’s, I suspect, at least what you need short-term.
I think those are all more gift economies than they are market economies. I’ve never been part of a dojo long-term but there was ‘hey want a ride’ ‘sure I’ll pick up dinner’ at least a bit even with the one I was part of briefly. Buying and selling MtG was something you did with the store, not the other players, almost always. (Some people were aggressive value traders but they were almost their own subculture because no one really enjoyed dealing with them.) And for the kind of card trades that didn’t look at all at what book value was and just swapped what seemed good to them… Man, I guess that’s at least 40% market economy but it sure didn’t feel transactional. And it died pretty rapidly when the aggressive value traders who were indisputably a market economy showed up, which feels like that’s a sign the market economy part was fragile and noncentral?
I have never been part of a (non-children’s) sports league but I would guess they did a lot of ‘sure do you a favor’ the same way I’d expect from dojos. And I avoid Toastmasters* like it’s radioactive, so IDK.
I think there’s a very fuzzy distinction between ‘technically gift economy but not ~integrated enough to meaningfully be trading’ and ‘not a gift economy’. Probably a similar distinction exists at the low end of market economy, though I suspect on no direct evidence it’s less fuzzy.
(*Even if I witnessed some Master Toasts trading things or doing favors, I still wouldn’t know, because my experience is that a Master Toast emits primarily Simalacrum-Level-4 statements with the occasional S3 or S2 and so I couldn’t begin to guess whether their ‘favors’ were actually favors.)
Is any subculture not a gift economy? Smuggling in ‘small’ connotations to exclude things like ‘the gay community’ (though ca. 1960 maybe them too) or ‘goths,’ but I think it’s the default and it’s notable when a subculture deviates from it.
Possible places to find exceptions, highly monetizing and/or systematizing, many of which I suspect are as much gift economies as us:
Anglophone quant traders
Pro and semi-pro poker players
Crypto coin boosters
/r/WallStreetBets
C.S. Lewis is the usual author to cite for the other direction, and I’ve met both. More common among rationalist types. The Moon Is Harsh Mistress is possibly more effective than Rand.
if we needed correct internal computations for qualia (and not just correct behavior) would mean the overall system would falsely believe to have a quale (like being in pain), it would act, in all ways, like it was in pain, but actually, it wouldn’t be in pain.
To all appearances LLMs already do that and have for several years now. So, yes, that is clearly possible for a non-conscious thing to do.
Your definition of qualia is nonstandard, and defines it out of meaningfulness. More standard definitions generally include at least one synonym for ‘ineffable’ and I believe them to be entirely mysterious answers to mysterious questions.
This seems kind of silly, as a complaint. Many people would be disappointed by this; therefore, a good future won’t have it more than necessary as a tradeoff with other good things. The important puzzles where solving them is better on net than leaving them to be solved, will be solved; puzzles where humanity will get more out of never being told hints toward the answer than we lose from having the problems still exist, will not be solved. It’s possible that the latter category ends up empty and all that’s left is play, that solving Fun Theory doesn’t leave ‘real’ problems, but in a good future that will only happen if it’s worth it.
Oh, my mistake, technically you just made sweeping claims without attempting to justify them in the slightest. That is not literally equivalent to claiming they’re obvious. However, that is the same thing in practice. If you want to say “Ontologically speaking, any physical system exhibiting the same input-output pattern as a conscious being has identical conscious states.” and then never explain why you believe this to be true or defend it in any way, even when challenged—which you did—then you are, in every way that matters, claiming that it is obvious to every possible interlocutor. That no interlocutor’s doubts make it worth your time to explain yourself or defend your position. Let alone make an attempt to convince someone who has different priors, or different experiences.
(This is, of course, what people claiming something is obvious mean. That no one, or no one who counts, could possibly deny them. This is why good teachers of philosophy, mathematics, and science strongly discourage their students from getting in the habit of saying things are obvious; because that is almost never true.)
Also, I reread the parts of the Sequences about the zombie argument and I stand by what I said—they’re basically with me, that qualia are irrelevant. No useful definition of consciousness relies on qualia. If your definition of consciousness relies on qualia it is not useful, because it necessarily makes no empirical predictions. It is not quite as ridiculous as full epiphenomenal zombieism, but it is bad for the same reason.
As I recall the Sequences, they are very negative on qualia as a concept. Belief in quales is a belief which does not pay rent. I am generally unconvinced humans have qualia. I don’t see that I do. There appears to be no means of demonstrating them by experiment, and so presumptively they have no predictive power and so might as well not exist; if they exist, they’re irrelevant, and therefore unparsimonious.
It does seem plausible that an abruptly-stopping mind upload is conscious. It does not seem obvious; there’s a boundary condition and consciousness is very plausibly sensitive to boundary conditions and abrupt jumps according to many varieties of the theory. Most of your claims are of this nature; if you stop making the arrogant and unjustified claims that they’re obvious, there would be no reason to make further objections, because they’re perfectly plausible.
Then nothing can be obvious.
Indeed, that’s probably true in most contexts. “Obvious” rarely if ever has explanatory or didactic power and most people’s vocabularies would be better served by dropping it. I make use of ‘seems obvious’ much more than ‘is obvious’ because it is much more useful as a statement about my mind (conveys information about my reaction) than about the world (makes a universal claim which is enormously difficult to justify).
Different truths are obvious to different people.
And if it’s not obvious to everyone, it isn’t obvious. That’s what it means to claim something is obvious.
That’s not possible in principle. No matter what you empirically observe in a system, there is a possibility it’s not a quale (because perhaps you were mistaken about what constitutes a quale).
Then it’s not possible in principle for it to become obvious and you should stop trying to convince people it is.
I gave an example where it is not necessary.
No, you gave examples where it was still necessary. In none of your thought experiments is it clear that the variant emulation is conscious. In the ‘chocolate ice cream’ example I’d say it is very likely it is not conscious, because you can’t just make a small change like that and not have it propagate to larger ones, and making an arbitrary spot change without that will disrupt what’s going on in a way that probably, at least temporarily, disrupts consciousness. Compare to a concussion with loss of memory, or blackout drunkenness, during which most people will agree no consciousness is taking place.
It is not a necessary condition for the claim that this is true. It is very much necessary for the claim that it is obvious.
This is a matter of philosophy. No research results can help here, nor they are needed.
If no empirical results will make it clear—and your thought experiments certainly wouldn’t! - then it will never, ever be obvious.
I believe this to be a severe misunderstanding of corrigibility. It may on its own be fatal to this argument; I’m unsure.
Distinguish three things:
Do what I say
Do what I mean
Do what I want
‘Do what I say’ is obviously bad—from Tithonus to the Sorcerer’s Apprentice and Amelia Bedelia we have extensive cultural warnings against it. ‘Do what I want’ is the ideal, the AI which does not need to be asked—in short, Friendly. CAST AI is neither. It is ‘Do what I mean.’ It will determine what you intended, and warn you about negative consequences, but if you clarify that you want it to do it anyway regardless of the reasons it believes (arguendo, correctly) that you will regret it, it will still proceed. In the limit it may suggest tasks it thinks you may desire, or ask for broad permission for actions on your behalf, but it will not act beyond its instructions. This is at the root of both why it is easier than value alignment and why it is safer to gradually approach than value alignment.
That does not imply it is successful. CAST is valuable because, if true, at every stage it is amenable to being corrected if its interpretation of the principal’s desires is in error. As complexity grows, so does the complexity of those errors. We should expect some detailed errors to be hidden before the scale and complexity of the AI’s potential actions grows; if this causes the helpful property to disappear as correction grows impossible, that reduces the value of CAST significantly.
I did not read any of the CAST sequence as claiming a relative advantage, but an absolute one. I read it as saying ‘Here is an approach with a high chance of success, and a much higher feasibility than value alignment.’ If these caveats leave it as best of a bad lot, but no longer with a high chance of success, then they are appropriate.
Also, if corrigibility fails only as power and intelligence grows, then it is effectively deceptively aligned.