localdeity

• localdeity 12 Jun 2022 6:03 UTC

  LW: 90 AF: 29

  62

  AF

  on: Godzilla Strategies

  He discovered several papers that described software-assisted hardware recovery. The basic idea was simple: if hardware suffers more transient failures as it gets smaller, why not allow software to detect erroneous computations and re-execute them? This idea seemed promising until John realized THAT IT WAS THE WORST IDEA EVER. Modern software barely works when the hardware is correct, so relying on software to correct hardware errors is like asking Godzilla to prevent Mega-Godzilla from terrorizing Japan. THIS DOES NOT LEAD TO RISING PROPERTY VALUES IN TOKYO.

  I happen to work for a company whose software uses checksums at many layers, and RAID encoding and low-density parity codes at the lowest layers, to detect and recover from hardware failures. It works pretty well, and the company has sold billions of dollars of products of which that is a key component. Also, many (most?) enterprise servers use RAM with error-correcting codes; I think the common configuration allows it to correct single-bit errors and detect double-bit errors, and my company’s machines will reset themselves when they detect double-bit errors and other problems that impugn the integrity of their runtime state.

  One could quibble about whether “retrieving and querying the data that was written” counts as a “computation”, and the extent to which the recovery is achieved through software as opposed to hardware^[1], but the source material is a James Mickens comedic rant in any case.

  I’d say the important point here is: There is a science to error correction, to building a (more) perfect machine out of imperfect parts, where the solution to unreliable hardware is more of the unreliable hardware, linked up in a clever scheme. They’re good enough at it that each successive generation of data storage technology uses hardware with higher error rates. You can make statements like “If failures are uncorrelated, and failures happen every X time units on average per component, and it takes Y time units to detect and recover a failure, and we can recover from up to M failures out of every group of N components, then on average we will have an unrecoverable failure every Z time units”; then you can (a) think about how to arrange it so that Z >> X and (b) think about the dangers of correlated failures.

  (The valid complaint that Mickens’s character makes is that it would suck if every application needed to weave error correction into every codepath, implement its own RAID, etc. It works much better if the error correction is done by some underlying layer that the application treats as an abstraction—using the abstraction tends to be more complex than pretending errors don’t exist (and for noncritical applications the latter is a valid strategy), but not terrible.^[2])

  With regard to AI. It seems likely that we’ll end up making use of potentially-dangerous AIs to do things. If we do, then we’d want powerful safeguards. It seems unlikely that we’d have 100% confidence in any particular safeguard (i.e. unlikely we’d have formal proofs of the safety properties we’d want). Then we’d want to keep adding more safeguards as long as their failure modes weren’t totally covered by those of preexisting safeguards (and as long as they’re affordable); ideally we’d try to estimate (ideally measure) the chance of failure of each safeguard and how independent they are.

  Now, would some of these safeguards be built with the aid of earlier versions of AI? I’d guess yes. This could take a few forms, ranging from “formally-verified direct programs that the earlier AI generated along with their proofs of safety and effectiveness” to “smaller neural nets (cut-down versions of earlier AIs?) that monitor the training of the new AI’s net” to “weaker full-blown AIs watching the decisions output by the new full-blown AI”. I suspect any of these might be useful.

  The Godzilla complaint seems to be about that last thing, having weaker AIs monitor the new AI. Well, um, it seems to me that, if the weaker AIs’ output is restricted to the one-bit signal “do we hit the killswitch on the new AI” (and that signal is executed by a machine), then there’s no danger of Godzilla destroying Tokyo after it has taken down Mega-Godzilla. (It’s like cutting off Godzilla’s limbs and tying him to Mega-Godzilla with his jaws open around Mega-Godzilla’s neck, so his only available action is biting down.) There is a danger, if this happens a lot, of the subsequent training of the new AI merely teaching it to evade detection by the weaker predecessors; this danger should be taken seriously; but it is also less bad than having Mega-Godzilla kill us the first time it started going rogue; the safeguard is net positive.

  Addressing specific claims:

  If we use one AI to oversee another AI, and something goes wrong, that’s not a recoverable error; we’re using AI assistance in the first place because we can’t notice the relevant problems without it.

  If, among our many precautions, we have AI 1 oversee AI 2 (in this scheme I describe where AI 1′s only available action is to output a bit that shuts off AI 2)^[3], and AI 2 starts to go rogue, and AI 1 fails in its task, then we’re no worse off than if we lacked that particular precaution—either the other precautions save us or they don’t. If AI 1 succeeds, then we have at least some chance of being able to debug what went wrong with AI 2, and save ourselves in the future. Using AI 1 as overseer is strictly beneficial, minus implementation costs. (There’s a second-order risk of someone overestimating how much it protects us and then choosing to be more reckless, but that’s a different kind of thing.)

  If we use one maybe-somewhat-aligned AI to build another, and something goes wrong, that’s not a recoverable error

  If “goes wrong” means “it kills us all”, then yeah. If “goes wrong” means “it starts acting dangerous but we manage to shut it off”, that is potentially recoverable.

  if we had better ways to detect misalignment in the child we’d already have used them on the parent.

  The parent could be aligned but incompetent (heh, I think many people know parents like that), while the child is unaligned and more competent. (Why would the programmers create the child if it weren’t expected to be more competent than the parent?) It seems easy to imagine alignment checks that the parent would pass while the child would fail.

  1. ^

     The checksums are done by the CPU but hardware-accelerated; I think the RAID stuff is clearly software; the LDPC stuff is done by more specialized hardware; the ECC RAM is hardware, but the choice of how to respond to double bit errors is software.

  2. ^

     ECC RAM is one example: the application can just assume that memory accesses are correct, and that double-bit errors will cause the operating system to kill the application—so the application itself needs zero lines of code about RAM errors, though now there does need to be monitoring to restart the process and/​or fail over to another machine.

     Another example is using a durable database, and going from “just increment and decrement the variables in memory” to “create a database transaction, to be attempted /​ rolled back as a unit”. That does require code changes in the application itself.

  3. ^

     In practice we’d probably want it to output something more like a number between 0 and 1 representing its worry level, and likely the killswitch goes off if either a single overseer exceeds some danger threshold or multiple overseers exceed some lower threshold.

• localdeity 16 Mar 2022 23:02 UTC

  59 points

  in reply to: StellaAthena’s comment on: Challenges to Yud­kowsky’s Pro­noun Re­form Proposal

  Not OP, but for what it’s worth, I consider it unreasonable to request that other people think of you in a certain way (be it gender, or having personal traits or skills or anything), or at least for there to be any sense of expectation or obligation that they will fulfill such a request. That would be actual thought-policing, and abhorrent to me. It’s reasonable to want people to think of you a certain way, to hope that they will, to take actions that will hopefully increase the likelihood of it, and possibly to only be close friends with people who do think of you that way (although I think it’s usually wise to try not to care too much about what others think about you). But I feel strongly that people have a right to think whatever thoughts they want, and that anything that seems to be punishing people for thoughts (as opposed to actions or speech) should raise major alarm bells.

  Therefore, to the extent that people are told “You should accommodate the desires of trans people, and will be told you’re a bad person and possibly face social consequences if you don’t”, expected accommodations of the form “Think of them as female even if your brain naturally classifies them as male” are unreasonable. “Avoid speech or other visible actions that rub in their face the fact that you think of them as male” is polite, and may be reasonable to expect; “Use these pronouns when referring to this person in front of them” is an obvious example of that.

  So that is the strong-request/​demand that it’s reasonable for people to get from “society”. (If people in power were unambiguously saying “In order to be polite and not be called bad, you must think of these people in a certain way”, then I think there would be revolts.) If someone hasn’t become emotionally close friends with any trans people, I’d say it’s not too surprising if they haven’t picked up on something subtler than “socially enforced rules”.

• localdeity 3 Feb 2023 22:28 UTC

  43 points

  8

  in reply to: Eliezer Yudkowsky’s comment on: You Don’t Ex­ist, Duncan

  In case anyone finds it validating or cathartic, you can read user interaction professionals explain that, yes, things are often designed with horrible, horrible usability.^[1] Bruce Tognazzini has a vast website.

  Here is one list of design bugs. The first one is the F-16 fighter jet’s flawed weapon controls, which caused pilots to fire its gun by mistake during training exercises (in one case shooting a school—luckily not hitting anyone) on four occasions in one year; on the first three occasions, they blamed pilot error, and on the fourth, they still blamed pilot error but also acknowledged that “poorly-designed controls” contributed to the incident.

  Here is another list. Item 3 I’ll quote below:

  Bug Name: Automobile Self-Destruct Switch

  Product: Remco Lube Pump for Lexus RX-300

  Bug: The driver must accurately toggle a hidden, completely unlabelled switch inside the engine compartment in response to changing conditions. If, even once, the switch is forgotten or flipped the wrong way, it will destroy the $5000 engine and transmission within five minutes.

  More detail:

  Calling the company was of no help. The engineer who answered responded that nothing was wrong with the design of the switch that extremely careful operation would not overcome. He’d been using it for months with no problem.

  The problem could be easily corrected by the manufacturer replacing this manual switch with a solenoid-driven switch that only kicks in when the car has been connected for towing. This would add little to the already high price and would replace certain anxiety and uncertain calamity with a solid, dependable result.

  The Design of Everyday Things (by Don Norman, Tognazzini’s more-famous colleague) is an entire book about good and bad design. Excerpting from the chapter “Why Designers Go Astray”:

  “It probably won a prize” is a disparaging phrase in this book. Why? Because prizes tend to be given for some aspects of a design, to the neglect of all others—usually including usability. Consider the following example, in which a usable, livable design was penalized by the design profession. The assignment was to design the Seattle offices of the Federal Aviation Administration (FAA). The most noteworthy feature of the design process was that those who would work in the building had a major say in the planning. [...]

  So there really were two designs: one in Seattle, with heavy participation by the users, and one in Los Angeles, designed in the conventional manner by architects. Which design do the users prefer? Why the Seattle one, of course. Which one got the award? Why the Los Angeles one, of course. [...]

  Aesthetics, not surprisingly, comes first at museums and design centers. I have spent much time in the science museum of my own city, San Diego, watching visitors try out the displays. The visitors try hard, and although they seem to enjoy themselves, it is quite clear that they usually miss the point of the display. The signs are highly decorative; but they are often poorly lit, difficult to read, and have lots of gushing language with little explanation. Certainly the visitors are not enlightened about science (which is supposed to be the point of the exhibit). Occasionally I help out when I see bewildered faces by explaining the scientific principles being demonstrated by the exhibit (after all, many of the exhibits in this sort of museum are really psychology demonstrations, many of which I explain in my own introductory classes). I am often rewarded with smiles and nods of understanding. I took one of my graduate classes there to observe and comment; we all agreed about the inadequacy of the signs, and, moreover, we had useful suggestions. We met with a museum official and tried to explain what was happening. He didn’t understand. His problems were the cost and durability of the exhibits. “Are the visitors learning anything?” we asked. He still didn’t understand. Attendance at the museum was high. It looked attractive. It had probably won a prize. Why were we wasting his time?

  1. ^

     At first glance, this is because designers are stupid assholes. At second glance, designers (a) are usually rather different people than the intended user on several dimensions [which is difficult to compensate for even when you’re trying], and (b) often face bad incentives, such as (c) design competition awards being based solely on aesthetics rather than functionality and (d) the purchaser of a product being unsophisticated and not the intended user [e.g. a manager or director buys a product that the lowest-level employees will use], and hence having little to go on except aesthetics (and reputation).

     At third glance, some designers really are stupid assholes. Seriously:

     [Frank Lloyd Wright’s] chair design originally had only three legs, supposedly to encourage better posture (because one would have to keep both feet on the ground at all times to sit in it). However, the chair proved unstable, tipping very easily. Purportedly, Wright redesigned the chairs after Herbert Johnson asked him to sit in one, and he fell out of it.

• localdeity 7 Sep 2021 18:33 UTC

  42 points

  on: I read “White Frag­ility” so you don’t have to (but maybe you should)

  Racism, as DiAngelo uses the word, does not mean the explicit profession that there are essentially different human races and that some are better than others. That, she says, is an unsophisticated folk definition of racism (I’ll call that “racism_F”).

  The definition she prefers (what I’ll call “racism_S”) is that racism_S is a systemic, usually (nowadays) non-explicit or euphemistic, often subconscious, interlocking and pervasive set of social, cultural, and political devices that reinforce white supremacy. [...]

  The folk definition, racism_F, is in fact one of the pillars of white fragility. Because, according to this definition, racism_F is the conscious, explicit endorsement of an unconscionable belief system — all we white people have to do to stop participating in racism_F is to disavow racial bigotry and then congratulate ourselves for our good sense.

  I would maintain the opposite: that racism_F is the original definition of racism, which deservedly acquired its strong negative connotations (think of the image of a good, virtuous, even altruistic minority member, consistently better behaved than most white people, and a white person saying “Yeah, well, he’s still bad and not to be trusted because he isn’t white”); and that activists like DiAngelo are trying to push their own new definition of racism as racism_S, probably for political purposes. (Ask yourself why they don’t just use a different term to avoid confusion. They could consistently say “structural racism”, but do they?) Specifically, it seems that they’re trying to take advantage of the negative affect towards racism_F and everyone’s intuitive reaction of “Well, of course racism is bad and no one should be racist”, by claiming that being against racism means being on board with all of their increasingly extreme proposals, and threatening to tar any opposition as racist. It seems to be a disappointingly effective tactic.

  Some of the activists probably genuinely believe that racism_S is the correct definition. Likely even the majority; it only takes a few at the top of their intellectual hierarchy to promulgate a new definition in academia, and then those who look up to the few might just assume they’re justified and propagate it to the rest. However, since DiAngelo is in the business of saying “just because you don’t have any bad intentions doesn’t excuse you from participating in a system that leads to the same result”, I feel I should extend to her the same level of charity.

• localdeity 27 Apr 2023 5:08 UTC

  39 points

  16

  in reply to: habryka’s comment on: Moder­a­tion notes re: re­cent Said/​Dun­can threads

  All right, I’ll give it a try (cc @Said Achmiz).

  Enforcing norms of any kind can be done either by (a) physically preventing people from breaking them—we might call this “hard enforcement”—or (b) inflicting unpleasantness on people who violate said norms, and/​or making it clear that this will happen (that unpleasantness will be inflicted on violators), which we might call “soft enforcement”.^[1]

  Bans are hard enforcement. Downvotes are more like soft enforcement, though karma does matter for things like sorting and whether a comment is expanded by default, so there’s some element of hardness. Posting critical comments is definitely soft enforcement; posting a lot of intensely critical comments is intense soft enforcement. Now, compare with Said’s description elsewhere:

  On Less Wrong, there are moderators, and they unambiguously have a multitude of enforcement powers, which ordinary users lack. Ordinary users have very few powers: writing posts and comments, upvotes/​downvotes, and bans from one’s posts.

  Writing posts and comments isn’t anything at all like “enforcement” (given that moderators exist, and that users can ignore other users, and ban them from their posts).

  Said is clearly aware of hard enforcement and calls that “enforcement”. Meanwhile, what I call “soft enforcement”, he says isn’t anything at all like “enforcement”. One could put this down to a mere difference in terms, but I think there’s a little more.

  It seems accurate to say that Said has an extremely thick skin. Probably to some extent deliberately so. This is admirable, and among other things means that he will cheerfully call out any local emperor for having no clothes; the prospect of any kind of social backlash (“soft enforcement”) seems to not bother him, perhaps not even register to him. Lots of people would do well to be more like him in this respect.

  However, it seems that Said may be unaware of the degree to which he’s different from most people in this^[2]. (Either in naturally having a thick skin, or in thinking “this is an ideal which everyone should be aspiring to, and therefore e.g. no one would willingly admit to being hurt by critical comments and downvotes”, or something like that.) It seems that Said may be blind to one or more of the below:

  • That receiving comments (a couple or a lot) requesting more clarification and explanation could be perceived as unpleasant.

  • That it could be perceived as so unpleasant as to seriously incentivize someone to change their behavior.

  I anticipate a possible objection here: “Well, if I incentivize people to think more rigorously, that seems like a good thing.” At this point the question is “Do Said’s comments enforce any norm at all?”, not “Are Said’s comments pushing people in the right direction?”. (For what it’s worth, my vague memory includes some instances of “Said is asking the right questions” and other instances of “Said is asking dumb questions”. I suspect that Said is a weird alien (most likely “autistic in a somewhat different direction than the rest of us”—I don’t mean this as an insult, that would be hypocritical) and that this explains some cases of Said failing to understand something that’s obvious to me, as well as Said’s stated experience that trying to guess what other people are thinking is a losing game.)

  Second anticipated objection: “I’m not deliberately trying to enforce anything.” I think it’s possible to do this non-deliberately, even self-destructively. For example, a person could tell their friends “Please tell me if I’m ever messing up in xyz scenarios”, but then, when a friend does so, respond by interrogating the friend about what makes them qualified to judge xyz, have they ever been wrong about xyz, were they under any kind of drugs or emotional distraction or sleep deprivation at the time of observation, do they have any ulterior motives or reasons for self-deception, do their peers generally approve of their judgment, how smart are they really, what were their test scores, have they achieved anything intellectually impressive, etc. (This is avoiding the probably more common failure mode of getting offended at the criticism and expressing anger.) Like, technically, those things are kind of useful for making the report more informative, and some of them might be worth asking in context, but it is easy to imagine the friend finding it unpleasant, either because it took far more time than they expected, or because it became rather invasive and possibly touched on topics they find unpleasant; and the friend concluding “Yeesh. This interaction was not worth it; I won’t bother next time.”

  And if that example is not convincing (which it might not be for someone with an extremely thick skin), then consider having to file a bunch of bureaucratic forms to get a thing done. By no means impossible (probably), but it’s unpleasant and time-consuming, and might succeed in disincentivizing you from doing it, and one could call it a soft forbiddance.^[3] (See also “Beware Trivial Inconveniences”.)

  Anyway, it seems that the claim from various complainants is that Said is, deliberately or not, providing an interface of “If your posts aren’t written in a certain way, then Said is likely to ask a bunch of clarifying questions, with the result that either you may look ~unrigorous or you have to write a bunch of time-consuming replies”, and thus this constitutes soft-enforcing a norm of “writing posts in a certain way”.

  Or, regarding the “clarifying questions need replies or else you look ~unrigorous” norm… Actually, technically, I would say that’s not a norm Said enforces; it’s more like a norm he invokes (that is, the norm is preexisting, and Said creates situations in which it applies). As Said says elsewhere, it’s just a fact that, if someone asks a clarifying question and you don’t have an answer, there are various possible explanations for this, one of which is “your idea is wrong”.^[4] And I guess the act of asking a question implies (usually) that you believe the other person is likely to answer, so Said’s questions do promulgate this norm even if they don’t enforce it.

  Moreover, this being the website that hosts Be Specific, this norm is stronger here than elsewhere. Which… I do like; I don’t want to make excuses for people being unrigorous or weak. But Eliezer himself doesn’t say “Name three examples” every single time someone mentions a category. There’s a benefit and a cost to doing so—the benefit being the resulting clarity, the cost being the time and any unpleasantness involved in answering. My brain generates the story “Said, with his extremely thick skin (and perhaps being a weird alien more generally), faces a very difficult task in relating to people who aren’t like him in that respect, and isn’t so unusually good at relating to others very unlike him that he’s able to judge the costs accurately; in practice he underestimates the costs and asks too often.”

  1. ^

     And usually anything that does (a) also does (b). Removing someone’s ability to do a thing, especially a thing they were choosing to do in the past, is likely unpleasant on first principles; plus the methods of removing capabilities are usually pretty coarse-grained. In the physical world, imprisonment is the prototypical example here.

  2. ^

     It also seems that Duncan is the polar opposite of this (or at least is in that direction), which makes it less surprising that it’d be difficult for them to come to common understanding.

  3. ^

     There was a time at work where I was running a script that caused problems for a system. I’d say that this could be called the system’s fault—a piece of the causal chain was the system’s policy I’d never heard of and seemed like the wrong policy, and another piece was the system misidentifying a certain behavior.

     In any case, the guy running the system didn’t agree with the goal of my script, and I suspect resented me because of the trouble I’d caused (in that and in some other interactions). I don’t think he had the standing to say I’m forbidden from running it, period; but what he did was tell me to put my script into a pull request, and then do some amount of nitpicking the fuck out of it and requesting additional features; one might call it an isolated demand for rigor, by the standards of other scripts. Anyway, this was a side project for me, and I didn’t care enough about it to push through that, so I dropped it. (Whether this was his intent, I’m not sure, but he certainly didn’t object to the result.)

  4. ^

     Incidentally, the more reasonable and respectable the questioner looks, that makes explanations like “you think the question is stupid or not worth your time” less plausible, and therefore increases the pressure to reply on someone who doesn’t want to look wrong. (One wonders if Said should wear a jester’s cap or something, or change his username to “troll”. Or maybe Said can trigger a “Name Examples Bot”, which wears a silly hat, in lieu of asking directly.)

• localdeity 1 Feb 2023 6:48 UTC

  35 points

  14

  on: Aiming for Con­ver­gence Is Like Dis­cour­ag­ing Betting

  So why is the advice “behave as if your interlocutors are also aiming for convergence on truth”, rather than “seek out conversations where you don’t think your interlocutors are aiming to converge on truth, because those are exactly the conversations where you have something substantive to say instead of already having converged”?

  [...] To see why, substitute “making money on prediction markets” for “moving closer to truth”, “betting” for “updating”, and “trying to make money on prediction markets” for “seeking truth”

  The one should not be substituted for the other, because there are important differences in the goals.

  On a betting market, if you have a knowledge edge, it’s in your interest to keep it that way, to the extent possible. Obviously, the fact of your making bets leaks information, but you don’t want information to leak via any other means. If you have a brilliant weather model that’s 10x more accurate than everyone else’s, you definitely don’t want to publish it on your website; you want to keep winning bets against people with worse models. In fact, if you have the opportunity to verbally praise and promote the wrong models, it’s in your interest to do so; and if, for some reason, you have to publish the details of your weather model, it’s in your interest to make your writeup as confusing, inscrutable, and hard to implement as possible.

  If, on a forum, you think you “win points” solely by writing correct arguments when others are wrong, then it’s in your interest to make sure no one else learns from the things you write, so you can keep winning. If you have an opportunity to phrase something more offensively, take it, so your opponents are more likely to get angry, reject your correct arguments, and stay wrong. And, for that matter, why explain your reasoning? Why not just say “You’re wrong, you stupid f***; X is the truth”?

  I don’t think you actually believe that you “win points” solely by writing correct arguments when others are wrong. I suspect you have a notion of what “making proper arguments” is—and it involves clearly explaining your reasoning and such—and view participation in the forum as a game in which participants are trying to be the best at “making proper (and novel) arguments”. Well, it seems like we could choose whatever notion of a “proper argument” we liked, and upvote arguments to the extent that they match the ideal, and at least in theory we’d end up with posts of the type we’re rewarding—so we need to decide what we want to reward, and presumably “clearly stated arguments that aren’t deliberately trying to enrage people” are part of what we’d like to end up with.

  So, exactly what type of posts do we want people to be trying to write? One strategic decision, which I think Duncan makes and I’m not sure of your opinion on, is to try to get lots of value from participants who are fairly good but imperfect—specifically, are at least somewhat prone to turn arguments into slap fights if they feel like they’ve been slapped (and evolutionary processes have created memes that encourage people to view lots of things as slaps)—and therefore to have the “ideal posting goals” call for error-correcting mechanisms and stuff that make this less likely.

  (An alternate strategy would be “Assume that all participants we care about are the platonic ideal, who won’t take any bait and never let anger or any other emotion bring them to any wrong decisions; rely on downvotes to purge any bad behavior.” This could be a good approach, especially if you think this platonic ideal is easy to achieve. However, if there are actually quite a lot of imperfect participants, this could go badly. I will merely say that this would be more appropriate for a website called Never Wrong.)

  [Why not] “seek out conversations where you don’t think your interlocutors are aiming to converge on truth, because those are exactly the conversations where you have something substantive to say instead of already having converged”?

  It depends somewhat on one’s model here. Ideally, interlocutors who aren’t aiming to converge on the truth, and write bad posts as a result, will get downvoted, and then we don’t need to care about them. Or maybe the socially enforced rules will end up pushing them into writing posts that are actually good even if they didn’t mean them to be; that’s a fine outcome. Also, given that somewhat bad posts exist, another strategy is to find them and write a really good reply that enlightens the readers and may even push the authors of the bad posts to write better replies themselves; that also seems like a good outcome, therefore one we’d want to reward. (A possible downside: replying at all does attract more eyes to the conversation—e.g. the frontpage does show recent comments—and if the conversation leading up to your post is bad enough, it may be net negative to the reader if your great reply wasn’t so great as to outweigh that.)

  So, yes, it may in fact make sense to seek out conversational cesspools and write comments to improve them. The difference with betting markets is: with a betting market, you hope this keeps happening so you can keep profiting off others’ ignorance; but on a forum where the goals are what I think they are, you hope that the participants and observers learn to stop creating cesspools—or, well, you hope for whatever you want^[1], but you act as though that’s your goal, and do your best in your comment to encourage future good behavior, because that’s what the forum ideally rewards.

  1. ^

     There is potentially the issue, pointed out in some fictional stories and sometimes in real life, where if someone’s identity /​ fulfillment /​ most profitable career path is “swooping in to save everyone from instances of problem X”, then they may have the perverse incentive to discourage anyone else from solving X in general. Luckily, the tragedy of the commons can help us here: though it might e.g. benefit cardiologists collectively if everyone had horrible nutrition, it’s unlikely to be worthwhile to any individual cardiologist to spend the effort lobbying for that.

• localdeity 26 Sep 2022 23:49 UTC

  31 points

  11

  in reply to: Amelia Bedelia’s comment on: An­nounc­ing Balsa Research

  If you haven’t seen it, there’s a thread here with links to Sarah Constantin’s postmortem and Zvi’s semi-postmortem, plus another comment from each of them.

  I’ll excerpt Zvi’s comment from that thread:

  Most start-ups fail. Failing at a start-up doesn’t even mean that you, personally are bad at start-ups. If anything the SV-style wisdom is that it means you have experience and showed you will give it your all, and should try again! You don’t blow your credibility by taking investor money, having a team that gives it their all for several years, and coming up short.

• localdeity 16 Jul 2023 3:55 UTC

  30 points

  17

  in reply to: tailcalled’s comment on: A Hill of Val­idity in Defense of Meaning

  The way I had imagined the situation is, someone working with the Googlegeist had noticed that a lot of women reported anxiety or whatever, and had decided they need to work with women to figure out what’s going on here, to solve it. And then James Damore felt that this was one instance of people looking at a disparity and claiming injustice, and that since he finds it biologically inevitable that women would be anxious, this shouldn’t be treated as indicative of an external problem, but instead should be medicalized and treated psychologically (or psychiatrically?). [italics added]

  As a side note, I consider the italicized part a rather weighty accusation. I think one should therefore be careful about making such an accusation. I guess, in this case, you were just honestly reporting the contents of your brain on the matter, not necessarily making an accusation.

  Still, I think this to some extent illustrates an epistemic environment where it’s normal to throw around damaging accusations whose truth value is somewhere between “extremely uncharitable interpretation” and “objectively false”. Precisely the type that got Damore fired, in other words. Do we have such an environment even among rationalists? That is at the heart of Zack’s adventure.

  (Incidentally, imagine if Damore had claimed the opposite—”Women are less prone to anxiety and can handle stress more easily.” Wouldn’t that also lead to accusations that Damore was saying we can ignore women’s problems?)

  Anyway, on to object level. I think Damore’s point, in bringing it up, was that the stress in (some portion of) tech jobs may be a reason there are fewer women than men in tech. Reasons to think this:

  • The title of the super-section containing the “neuroticism” quote is “Possible non-bias causes of the gender gap in tech”.

  • The super-section is preceded by “For the rest of this document, I’ll concentrate on the extreme stance that all differences in outcome are due to differential treatment [italics added] and the authoritarian element that’s required to actually discriminate to create equal representation.”

  • The last sentence in the section (“Personality differences”) is “We need to stop assuming that gender gaps imply sexism.”

  • As already quoted, he says that the anxiety thing implies that “Mak[ing] tech and leadership less stressful” would be a “non-discriminatory way to reduce the gender gap”.

  If Damore had said “Here are some issues women reported; and we should discount these reports because women are extra-anxious”, then your model would be well-founded. I don’t see him saying anything like that in the document, though. In the whole document, Damore doesn’t mention anything reported by women on Googlegeist, other than the anxiety thing. (I would be surprised if he, being an engineer and not in HR or leadership, had access to the arbitrary text field submissions from the other employees; I would guess he saw aggregated results on numerical questions, plus any items leadership chose to share with everyone.) Googlegeist itself is mentioned only two other times in the document; both times it’s him suggesting something be done with future Googlegeist surveys.

  He does mention another item as a (primarily) women’s issue, although the source is a 2006 paper rather than Googlegeist. Again, he does advocate doing something about it (with caveats):

  Non-discriminatory ways to reduce the gender gap

  [...]

  • Women on average look for more work-life balance while men have a higher drive for
    status on average
    ○ Unfortunately, as long as tech and leadership remain high status, lucrative
    careers, men may disproportionately want to be in them. Allowing and truly
    endorsing (as part of our culture) part time work though can keep more women in
    tech.

  Now, at the end, he says this:

  Philosophically, I don’t think we should do arbitrary social engineering of tech just to make it appealing to equal portions of both men and women. For each of these changes, we need principled reasons for why it helps Google; that is, we should be optimizing for Google—with Google’s diversity being a component of that. For example, currently those willing to work extra hours or take extra stress will inevitably get ahead and if we try to change that too much, it may have disastrous consequences. Also, when considering the costs and benefits, we should keep in mind that Google’s funding is finite so its allocation is more zero-sum than is generally acknowledged.

  The most uncharitable reader could say “Aha, so he’s laid the groundwork to not follow through with anything that actually helps women, keeping the status quo, and everything he’s said before is just a trick.” If the reader comes in with that kind of implicit assumption about Damore’s character, then they’ll probably stick with it; all I can say is, evidence for such a belief does not come from the document. (Incidentally, I’ve met Damore at a party; I read him as a well-meaning nerd, who thought that if he made a sufficiently comprehensive, careful, well-cited, and constructively oriented writeup, he could cut through the hostility and they’d work out some solutions that would make everyone happier. The result is really tragic in that light.)

  I think, to come up with your conclusion, you have to do a lot of reading into the text, and a lot of not reading the actual text. Which, I think, was par for the course for most negative takes on Damore. I am surprised and somewhat perturbed by your report that you originally supported Damore, and wonder what happened since then. Perhaps memory faded and “osmosis” brought in others’ takes?

• localdeity 6 Nov 2021 11:22 UTC

  30 points

  on: Speak­ing of Stag Hunts

  My brain notes, in passing, that Eternal September, and the Septembers that preceded it, can be described in terms of concentration of force. If a forum has a certain culture, and a bunch of noobs come in without it (instead exhibiting some kind of “mainstream lowest-common-denominator” culture, incompatible with that of the forum)… If they come in one by one, then they’ll face negative reinforcement (downvotes and/​or critical comments), pushing them to either adopt the forum’s culture or leave; if they arrive in a clump, then they’ll be in a position to positively support each other, reducing the negative-reinforcement effect. If enough of them arrive in a group, then the forum’s “immune system” may fail to stop them, and they may end up changing the forum’s culture.

  Today, a sudden influx of users tends to come from some big event. Like when a hugely popular site drops a link to your forum, or when there’s a huge story on your forum that draws a lot of interest from outsiders. (The Leverage story is one of the latter; a story like Zoe Curzi’s is fascinating to humans. I told a non-rationalist friend about it, and he said it was very juicy gossip. It also came up in discussions with rationalists, some of whom are more active than others on LW; I wonder if the posts were also linked to on other sites. I wonder if LW admins could confirm patterns like “The recent huge threads about Leverage and about MIRI had a higher proportion of non-users, of new users, and of less-regular users than most other threads.” HTTP referrer headers can give the proximate source of inbound links, while interpersonal gossip is harder to track. Actually, regarding links, habryka posted an image recently [incidentally, the top post, “I don’t know how to count that low”, is an example of getting linked to by Hacker News] … but although it’s interesting, I don’t think it directly addresses my above question.)

  So, it seems like sudden large influxes of users, attributable to big stories or big inbound links, are a danger to a forum’s culture. That seems to be “common/​received wisdom” from some portions of the internet I’ve occupied.

  This brings to mind a funny episode from Hacker News’s history, where its creator posted something like “We’ve gotten written about by a major news site and will probably be flooded by mainstream people who want to talk about politics and don’t care about technical subjects, so please make an extra effort to post and upvote links about Erlang internals and things like that to discourage those people”, and existing literal-minded users took this to heart and filled the front page with nothing but Erlang links.

  Anyway, combining one of the proposals with “concentration of force”, this generates the following idea: If you have something like “dedicated volunteers or paid posters”, then by far the best time to deploy them is when you have what looks like one of these big influxes. (It seems an obvious enough combination that I’m mildly surprised that this wasn’t on Duncan’s list of terrible-idea suggestions. Perhaps because it goes against the focus on small things? Heh, Gunnar_Zarncke has the same suggestion.) My impression is that existing moderators did put a bunch of time into following the huge threads; but, like, if you had a squad of “reserve forces” of extra moderators kept on retainer, who only get called in occasionally, that’s probably more efficient and effective.

• localdeity 17 Jul 2023 7:21 UTC

  29 points

  11

  in reply to: tailcalled’s comment on: A Hill of Val­idity in Defense of Meaning

  I’ll address this first:

  More abstractly, what I’ve generally noticed is:

  • These sorts of people are not very interested in actually developing substantive theory or testing their claims in strong ways which might disprove them.

  • Instead they are mainly interested in providing a counternarrative to progressive theories.

  • They often use superficial or invalid psychometric methods.

  • They often make insinuations that they have some deep theory or deep studies, but really actually don’t.

  These things are bad, but, apart from point 2, I would ask: how do they compare to the average quality of social science research? Do you have high standards, or do you just have high standards for one group? I think most of us spend at least some time in environments where the incentive gradients point towards the latter. Beware isolated demands for rigor.

  Research quality being what it is, I would recommend against giving absolute trust to anyone, even if they appear to have earned it. If there’s a result you really care about, it’s good to pick at least one study and dig into exactly what they did, and to see if there are other replications; and the prior probability of “fraud” probably shouldn’t go below 1%.

  As for point 2—if you were a researcher with heretical opinions, determined to publish research on at least some of them, what would you do? It seems like a reasonable strategy is to pick something heretical that you’re confident you can defend, and do a rock-solid study on it, and brace for impact. Is it still the case that disproving the blank-slate hypothesis would constitute progress in some academic subfields? If so, then expect people to continue trying it.

  Now, digging into the examples:

  Here’s a classical example; an IQ researcher who is so focused on providing a counternarrative to motivational theories that he uses methods which are heavily downwards biased to “prove” that IQ test scores don’t depend on effort.

  The study says there was “a meta-analysis concluding that small monetary incentives could improve test scores by 0.64 SDs” (roughly 10 IQ points); looks to be Duckworth et all 2011. The guy says it seemed sketchy—the studies had small N, weird conditions, and/​or fraudulent researchers. Looking at table S1 from Duckworth, indeed, N is <100 on most of the studies; “Bruening and Zella (1978)” sticks out as having a large effect size and a large N, and, when I google for more info about that, I find that Bruening was convicted by an NIMH panel of scientific fraud. Checks out so far.

  The guy ran a series of studies, the last of which offered incentives of nil, £2, and £5-£10 for test performance, with the smallest subgroup being N=150, taken from the adult population via “prolific academic”. He found that £2 and £5-£10 had similar effects, those being apparently 0.2 SD and 0.15 SD respectively, which would be 3 IQ points or a little less. (Were the “small monetary incentives” from Duckworth of that size? The Duckworth table shows most of the studies as being in the $1-$9 or <$1 range; looks like yes.) So, at least as a “We suspected these results were bogus, tried to reproduce them, and got a much smaller effect size”, this seems all in order.

  Now, you say:

  IQ test effort correlates with IQ scores, and they investigate whether it is causal using incentives. However, as far as I can tell, their data analysis is flawed, and when performed correctly the conclusion reverses.

  [...] Incentives increase effort, but they only have marginal effects on performance. Does this show that effort doesn’t matter? No, because incentives also turn out to only have marginal effects on effort! Surely if you only improve effort a bit, you wouldn’t expect to have much influence on scores. We can solve this by a technique called instrumental variables. Basically, we divide the effect of incentives on scores by the effect of incentives on effort.

  Your analysis essentially proposes that, if there were some method of increasing effort by 3-4x as much as he managed to increase it, then maybe you could in fact increase IQ scores by 10 points. This assumes that the effort-to-performance causation would stay constant as you step outside the tested range. That’s possible, but… I’m quite confident there’s a limit to how much “effort” can increase your results on a timed multiple-choice test, that you’ll hit diminishing marginal returns at some point (probably even negative marginal returns, if the incentive is strong enough to make many test-takers nervous), and extrapolating 3-4x outside the achieved effect seems dubious. (I also note that the 1x effect here means increasing your self-evaluated effort from 4.13 to 4.28 on a scale that goes up to 5, so a 4x effect would mean going to 4.73, approaching the limits of the scale itself.)

  You say, doing your analysis:

  For study 2, I get an effect of 0.54. For study 3, I get an effect of 0.37. For study 4, I get an effect of 0.39. The numbers are noisy for various reasons, but this all seems to be of a similar order of magnitude to the correlation in the general population, so this suggests the correlation between IQ and test effort is due to a causal effect of test effort increasing IQ scores.

  That is interesting… Though the correlation between test effort and test performance in the studies is given as 0.27 and 0.29 in different samples, so, noise notwithstanding, your effects are consistently larger by a decent margin. That would suggest that there’s something else going on than the simple causation.

  The authors say:

  6.1. Correlation and direction of causality

  Across all three samples and cognitive ability tests (sentence verification, vocabulary, visual-spatial reasoning), the magnitude of the association between effort and test performance was approximately 0.30, suggesting that higher levels of motivation are associated better levels of test performance. Our results are in close accord with existing literature [...]

  As is well-known, the observation of a correlation is a necessary but not sufficient condition for causality. The failure to observe concomitant increases in test effort and test performance, when test effort is manipulated, suggests the absence of a causal effect between test motivation and test performance.

  That last sentence is odd, since there was in fact an increase in both test effort and test performance. Perhaps they’re equivocating between “low effect” and “no effect”? (Which is partly defensible in that the effect was not statistically significant in most of the studies they ran. I’d still count it as a mark against them.) The authors continue:

  Consequently, the positive linear assocation between effort and performance may be considered either spurious or the direction of causation reversed – flowing from ability to motivation. Several investigations have shown that the correlation between test-taking anxiety and test performance likely flows from ability to test-anxiety, not the other way around (Sommer & Arendasy, 2015; Sommer, Arendasy, Punter, Feldhammer-Kahr, & Rieder, 2019). Thus, if the direction of causation flows from ability to test motivation, it would help explain why effort is so difficult to shift via incentive manipulation.

  6.2. Limitations & future research

  We acknowledge that the evidence for the causal direction between effort and ability remains equivocal, as our evidence rests upon the absence of evidence (absence of experimental incentive effect). Ideally, positive evidence would be provided. Indirect positive evidence may be obtained by conducting an experiment, whereby half the subjects are given a relatively easy version of the paper folding task (10 easiest items) and the other half are given a relatively more difficult version (10 most difficult items). It is hypothesized that those given the relatively easier version of the paper folding task would then, on average, self-report greater levels of test-taking effort. Partial support for such a hypothesis is apparent in Table 1 of this investigation. Specifically, it can be seen that there is a perfect correspondence between the difficulty of the test (synonyms mean 73.4% correct; sentence verification mean 53.8% correct; paper folding mean 43.3%) and the mean level of reported effort (synonyms mean effort 4.42; sentence verification mean 4.11; paper folding mean 3.83).

  That is a pretty interesting piece of evidence for the “ability leads to self-reported effort” theory.

  Overall… The study seems to be a good one: doing a large replication study on prior claims. The presentation of it… The author on Twitter said “testing over N= 4,000 people”, which is maybe what you get if you add up the N from all the different studies, but each study is considerably smaller; I found that somewhat misleading, but suspect that’s a common thing when authors report multiple studies at once. On Twitter he says “We conclude that effort has unequivocally small effects”, which omits caveats like “our results are accurate to the degree that alternative incentives do not yield appreciably larger effects” which are in the paper; this also seems like par for the course for science journalism (not to mention Twitter discourse). And they seem to have equivocated in places between “low effect” and “no effect”. (Which I suspect is also not rare, unfortunately.)

  Now. You presented this as:

  Here’s a classical example; an IQ researcher who is so focused on providing a counternarrative to motivational theories that he uses methods which are heavily downwards biased to “prove” that IQ test scores don’t depend on effort.

  The “focused on providing a counternarrative” part is plausibly correct. However, the “uses methods which are heavily downwards biased to “prove” [...]” is not. The “downwards biased methods” are “offering a monetary incentive of £2-£10, which turned out to be insufficient to change effort much”. The authors were doing a replication of Duckworth, in which most of the cited studies had a monetary incentive of <$10—so that part is correctly matched—and they used high enough N that Duckworth’s claimed effect size should have shown up easily. They also preregistered the first of their incentive-based studies (with the £2 incentive), and the later ones were the same but with increased sample size, then increased incentive. In other words, they did exactly what they should have done in a replication. To claim that they chose downwards-biased methods for the purpose of proving their point seems quite unfair; those methods were chosen by Duckworth.

  This seems to be a data point of the form “your priors led you to assume bad faith (without having looked deeply enough to discover this was unjustified), which then led you to take this as a case to justify those priors for future cases”. (We will see more of these later.) Clearly this could be a self-reinforcing loop that, over time, could lead one’s priors very far astray. I would hope anyone who posts here would recognize the danger of such a trap.

  Second example. “Simon Baron-Cohen playing Motte-Bailey with the “extreme male brain” theory of autism.” Let’s see… It seems uncontroversial (among the participants in this discussion) that there are dimensions on which male and female brains differ (on average), and on which autists are (on average) skewed towards the male side, and that this includes the empathizing and systematizing dimensions.

  You quote Baron-Cohen as saying “According to the ‘extreme male brain’ theory of autism, people with autism or AS should always fall in the [extreme systematizing range]”, and say that this is obviously false, since there exist autists who are not extreme systematizers—citing a later study coauthored by Baron-Cohen himself, which puts only ~10% of autists into the “Extreme Type S” category. You say he’s engaging in a motte-and-bailey.

  After some reading, this looks to me like a case of “All models are wrong, but some are useful.” The same study says “Finally, we demonstrate that D-scores (difference between EQ and SQ) account for 19 times more of the variance in autistic traits (43%) than do other demographic variables including sex. Our results provide robust evidence in support of both the E-S and EMB theories.” So, clearly he’s aware that 57% of the variance is not explained by empathizing-systematizing. I think it would be reasonable to cast him as saying “We know this theory is not exactly correct, but it makes some correct predictions.” Indeed, he counts the predictions made by these theories:

  An extension of the E-S theory is the Extreme Male Brain (EMB) theory (11). This proposes that, with regard to empathy and systemizing, autistic individuals are on average shifted toward a more “masculine” brain type (difficulties in empathy and at least average aptitude in systemizing) (11). This may explain why between two to three times more males than females are diagnosed as autistic (12, 13). The EMB makes four further predictions: (vii) that more autistic than typical people will have an Extreme Type S brain; (viii) that autistic traits are better predicted by D-score than by sex; (ix) that males on average will have a higher number of autistic traits than will females; and (x) that those working in science, technology, engineering, and math (STEM) will have a higher number of autistic traits than those working in non-STEM occupations.

  Note also that he states the definition of EMB theory as saying “autistic individuals are on average shifted toward a more “masculine” brain type”. You say “Sometimes EMB proponents say that this isn’t really what the EMB theory says. Instead, they make up some weaker predictions, that the theory merely asserts differences “on average”.” This is Baron-Cohen himself defining it that way.

  Would it be better if he used a word other than “theory”? “Model”? You somewhat facetiously propose “If the EMB theory had instead been named the “sometimes autistic people are kinda nerdy” theory, then it would be a lot more justified by the evidence”. How about, say, the theory that “There are processes that masculinize the brain in males; and some of those processes going into overdrive is a thing that causes autism”? (Which was part of the original paper: “What causes this shift remains unclear, but candidate factors include both genetic differences and prenatal testosterone.”) That is, in fact, approximately what I found when I googled for people talking about the EMB theory—and note that the article is critical of the theory:

  This hypothesis, called the ‘extreme male brain’ theory, postulates that males are at higher risk for autism as a result of in-utero exposure to steroid hormones called androgens. This exposure, the theory goes, accentuates the male-like tendency to recognize patterns in the world (systemizing behavior) and diminishes the female-like capacity to perceive social cues (socializing behavior). Put simply, boys are already part way along the spectrum, and if they are exposed to excessive androgens in the womb, these hormones can push them into the diagnostic range.

  That is the sense in which an autistic brain is, hypothetically, an “extreme male brain”. I guess “extremely masculinized brain” would be a bit more descriptive to someone who doesn’t know the context.

  The problem with a motte-and-bailey is that someone gets to go around advancing an extreme position, and then, when challenged by someone who would disprove it, he avoids the consequences by claiming he never said that, he only meant the mundane position. According to you, the bailey is “they want to talk big about how empathizing-systematizing is the explanation for autism”. According to the paper, it was 43% of the explanation for autism, and the biggest individual factor? Seems pretty good.

  Has Baron-Cohen gone around convincing people that empathizing-systematizing is the only factor involved in autism? I suspect that he doesn’t believe it, he didn’t mean to claim it, almost no one (except you) understood him as claiming it, and pretty much no one believes it. Maybe he picked a suboptimal name, which lent itself to misinterpretation. Do you have examples of Baron-Cohen making claims of that kind, which aren’t explainable as him taking the “This theory is not exactly correct, but it makes useful predictions” approach?

  The context here is explaining why you’ve “become horrified at what [you] once trusted”, which you now call “supposed science”. I’m… underwhelmed by what I’ve seen.

  Back to Damore...

  I think Damore’s point, in bringing it up, was that the stress in (some portion of) tech jobs may be a reason there are fewer women than men in tech.

  You may or may not be right that this is what he meant.

  ...I thought it was overkill to cite four quotes on that issue, but apparently not. Such priors!

  (I think it’s a completely wrong position, because the sex difference in neuroticism is much smaller (by something like 2x) than the sex difference in tech interests and tech abilities, and presumably the selection effect for neuroticism on career field is also much smaller than that of interests. So I’m not sure your reading on it is particularly more charitable, only uncharitable in a different direction; assuming a mistake rather than a conflict.)

  It seems you’re saying Damore mentions A but not B, and B is bigger, therefore Damore’s “comprehensive” writeup is not so, and this omission is possibly ill-motivated. But, erm, Damore does mention B, twice:

  • [Women, on average have more] Openness directed towards feelings and aesthetics rather than ideas. Women generally also have a stronger interest in people rather than things, relative to men (also interpreted as empathizing vs. systemizing).
    ○ These two differences in part explain why women relatively prefer jobs in social or artistic areas. More men may like coding because it requires systemizing and even within SWEs, comparatively more women work on front end, which deals with both people and aesthetics.

  [...]

  • Women on average show a higher interest in people and men in things
    ○ We can make software engineering more people-oriented with pair programming and more collaboration. Unfortunately, there may be limits to how people-oriented certain roles at Google can be and we shouldn’t deceive ourselves or students into thinking otherwise (some of our programs to get female students into coding might be doing this).

  This suggests that casting aspersions on Damore’s motives is not gated by “Maybe I should double-check what he said to see if this is unfair”.

  I think the anxiety/​stress thing is more relevant for top executive roles than for engineer roles; a population-level difference is more important at the extremes. Damore does talk about leadership specifically:

  We always ask why we don’t see women in top leadership positions, but we never ask why we
  see so many men in these jobs. These positions often require long, stressful hours that may not
  be worth it if you want a balanced and fulfilling life.

  Next:

  (Incidentally, imagine if Damore had claimed the opposite—”Women are less prone to anxiety and can handle stress more easily.” Wouldn’t that also lead to accusations that Damore was saying we can ignore women’s problems?)

  The correct thing to claim is “We should investigate what people are anxious/​stressed about”. Jumping to conclusions that people’s states are simply a reflection of their innate traits is the problem.

  Well, he lists one source of stress above, and he does recommend to “Make tech and leadership less stressful”.

  I don’t think this is at the heart of Zack’s adventure? Zack’s issues were mainly about leading rationalists jumping in to rationalize things in the name of avoiding conflicts.

  And why would these rationalists care so much about avoiding these conflicts, to the point of compromising the intellectual integrity that seems so dear to them? Fear that they’d face the kind of hostility and career-ruining accusations directed at Damore, and things downstream of fears like that, seems like a top candidate explanation.

  Anyway, making weighty claims about people is core to what differential psychology is about.

  Um. Accusations are things you make about individuals, occasionally organizations. I hope that the majority of differential psychology papers don’t consist of “Bob Jones has done XYZ bad thing”.

  It’s possible that some of my claims about Damore are false, in which case we should discuss that and fix the mistakes. However, the position that one should just keep quiet about claims about people simply because they are weighty would also seem to imply that we should keep quiet about claims about trans people and masculinity/​femininity, or race and IQ, or, to make the Damore letter more relevant, men/​women and various traits related to performance in tech.

  You are equivocating between reckless claims of misconduct /​ malice by an individual, and heavily cited claims about population-level averages that are meant to inform company policy. Are you seriously stating an ethical principle that anyone who makes the latter should expect to face the former and it’s justified?

  Somewhat possible this is true. I think nerdy communities like LessWrong should do a better job at communicating the problems with various differential psychology findings and communicating how they are often made by conservatives to promote an agenda. If they did this, perhaps Damore would not have been in this situation.

  I think Damore was aware that there are people who use population-level differences to justify discriminating against individuals, and that’s why he took pains to disavow that. As for “the problems with various differential psychology findings”—do you think that some substantial fraction, say at least 20%, of the findings he cited were false?

• localdeity 13 Apr 2023 21:54 UTC

  27 points

  15

  in reply to: dr_s’s comment on: Killing Socrates

  Looking at the strip itself, it’s pretty weird. The sea lion’s words are polite, but being in someone’s house uninvited is a violation of their property rights. How did he even get in—breaking and entering? Meanwhile, the person’s reaction is not to say “oh my god, how did he get in?” or “get out of here or I will call the police”, but to be unsurprised and annoyed.

  In other panels where they’re having tea or breakfast, it’s unclear whether it’s in their house—in which case the same stuff as above applies—or in some third-party restaurant place, in which case they would at least have the recourse of complaining to the owner and trying to persuade him that this sea lion is bothering his patrons and driving away business. Even in public, if such behavior were persistent enough, I imagine it could rise to the level of “stalking” and be punishable by law; though this is probably a grey area, and let’s assume the sea lion is doing his best to remain technically within the law.

  The strip could have had the sea lion follow them around while they’re outside in public, and maybe had them sigh in relief when they close the door to their house. It could have shown them arguing with an unsympathetic proprietor about his behavior. But no, they’re not even looking for any recourse against him. Instead, it showed the sea lion following them inside, everywhere, into their bedroom while they’re trying to sleep, and this is presented as just a fact of life. Why?

  I won’t assume the worst motives—it’s possible that the author made the situation more extreme for absurdity and humor—but whatever the intent, clearly “he follows you into your bedroom when you’re trying to sleep” would intensely strengthen the elements of annoyance and of “oh god I can’t escape him” for a real person. If all the scenes with the sea lion were in public, the emotional point wouldn’t land as heavily… for those who don’t think about the assumptions. (The fact that it’s a talking sea lion generally encourages suspension of disbelief. It’s also a key part of the comic: a man following a woman into her bedroom uninvited would be interpreted very differently.)

  The strip embodies an acute lack of distinction between “criminal trespass and stalking” and “annoying but maybe-permissible public behavior”. Blurring these together seems to be a key part of how it gives emotional weight to its point, which perhaps made it resonate with many more people than it otherwise would have.

  The strip embodies the exact mindset that leads to “complaining that random strangers are engaging you when you’re literally on the Engage Random Strangers Platform”. I don’t think there’s a way it could make sense outside the context of social media platforms like that; the specific thing of “communications from randos appearing in your bedroom at night, and this being unsurprising” perfectly fits internet tech and not much else. (Getting phone calls all night would be similar, and actually worse if it prevented you sleeping, but you could unplug the phone—which is kind of the issue in a nutshell. I suspect the type of person with this mindset has their phone set to give audible notifications on all kinds of social media messages, keeps their phone nearby, and might not even set it to do-not-disturb mode at night.) The same applies to how the sea lion magically overhears the conversation and appears from nowhere in the first place, and no one is surprised at this, nor at how he keeps magically appearing in all subsequent places.

  You could look at the strip and say “the point is that the sea lions are annoying, persistent, and their words are polite, and you’re supposed to abstract away everything else”. I think that would be wrong; I think that would mean abstracting away most of the logic (and the appeal) of the comic, and, among other things, when choosing a name for bad behavior I want far better epistemics than that.

  I submit that “the point is that the people complaining about the sea lions have an immature attitude towards social media, plus they see nothing wrong with disparaging groups of people in public and are mad when called on it, and generally they are massive hypocrites”. (I know it’s not what they intend to say, but it is the meaning I take from their speech.) That being the case, it makes me uneasy when I see someone I respect use the term from the comic unironically.

• localdeity 25 Feb 2023 22:58 UTC

  27 points

  12

  in reply to: tailcalled’s comment on: [Link] A com­mu­nity alert about Ziz

  For an example, see the “rationalist fleet” post. Among other conflicts, it describes getting into a drawn-out conflict with a roommate/​subletter (who by Ziz’s account was pretty abusive), ending with the below; it seems pretty illustrative of Ziz’s thought-process (and has nothing to do with veganism):

  We all had reports to make to CPS. We called the landlord. The nanny reported him for driving drunk to Uber. I went to the police again, showed them my bruise, they still said I couldn’t prove anything. I thought I had a deontological obligation not to let him profit by aggression meant to drive me out of my home for resources. I wondered if this was enough. I felt like maybe I was deontologically obligated to stay there, but, fuck. The door didn’t really close anymore. There was a hole in it. I heard his child was taken away, and was satisfied with that. Then I heard he got him back. I considered whether to show up at fuck o’clock in the morning and put something in his car’s gas tank to destroy it. Murphyjitsu: bring a charged cordless drill to create a hole if it was one of those gas tank caps that locked, and actually look up what things will destroy an engine. (Not done with Murphyjitsu here). But I decided to leave this as a story that I could tell.

  Up until writing this, I never gave him any further indication it was me who caused this.

  If you combine the logic of “I must retaliate hard enough that the person, had they known this would happen, wouldn’t have acted badly in the first place” (regardless of whether the person even knows about the revenge), with a propensity to escalate (I suspect destroying the car engine would inflict economic costs significantly greater than whatever rent-payments were involved), an obvious disregard for breaking laws and destroying property, and a further disregard for “morality” (described elsewhere, like in the “journey to the dark side” post) such that even committing murder is on the table… then it’s not especially surprising that they’d conclude that, say, killing Jamie’s apparently-abusive parents a decade after the abuse was “deontologically obligatory”.

• localdeity 9 Feb 2023 17:00 UTC

  26 points

  13

  on: Reli­gion is Good, Actually

  Murder is good, everyone!

  By “murder” I mean the process of deciding that you don’t like something and then acting to get rid of it. For example, if you have some old clothes that are starting to develop holes—don’t just tolerate them and be miserable, be proactive and get rid of them! Murder your worn-out clothes! For another example that I’m sure would appeal to rationalists, if you used to believe something but now you have strong evidence it’s wrong, don’t cling to your old position—murder your false beliefs!

  Of course, there are some distasteful things that people often think of when they hear “murder”. But I favor a wider definition of the term, encompassing many valuable and beneficial practices; I’m not saying every murder is good, but the general idea is good and has applications everywhere in life. With a proper understanding of what it means, we should all set out to commit murder every day!

  ---

  “Religion” means what it means—probably somewhat different things to different people. But I think most people would agree it implies “a set of beliefs matching certain criteria” (some would say those beliefs are the religion, others might say they’re merely an essential part of an organizational or cultural structure that is the religion). For rarely-used words, you might have a hope of deliberately changing the definition, but that’s not the case for “religion”. And as long as “religion” means what it does, adopting the slogan “Religion is good” means promoting mainstream religions, the Catholic Church, and so on.

  Theses you might have picked instead:

  • Religion can be good

  • Some parts of religion are good

  • Some religious-like practices are good

  • Community and rituals are good

  • ...

  What links here?

  • Richard_Kennaway's comment on Reli­gion is Good, Actually by Gordon Seidoh Worley (9 Feb 2023 17:04 UTC; 6 points)

• localdeity 15 Oct 2023 2:56 UTC

  24 points

  21

  on: Dishon­or­able Gos­sip and Go­ing Crazy

  I’ll list some benefits of gossiping about people who appear to have gone crazy. “Knowing to beware of those people” was mentioned, but here are others:

  • If you know what they had been doing prior to going crazy, which seems potentially causally related (e.g. taking certain drugs, being already in a mentally vulnerable state for other reasons, hanging out with certain crazy-ish people, and/​or obsessing about certain books or blogs), then you can update your beliefs about what’s dangerous to do. Which can inform your own behavior and possibly that of your friends.

  • If you know how that person behaves currently or in the past, you can update your model of how to estimate a given person’s current or future sanity based on their behavior.

  I’ll note it seems common for people, when they hear that someone died, to want to know how they died, especially if they died young. This seems obviously evolutionarily useful—learning about the dangers in your environment—and it seems plausibly an evolved desire. You can replace “went crazy” with “died” above (or with any significantly negative outcome), and most of it applies directly.

• localdeity 1 Apr 2022 21:43 UTC

  24 points

  in reply to: habryka’s comment on: Re­plac­ing Karma with Good Heart To­kens (Worth $1!)

  I am shocked, shocked, to find voting rings in this forum!

  Your upvotes, sir.

  Ah, thank you very much. Everybody out at once!

• localdeity 5 Oct 2022 1:26 UTC

  LW: 23 AF: 12

  18

  AF

  on: Smoke with­out fire is scary

  If we see precursors to deception (e.g. non-myopia, self-awareness, etc.) but suspiciously don’t see deception itself, that’s evidence for deception.

  Stated like this, it seems to run afoul of the law of conservation of expected evidence. If you see precursors to deception, is it then the case that both (a) seeing deception is evidence for deception and (b) not seeing deception is also evidence for deception? I don’t think so.

  The direct patch is “If you see precursors to deception, then you should expect that there is deception, and further evidence should not change your belief on this”—which does seem to be a first approximation to your position.

• localdeity 10 Apr 2022 5:05 UTC

  21 points

  on: Worse than an un­al­igned AGI

  Given the choice between “paperclip AI with humanity’s memories taking over the universe” and “the chance of some future alien civilization arising, turning out to be decent people, and colonizing the galaxy without being taken over by an AGI”, I am inclined to prefer the latter.

• localdeity 21 Oct 2021 21:05 UTC

  21 points

  in reply to: jessicata’s comment on: My ex­pe­rience at and around MIRI and CFAR (in­spired by Zoe Curzi’s writeup of ex­pe­riences at Lev­er­age)

  He specified “mission-critical”. An AI’s ability to take over other machines in the network, take over the internet, manufacture grey goo, etc. (choose your favorite doomsday scenario), is not really related to how mission-critical its original task was. (In fact, someone’s AI to choose the best photo filters to match the current mood on Instagram to maximize “likes” seems both more likely to have arbitrary network access and less likely to have careful oversight than a self-driving car AI.) Therefore I do think his comment was about the likelihood of failure in the critical task, and not about alignment.

  I think he meant something like this: The neural net, used e.g. to recognize cars on the road, makes most of its deductions based on accidental correlations and shortcuts in the training data—things like “it was sunny in all the pictures of trucks”, or “if it recognizes the exact shape and orientation of the car’s mirror, then it knows which model of car it is, and deduces the rest of the car’s shape and position from that, rather than by observing the rest of the car”. (Actually they’d be lower-level and less human-legible than this. It’s like someone parsing tables out of Wikipedia pages’ HTML, but instead of matching th/​tr/​td elements, it just counts “<” characters, and God help us if one of the elements has an extra < due to holding a link or something.) If you understood just how fragile and divorced from reality the shortcuts were, while you were sitting in such a car rushing down the highway, you would scream.

  (The counterargument to screaming, it seems to me, is that it’s relying on 100 different fragile accidental correlations, any 70 of which are sufficient—and it’s unlikely that more than 10 of them will break at once, especially if the neural net gets updated every few months, so the ensemble is robust even though the parts are not. I expect one could develop confidence in this by measuring just how overdetermined the “this is a car” deductions are, and how much they vary. But that requires careful measurement and calculation, and many people might not get past the intuitive “JFC my life depends on the equivalent of 100 of those reckless HTML-parsing shortcuts, I’m going to die”. And I expect there are plenty of applications where the ensemble really is fragile and has a >10% chance of serious failure within a few months.)

  (NB. I’ve never worked on neural nets.)

• localdeity 3 Jul 2022 4:46 UTC

  20 points

  20

  on: Sex­ual self-acceptance

  I decided, long ago, that I had the right to think whatever thoughts I wanted, including arbitrarily inappropriate sexual thoughts, as long as I kept my actions in check. (Which includes not telling people about thoughts they’d rather not hear about.) And the same goes for everyone else. I feel this is an important right, and will fite irl to defend it.

• localdeity 7 Sep 2021 20:31 UTC

  20 points

  in reply to: lsusr’s comment on: I read “White Frag­ility” so you don’t have to (but maybe you should)

  When I hear “I don’t see people in terms of race”, I translate it into “I am willfully ignorant of the ethnic dynamics (including power dynamics) around me”.

  Stated without any qualifiers, this seems to be the type of reaction that leads people to treat the autistic, naive, or otherwise socially impaired as malicious, and then possibly to punish them for something they don’t understand, which may bewilder them and further impair their social development. I hope you successfully avoid doing this.