Taran

There are (prob­a­bly) no su­per­hu­man Go AIs: strong hu­man play­ers beat the strongest AIs

• Taran 10 Mar 2019 11:30 UTC

  57 points

  on: In My Culture

  I use this technique sometimes (my lead-in phrase is the deliberately silly “Among my people...”), but it has a couple of flaws that force me to be careful with it.

  Most importantly, this framing is always about drawing contrasts: you’re describing ways that your culture _differs_ from that of the person you’re talking to. Keep this point in the forefront of your mind every time you use this method: you are describing _their_ culture, not just yours. When you say, “In my culture, we put peanut butter on bread”, then you are also saying “in your culture, you do not put peanut butter on bread”. At the very most you are asking a question: “does your culture also put peanut butter on bread?” So, do not ever say something like “In my culture we do not punish the innocent” unless you also intend to say “Your culture punishes the innocent”—that is, unless you intend to start a fight.

  Relatedly, you have to explicitly do the work of separating real cultural practices from aspirational ones—this framing will not help you. When you write “In my culture we do not punish the innocent”, probably you are thinking something like “In my culture, we think it’s important not to punish the innocent”, since mistakes do still happen from time to time. But statements like “In my culture we put peanut butter on bread” do not require this kind of aggressive interpretation, they can just be taken literally, so your listeners might reasonably take “In my culture we do not punish the innocent” as a (false) statement of literal fact. Clear and open communication is unlikely to follow.

  (If you feel like you grasp these points and agree with them, here’s an exercise: can the section of the OP that starts “In my culture, we distinguish between what a situation looks like and what it actually is.” be productively rewritten, and if so how?)

  Overall, although I do like this technique and use it from time to time, I don’t think it’s well-suited to important topics. For similar reasons it’s easy to use in bad faith. That’s why I present it in such a silly and sociological (instead of formally diplomatic) way.

• Taran 7 Nov 2021 8:26 UTC

  42 points

  in reply to: [DEACTIVATED] Duncan Sabien’s comment on: Speak­ing of Stag Hunts

  But it sure is damning that they feel that way, and that I can’t exactly tell them that they’re wrong.

  You could have, though. You could have shown them the many highly-upvoted personal accounts from former Leverage staff and other Leverage-adjacent people. You could have pointed out that there aren’t any positive personal Leverage accounts, any at all, that were downvoted on net. 0 and 1 are not probabilities, but the evidence here is extremely one-sided: the LW zeitgeist approves of positive personal accounts about Leverage. It won’t ostracize you for posting them.

  But my guess is that this fear isn’t about Less Wrong the forum at all, it’s about their and your real-world social scene. If that’s true then it makes a lot more sense for them to be worried (or so I infer, I don’t live in California). But it makes a lot less to bring to bring it up here, in a discussion about changing LW culture: getting rid of the posts and posters you disapprove of won’t make them go away in real life. Talking about it here, as though it were an argument in any direction at all about LW standards, is just a non sequitur.

  What links here?

  • Kerry Vaughan's comment on Tran­script for Ge­off An­ders and Anna Sala­mon’s Oct. 23 conversation by Rob Bensinger (10 Nov 2021 0:12 UTC; 11 points)

• Taran 11 Jan 2023 8:48 UTC

  33 points

  15

  in reply to: clone of saturn’s comment on: Es­cape Ve­loc­ity from Bul­lshit Jobs

  If you fire your sales staff your company will chug along just fine, but won’t take in new clients and will eventually decline through attrition of existing accounts.

  If you fire your product developers your company will chug along just fine, but you won’t be able to react to customer requests or competitors.

  If you fire your legal department your company will chug along just fine, but you’ll do illegal things and lose money in lawsuits.

  If your fire your researchers your company will chug along just fine, but you won’t be able to exploit any more research products.

  If you fire the people who do safety compliance enforcement your company will chug along just fine, but you’ll lose more money to workplace injuries and deaths (this one doesn’t apply to Twitter but is common in warehouses).

  If you outsource a part of your business instead of insourcing (like running a website on the cloud instead of owning your own data centers, or doing customer service through a call center instead of your own reps) then the company will chug along just fine, and maybe not be disadvantaged in any way, but that doesn’t mean the jobs you replaced were bullshit.

  In general there are lots of roles at every company that are +EV, but aren’t on the public-facing critical path. This is especially true for ad-based companies like Twitter and Facebook, because most of the customer-facing features aren’t publicly visible (remember: if you are not paying, you’re not the customer).

How four guys helped redi­rect Ja­pan’s coro­n­avirus policy

• Taran 6 Dec 2021 8:26 UTC

  19 points

  on: In­ter­pret­ing Yud­kowsky on Deep vs Shal­low Knowledge

  I think you’ve identified a real through-line in Yudkowsky’s work, one I hadn’t noticed before. Thank you for that.

  Even so, when you’re trying to think about this sort of thing I think it’s important to remember that this:

  In our world, Einstein didn’t even use the perihelion precession of Mercury, except for verification of his answer produced by other means. Einstein sat down in his armchair, and thought about how he would have designed the universe, to look the way he thought a universe should look—for example, that you shouldn’t ought to be able to distinguish yourself accelerating in one direction, from the rest of the universe accelerating in the other direction.

  ...is not true. In the comments to Einstein’s Speed, Scott Aaronson explains the real story: Einstein spent over a year going down a blind alley, and was drawn back by—among other things—his inability to make his calculations fit the observation of Mercury’s perihelion motion. Einstein was able to reason his way from a large hypothesis space to a small one, but not to actually get the right answer.

  (and of course, in physics you get a lot of experimental data for free. If you’re working on a theory of gravity and it predicts that things should fall away from each other, you can tell right away that you’ve gone wrong without having to do any new experiments. In AI safety we are not so blessed.)

  There’s more I could write about the connection between this mistake and the recent dialogues, but I guess others will get to it and anyway it’s depressing. I think Yudkowsky doesn’t need to explain himself more, he needs a vacation.

• Taran 23 Jun 2021 11:30 UTC

  17 points

  on: The Point of Trade

  Maybe I was projecting a smarter argument onto my economics textbooks than was actually there, but I always understood them a little differently: two people gain by trading apples for oranges when they value (not like) their trade goods in different ways. How much you value something depends on how much you like it, but also on other things too: how easily you could find or make your own without trading, how much you value something else that the thing makes possible, how much of the thing you already have, how uncertain your expected future preferences are, and so on.

  I think this distinction dissolves the rest of the dialog. If you’re close to the apple orchards and I’m not, or you’re an expert apple farmer and I’m not, or you’ve invested capital in apple-growing machines and I haven’t, then you’ll have an easier time getting apples than I will and you won’t value them as much.

  In this account, trade creates wealth indirectly, by incentivizing production and specialization. In a world where there weren’t any distinctions of value—where people genuinely didn’t care whether they had apples, or basketballs, or copies of Twilight—gains by trade wouldn’t be possible. I can’t picture a world like that, though; even replicators don’t seem like quite enough.

• Taran 25 Aug 2021 15:39 UTC

  LW: 16 AF: 7

  AF

  on: The Codex Skep­tic FAQ

  What’s your take on the licensing issue? I know for sure that Codex won’t affect my day job in the near term, because I’m not allowed to use it; I guess most large companies, and open-source projects large enough to care about IP assignment, will have the same problem.

  This indirectly influences the speed of model improvement we should expect: the market for this kind of tool is smaller than you might think from the size of github’s userbase, so there’ll be less incentive to invest in it.

• Taran 5 Oct 2021 19:25 UTC

  15 points

  on: Do­minic Cum­mings : Regime Change #2: A plea to Sili­con Valley

  One nice thing about startups is that they mostly fail if they aren’t good. When MySpace stagnated there wasn’t one blessed successor, there were 100 different ones that had to fight it out. The winner is, modulo the usual capitalist alignment failure, a better company than MySpace was. Most of its competitors weren’t. From society’s perspective this filter is great, maybe the best thing about the whole startup ecosystem.

  Cummings doesn’t seem to know this. Replacing the Pentagon with a new organization ABC Inc. is not the hard part (although it is pretty hard). What’s hard is to know that you should pick ABC Inc. and not DEF GmbH. Cummings thinks what makes startups good is their youth (he wants to sunset them after 15 years, for example), but that’s wrong: most young startups aren’t good, and fail. To make it work you need 100 successor Pentagons, and some way of making them compete.

• Taran 8 Nov 2021 22:25 UTC

  14 points

  in reply to: dxu’s comment on: Speak­ing of Stag Hunts

  I want to reinforce the norm of pointing out fucky dynamics when they occur...

  Calling this subthread part of a fucky dynamic is begging the question a bit, I think.

  If I post something that’s wrong, I’ll get a lot of replies pushing back. It’ll be hard for me to write persuasive responses, since I’ll have to work around the holes in my post and won’t be able to engage the strongest counterarguments directly. I’ll face the exact quadrilemma you quoted, and if I don’t admit my mistake, it’ll be unpleasant for me! But, there’s nothing fucky happening: that’s just how it goes when you’re wrong in a place where lots of bored people can see.

  When the replies are arrant, bad faith nonsense, it becomes fucky. But the structure is the same either way: if you were reading a thread you knew nothing about on an object level, you wouldn’t be able to tell whether you were looking at a good dynamic or a bad one.

  So, calling this “fucky” is calling JenniferRM’s post “bullshit”. Maybe that’s your model of JenniferRM’s post, in which case I guess I just wasted your time, sorry about that. If not, I hope this was a helpful refinement.

• Taran 9 Feb 2021 13:37 UTC

  14 points

  on: Pro­mot­ing Pre­dic­tion Mar­kets With Mean­ingless In­ter­net-Point Badges

  People are tired of shitty media. There’s an enormous groundswell of media distrust from many angles, as far as I can tell. A measure like this is easy to understand, at least in the basics, and provides clear evidence of credibility for those who use it, entirely independent of trust.

  If this were true, we would expect to see declining media consumption—reduced viewership at Fox and CNN, for example. Instead the opposite is true, both reported record viewership this year. I take that to mean that the problem with journalism, insofar as there is one, is on the demand side rather than the supply side.

  So, in general I think this claim is false. I would focus on finding a small subgroup for which it’s true, and dedicate your efforts to them.

• Taran 14 Feb 2023 18:40 UTC

  13 points

  2

  on: Im­por­tant fact about how peo­ple eval­u­ate sets of arguments

  Any time you get a data point about X, you get to update both on X and on the process that generated the data point. If you get several data points in a row, then as your view of the data-generating process changes you have re-evaluate all of the data it gave it you earlier. Examples:

  • If somebody gives me a strong-sounding argument for X and several weak-sounding arguments for X, I’m usually less persuaded than if I just heard a strong-sounding argument for X. The weak-sounding arguments are evidence that the person I’m talking to can’t evaluate arguments well, so it’s relatively more likely that the strong-sounding argument has a flaw that I just haven’t spotted.

  • If somebody gives me a strong-sounding argument for X and several reasonable-but-not-as-strong arguments against X, I’m more persuaded than just by the strong argument for X. This is because the arguments against X are evidence that the data-generating process isn’t filtered (there’s an old Zack_M_Davis post about this but I can’t find it). But this only works to the extent that the arguments against X seem like real arguments and not strawmen: weak-enough arguments against X make me less persuaded again, because they’re evidence of a deceptive data-generating process.

  • If I know someone wants to persuade me of X, I mostly update less on their arguments than I would if they were indifferent, because I expect them to filter and misrepresent the data (but this one is tricky: sometimes the strong arguments are hard to find, and only the enthusiasts will bother).

  • If I hear many arguments for X that seem very similar I don’t update very much after the first one, since I suspect that all the arguments are secretly correlated.

  • On social media the strongest evidence is often false, because false claims can be better optimized for virality. If I hear lots of different data points of similar strength, I’ll update more strongly on each individual data point.

  None of this is cheap to compute; there are a bunch of subtle, clashing considerations. So if we don’t have a lot of time, should we use the sum, or the average, or what? Equivalently: what prior should we have over data-generating processes? Here’s how I think about it:

  Sum: Use this when you think your data points are independent, and not filtered in any particular way—or if you think you can precisely account for conditional dependence, selection, and so on. Ideal, but sometimes impractical and too expensive to use all the time.

  Max: Useful when your main concern is noise. Probably what I use the most in my ordinary life. The idea is that most of the data I get doesn’t pertain to X at all, and the data that is about X is both subject to large random distortions and probably secretly correlated in a way that I can’t quantify very well. Nevertheless, if X is true you should expect to see signs of it, here and there, and tracking the max leaves you open to that evidence without having to worry about double-updating. As a bonus, it’s very memory efficient: you only have to remember the strongest data favoring X and the strongest data disfavoring it, and can forget all the rest.

  Average: What I use when I’m evaluating an attempt at persuasion from someone I don’t know well. Averaging is a lousy way to evaluate arguments but a pretty-good-for-how-cheap-it-is way to evaluate argument-generating processes. Data points that aren’t arguments probably shouldn’t ever be averaged.

  Min: I don’t think this one has any legitimate use at all. Lots of data points are only very weakly about X, even when X is true.

  All of these heuristics have cases where they abjectly fail, and none of them work well when your adversary is smarter than you are.

• Taran 14 Dec 2022 22:44 UTC

  12 points

  on: In Defense of At­tempt­ing Hard Things, and my story of the Lev­er­age ecosystem

  I liked this post, but I don’t think it belongs in the review. It’s very long, it needs Zoe’s also-very-long post for context, and almost everything you’ll learn is about Leverage specifically, with few generalizable insights. There are some exceptions (“What to do when society is wrong about something?” would work as a standalone post, for example), but they’re mostly just interesting questions without any work toward a solution. I think the relatively weak engagement that it got, relative to its length and quality, reflects that: Less Wrong wasn’t up for another long discussion about Leverage, and there wasn’t anything else to talk about.

  Those things aren’t flaws relative to Cathleen’s goals, I don’t think, but they make this post a poor fit for the review: a didn’t make a lot of intellectual progress, and the narrow subfield it did contribute to isn’t relevant to most people.

• Taran 20 Mar 2022 17:56 UTC

  12 points

  on: How to pre­vent com­mu­nist re­volts?

  I’m not going to put this as an answer because you said you didn’t want to hear it, but I don’t think you’re in any danger. The problem is not very serious now, and has been more serious in the past without coming to anything.

  To get a sense of where I’m coming from I’d encourage you to read up on the history of communist movements in the United States, especially in the 1920s (sometimes called the First Red Scare, and IMO the closest the US has ever come to communist overthrow). The history of anarchism in the US is closely related, at least in that period (no one had invented anarcho-capitalism yet I don’t think, certainly it wasn’t widespread), so study that too. To brutally summarize an interesting period, USG dealt with a real threat of communist revolt through a mixture of infiltration/​police action (disrupting the leadership of communist movements and unions generally) and worker’s-rights concessions (giving the rank and file some of what they wanted, and so sapping their will to smash the state).

  For contrast, study the October revolution. Technically speaking, how was it carried off? How many people were required, and what did they have to do? How were they recruited?

  Also I’d encourage you to interrogate that “1% to 5%” figure pretty closely, since it seems like a lot of the problem is downstream of it for you. How did you come to believe that, and what exactly does it mean? Do you expect 1% of Americans to fight for communist revolt, as Mao’s guerillas did? If not, what proportion do you expect to fight? How does that compare to the successful revolutions you’ve read about?

  It might also be useful to role-play the problem from the perspective of a communist leader, taking into account the problems that other such leaders have historically faced. Are you going to replace all US government institutions, or make your changes under the color of existing law? Each institution will have to be subverted or replaced—the army especially, but also the Constitution, Supreme Court, existing federal bureaucracies, and so on. Think through how you might solve each of those problems, being as specific as you can.

  Again, I know you said you didn’t want this, but sometimes when you look through your telescope and see a meteor coming toward the earth, it’s going to miss.

• Taran 7 Nov 2021 9:45 UTC

  12 points

  in reply to: [DEACTIVATED] Duncan Sabien’s comment on: Speak­ing of Stag Hunts

  When it comes to the real-life consequences I think we’re on the same page: I think it’s plausible that they’d face consequences for speaking up and I don’t think they’re crazy to weigh it in their decision-making (I do note, for example, that none of the people who put their names on their positive Leverage accounts seem to live in California, except for the ones who still work there). I am not that attached to any of these beliefs since all my data is second- and third-hand, but within those limitations I agree.

  But again, the things they’re worried about are not happening on Less Wrong. Bringing up their plight here, in the context of curating Less Wrong, is not Lawful: it cannot help anybody think about Less Wrong, only hurt and distract. If they need help, we can’t help them by changing Less Wrong; we have to change the people who are giving out party invites and job interviews.

• Taran 24 Jun 2021 16:04 UTC

  12 points

  on: Covid 6/​24: The Span­ish Prisoner

  Ugh, the McAfee stuff is way less important than the covid stuff, but I don’t have any interesting covid commentary, just incohate frustration. So, in re McAfee: right now I do think he killed himself, following three lines of thought:

  1. I don’t understand who’s supposed to have benefitted from his death. “Preventing him from talking” doesn’t really make sense as a motive; there’s nothing publicly known that would identify “now” as a good time to kill him for that, as opposed to six months ago or whenever.

  2. I didn’t read his twitter feed until today, so I’m reading it in light of his death, but to me it reads like a man in decline and losing hope. I grant this is pretty subjective, but go read his tweets from last October and compare them to the ones from April, and decide for yourself how they grab you.

  3. People aren’t good at modeling changes to themselves over time, and especially aren’t good at tracking their own changes of mood (as when a patient’s family notices the effect of an antidepressant before the patient does). In general, I’d take a person saying “I’m definitely not going to kill myself, even after months of isolation in prison” as only very weak evidence.

  If he was killed after all, I would guess it was by local forces (drug gang, government, etc) in retaliation for something fairly small-scale that we haven’t heard about. This explains the timing: once he was extradited, he’d be beyond their reach.

• Taran 22 Dec 2019 10:02 UTC

  12 points

  in reply to: Matt Goldenberg’s comment on: De­cou­pling vs Con­tex­tu­al­is­ing Norms

  I’d love to see the post cleaned up to make it clear that you’re talking about “contextualizing as understanding how your words will have an effect in the context that you’re in” and decoupling as “decoupling what you say from the effects it may create.”

  I don’t think there’s a general consensus that this post does, or should, mean that. For example, Raemon’s review suggests “jumbled” as an antonym to “decoupled”, and gives a description that’s more general than yours. For another example, you described your review of Affordance Widths as a decoupled alternative to the contextualizing reviews that others had already written, but the highest-voted contextualizing review is explicitly about the truth value of ialdabaoth’s post—it incorporates information about the author, but only to make the claim that the post contains an epistemic trap, one which we could in principle have noticed right away but which in practice wasn’t obvious without the additional context of ialdabaoth’s bad behavior. This is clearly contextualizing in some sense, but doesn’t match the definition you’ve given here.

  I think this post is fundamentally unfinished. It drew a distinction that felt immediate and important to many commenters here, but a year and a half later we still don’t have a clear understanding of what that distinction is. I think that vagueness is part of what has made this post popular: everyone is free to fill in the underspecified parts with whatever interpretation makes the most sense to them.

• Taran 3 Feb 2023 11:16 UTC

  11 points

  0

  in reply to: [DEACTIVATED] Duncan Sabien’s comment on: Ba­sics of Ra­tion­al­ist Discourse

  Additionally, though this is small/​circumstantial, I’m pretty sure your comment came up much faster than even a five-minute timer’s worth of thought would have allowed, meaning that you spent less time trying to see the thing than it would have taken me to write out a comment that would have a good chance of making it clear to a five-year-old.

  Another possibility is that he did some of his thinking before he read the post he was replying to, right? On my priors that’s even likely; I think that when people post disagreement on LW it’s mostly after thinking about the thing they’re disagreeing with, and your immediate reply didn’t really add any new information for him to update on. Your inference isn’t valid.

• Taran 26 Feb 2021 13:34 UTC

  11 points

  in reply to: Raemon’s comment on: Men­tor­ship, Man­age­ment, and Mys­te­ri­ous Old Wizards

  FWIW I also think your summary of the 2015 article is inaccurate. For example, “EA needs very specific talents that are missing.” isn’t consistent with the section titled “Less Earning to Give”, which states very clearly that more than 20% of EAs, total, should be doing direct work. “EA needs lots of generally talented people” is a much better fit. My own experiences are consistent with that: the people I know who got career advice from 80k or other EA thought leaders in that era were all told to do direct work, typically operations at EA orgs.

  Normally this wouldn’t be worth talking about; who really cares whether an article from 2015 was unclear, or clearly communicated something its authors now disagree with? Here I think the distinction matters, because it’s a load-bearing part of the argument that mentorship is a bottleneck for EA specifically. People who got top-tier mentorship in 2015 were told things we now agree aren’t true, but that were consistent with the articles available at the time. People who got top-tier mentorship in 2020 got different advice (I assume, I haven’t kept up since covid started), but how much better was it, in terms of knowledge, than the articles available?

  I could definitely buy that EA has a shortage of mysterious old wizards, though.

• Taran 12 Mar 2019 8:10 UTC

  11 points

  in reply to: ZeitPolizei’s comment on: In My Culture

  Does this also apply to your own personal culture (whether aspiring or as-is), or “just” the broader context culture?

  We’re talking about a tool for communicating with many different people with many different cultures, and with people whose cultures you don’t necessarily know very much about. So the bit you quoted isn’t just making claims about my culture, or even one of the (many) broader context cultures, it’s making claims about the correct prior over all such cultures.

  But what claims exactly? I intended these two:

  1. When you say, “In my culture X”, you’re also saying “In your culture plausibly not X”.

  2. For some values of X, this will start fights (or hurt feelings, or sow mistrust, or have other effects you likely don’t want).

  It seems like you came to agree with point #1, so I won’t belabor it further—let me know if I misread you and we can circle back. For point #2, I definitely agree that, the more charity the listener extends to you, the smaller the set of hurtful Xs is. But if you rely on that, you’re limiting the scope of this method to people who’ll apply that charity and whom you know will apply that charity. I picked “punishing the innocent” for my example value of X because I expect it to be broadly cross-cultural: if you go find 100 random people and ask them whether they punish the innocent, I expect that most of them will take offense. If you also expect that, you should build that expectation into your communication strategy, regardless of what your own culture would have you do in those kinds of situations.

  Now, the better your know the person you’re talking to, the less important these warnings are. Then again, the better you know the person you’re talking to, the less you need the safety of the diplomatic/​sociological frame, you can just discuss your values directly. That’s why I feel comfortable using all that highly absolutist “always/​never” language above; it’s the same impulse that says “it’s always better to bet that a die will roll odd than that it’ll roll a 1, all else held equal”.

  Thinking about it more, I suspect the real rule is that this method shouldn’t be used to talk about cultural values at all, just cultural practices—things that have little or no moral valence. That phrasing doesn’t quite capture the distinction I want—the Thai businessman who won’t shake hands with you doesn’t think his choice is arbitrary, after all—but it’s close. Another rule might be “don’t use these statements to pass moral judgement”, but that’s hard to apply; as you saw it can be difficult to notice that you’re doing it until after the fact.