Vaniver 21 Oct 2020 17:51 UTC
84 points
in reply to: NoSignalNoNoise’s comment on: A tale from Communist China
In general, I think people try to time markets much more than they have skill for. Suppose you think there’s a stock bubble; the temptation is to buy, hold as it rises, and then sell at the top of the bubble before everything comes crashing down. But enough people are trying to do that that you need some special skill in order to be in the leading edge, able to sell when there’s a buyer. It’s much less risky to just sell the stocks as soon as you think there’s a bubble, which foregoes any additional gains but means you avoid the crash entirely (by taking it on voluntarily, sort of).
A similar thing happened at the start of the pandemic, where my plan was basically “look, I don’t think I’m going to be especially good at the risk assessment, I want to just lock down now instead of staying open to get the marginal few weeks of meetings or hangouts or whatever,” and various other friends said “look, I believe in math, I think it’s just paranoia to lock down immediately instead of doing so based on a cost-benefit analysis.” None of us knew it at the time, but the official tests were faulty and delayed and other testing was being suppressed, and so the numbers being used for the cost-benefit analyses were significantly underestimating the true amount of viral transmission.
Also relevant was at the start of the pandemic, there were various border closures and regional quarantines that by design had as little warning as possible. Suppose Disease City has too many cases, and also people in Disease City would want to leave if they think they’re going to get locked in (because regardless of whether or not they have the disease, it’s better for them to be out than in), and the regional / national government wants to lock them in; then the government want to impose the lockdown and then announce it, since that minimizes the number of people who can flee.
Noticeably, if there’s a revolution in some country, it’s much better for the new government to murder all of the people who would want to leave than let them leave. Contrast “Who, after all, speaks today of the annihilation of the Armenians?” with anti-Cuban politics in the US being driven by Cuban expatriates who felt wronged by the government they fled, or various potential leaders kept by other powers as potential puppets ready to be installed when the time is right. If more educated / capitalist Chinese had been allowed to leave China, and ended up in the US, it seems likely could have been a voting bloc much like the Cuban voting bloc, and impacted US-Chinese relations accordingly.
[As it happens, most of the people that I know who speak of the annihilation of the Armenians are themselves part of the Armenian diaspora. Out of the various genocides and purges I’m familiar with, people mostly seem interested in ones to people similar to them, and curiously uninterested in ones that are of their political enemies, or even of the political enemies of people they would like to be allied with.]
In summary, there’s no fire alarm for when to leave the country, in part because the situation is adversarial, and this isn’t the sort of thing you should expect people to be well-calibrated on, since they don’t have many examples to learn from.
---
I think there’s also a factor where ‘the best time to plant a tree is 20 years ago, and the second best time is today.’ If you expect to want to be in Canada instead of the US in 2030, say, then you might expect to prefer the timeline where you move to Canada in 2020 and live there for ten years to the timeline where you live in the US for nine and a half years and then move to Canada the night before the civil war breaks out (or whatever), and then live there for six months. If the first, you’ll have been able to invest in lots of things that you expect to stick around; in the second, you might skip making useful home improvements because they aren’t mobile. [Similarly, imagining our Chinese intellectuals in the 1950s, if they knew they would want to move to the US in 1965, they might have decided to get the move over with as soon as possible, since then their children might have grown up speaking English, they might have been able to get positions at universities before there was a flood of similar refugees, they might have been able to liquidate Chinese assets at more favorable prices, and so on. Most of the benefit of delaying comes from the hope that actually you won’t have to move; this is why the illusory short-term positive signs are one of the most important parts of the OP.]
This story is less obvious when there’s a huge difference in value between the options. If you’re making net $60k a year in the US and would make net $30k a year in Canada, or whatever, each year you delay moving doubles your effective income. But if the difference is smaller, or you’re weighing different psychological benefits against each other, it’s probably better to get it over with than procrastinate.
What links here?
- Ben Pace's comment on Location Discussion Takeaways by mingyuan (2 Nov 2020 21:36 UTC; 14 points)
- MondSemmel's comment on A tale from Communist China by Wei Dai (15 Jan 2022 14:45 UTC; 5 points)

Vaniver 4 Oct 2019 16:16 UTC
LW: 84 AF: 24
AF
on: Debate on Instrumental Convergence between LeCun, Russell, Bengio, Zador, and More
There’s a dynamic that’s a normal part of cognitive specialization of labor, where the work other people are doing is “just X”; imagine trying to create a newspaper, for example. Most people will think of writing articles as “just journalism”; you pay journalists whatever salary, they do whatever work, and you get articles for your newspaper. Similarly the accounting is “just accounting,” and so on. But the journalist can’t see journalism as “just journalism”; if their model of how to write articles is “money goes in, article comes out” they won’t be able to write any articles. Instead they have lots of details about how to write articles, which includes what articles are and aren’t easy.
You could view both sides as doing something like this: the person who’s trying to make safeguards is saying “look, you can’t say ‘just add safeguards’, these things are really difficult” and the person who’s trying to make something worth safeguarding is saying “look, you can’t just ‘just build an autonomous superintelligence’, these things are really difficult.” (Especially since I think LeCun views them as too difficult to try to do, and instead is just trying to get some subcomponents.)
I think that’s part of what’s going on, but mostly in how it seems to obscure the core issue (according to me), which is related to Yoshua’s last point: “what safeguards we need when” is part of the safeguard science that we haven’t done yet. I think we’re in a situation where many people say “yes, we’ll need safeguards, but it’ll be easy to notice when we need them and implement them when we notice” and the people trying to build those safeguards respond with “we don’t think either of those things will be easy.” But notice how, in the backdrop of “everyone thinks their job is hard,” this statement provides very little ability to distinguish between worlds where this actually is a crisis and worlds where things will be fine!

Vaniver 7 Sep 2023 21:41 UTC
81 points
74
in reply to: KatWoods’s comment on: Sharing Information About Nonlinear
Ben Pace purposefully posted this without seeing our evidence first, which I believe is unethical and violates important epistemic norms.
For what it’s worth, I do not view this post as unethical or violating important epistemic norms. [I do think repeating hearsay is unseemly—I would prefer the post written by Alice and Chloe—but I see why Ben is doing it in this case.]
A factor that seems somewhat important to me, and perhaps underlies a major disagreement here, is that I think reputation, while it is about you, is not for you. It’s for the community you’re a part of, so that other people can have accurate expectations of what you’re like; both to help people who will appreciate interacting with you find you and help people who will regret interacting with you avoid you. Trying to manage your reputation is like trying to manage your bank balance: there are a small handful of ethical ways to do it and many unethical ways to do it.
And so the most concerning parts of the post (to me) are the parts where it sounds like you’re trying to suppress negative evidence, and the response from Nonlinear in the comments so far feels like it supports that narrative instead of undermining it.

Vaniver 3 Apr 2012 5:02 UTC
79 points
on: SotW: Be Specific
An example of this that will be familiar to any programmer, and was taught to me in grade school, is “give orders to a malicious idiot.” The teacher has the students write down the algorithm for a simple task, like “sharpen a pencil,” with a wooden pencil and an old crank-operated sharpener as the props.

Typically, people begin with something like “stick the pencil into the sharpener, then turn the crank,” which the teacher will do by ineffectually pushing the side of the pencil against the sharpener while turning the crank. The students revise to “stick the end of the pencil into the hole in the sharpener, then turn the crank,” which the teacher will do by sticking the eraser into sharpener. (There are, if I remember correctly, four or five different features you can require the pencil-sharpening algorithm have, like which end of the pencil to stick into what part of the sharpener, which way to turn the crank, to hold the pencil still so it doesn’t just spin with the crank or fall out if the sharpener is oriented poorly.)

(This will be familiar to programmers because going from the basic algorithm to code requires a level of detail that can’t be faked.)
What links here?
- Vaniver's comment on SotW: Be Specific by Eliezer Yudkowsky (3 Apr 2012 14:20 UTC; 1 point)
- handoflixue's comment on SotW: Be Specific by Eliezer Yudkowsky (5 Apr 2012 21:24 UTC; 0 points)

Vaniver 18 Oct 2021 17:36 UTC
78 points
in reply to: Unreal’s comment on: My experience at and around MIRI and CFAR (inspired by Zoe Curzi’s writeup of experiences at Leverage)
On the other side of it, why do people seem TOO DETERMINED to turn him into a scapegoat? Most of you don’t sound like you really know him at all.
A blogger I read sometimes talks about his experience with lung cancer (decades ago), where people would ask his wife “so, he smoked, right?” and his wife would say “nope” and then they would look unsettled. He attributed it to something like “people want to feel like all health issues are deserved, and so their being good / in control will protect them.” A world where people sometimes get lung cancer without having pressed the “give me lung cancer” button is scarier than the world where the only way to get it is by pressing the button.
I think there’s something here where people are projecting all of the potential harm onto Michael, in a way that’s sort of fair from a ‘driving their actions’ perspective (if they’re worried about the effects of talking to him, maybe they shouldn’t talk to him), but which really isn’t owning the degree to which the effects they’re worried about are caused by their instability or the them-Michael dynamic.
[A thing Anna and I discussed recently is, roughly, the tension between “telling the truth” and “not destabilizing the current regime”; I think it’s easy to see there as being a core disagreement about whether or not it’s better to see the way in which the organizations surrounding you are ___, and Michael is being thought of as some sort of pole for the “tell the truth, even if everything falls apart” principle.]
What links here?
- Benquo's comment on Dr Fauci as Machiavellian Boddhisattva by Benquo (6 Apr 2022 19:55 UTC; 6 points)

A Pattern Language For Rationality

Vaniver5 Jul 2022 19:08 UTC

75 points

14 comments15 min readLW link

2020 Review Article

Vaniver14 Jan 2022 4:58 UTC

74 points

3 comments7 min readLW link

Vaniver 14 Apr 2023 18:32 UTC
74 points
9
on: Moderation notes re: recent Said/Duncan threads
First, my read of both Said and Duncan is that they appreciate attention to the object level in conflicts like this. If what’s at stake for them is a fact of the matter, shouldn’t that fact get settled before considering other issues? So I will begin with that. What follows is my interpretation (mentioned here so I can avoid saying “according to me” each sentence).
In this comment, Said describes as bad “various proposed norms of interaction such as “don’t ask people for examples of their claims” and so on”, without specifically identifying Duncan as proposing that norm (tho I think it’s heavily implied).
Then gjm objects to that characterization as a straw man.
In this comment Said defends it, pointing out that Duncan’s standard of “critics should do some of the work of crossing the gap” is implicitly a rule against “asking people for examples of their claims [without anything else]”, given that Duncan thinks asking for examples doesn’t count as doing the work of crossing the gap. (Earlier in the conversation Duncan calls it 0% of the work.) I think the point as I have written it here is correct and uncontroversial; I think there is an important difference between the point as I wrote it and the point as Said wrote it.
In the response I would have wanted to see, Duncan would have clearly and correctly pointed to that difference. He is in favor of people asking for examples [combined with other efforts to cross the gap], does it himself, gives examples himself, and so on. The unsaid [without anything else] part is load-bearing and thus inappropriate to leave out or merely hint at. [Or, alternatively, using “ask people for examples” to refer to comments that do only that, as opposed to the conversational move which can be included or not in a comment with other moves.]
Instead we got this comment, where Duncan interprets Said’s claim narrowly, disagrees, and accuses Said of either lying or being bad at reading comprehension. (This does not count as two hypotheses in my culture.)
Said provides four examples; Duncan finds them unconvincing and calls using them as citations a blatant falsehood. Said leaves it up to the readers to adjudicate here. I do think this was a missed opportunity for Said to see the gap between what he stated and what I think he intended to state.
From my perspective, my reading of Said’s accusation is not clearly suggested in the comment gjm objected to, is obviously suggested from the comment Duncan responds to, with the second paragraph^[1] doing most of the work, and then further pointed at by later comments. If Said ate breakfasts of only cereal, and Duncan said that was unhealthy and he shouldn’t do it, it is not quite right to say Duncan ‘thinks you shouldn’t eat cereal’, as he might be in favor of cereal as part of a balanced breakfast; but also it is not quite right for Duncan to ignore Said’s point that one of the main issues under contention is whether Said can eat cereal by itself (i.e. asking for examples without putting in interpretative labor). This looks like white horses are not horses.
So what about Said’s four examples? As one might expect, all four are evidence for my interpretation, and none of the four are evidence for Duncan’s interpretation. I would not call this a blatant falsehood,^[2] and think all four of Duncan’s example-by-example responses are weak. Do we treat the examples as merely ‘evidence for the claim’, or also as ‘identification of the claim’?
So then we have to step back and consider non-object-level considerations, of which I see a few:
1. I think this situation is, on some level, pretty symmetric.
  1. I think the features of Said’s commenting style that people (not just Duncan!) find annoying are things that Said is deliberately optimizing for or the results of principled commitments he’s made, so it’s not just a simple bug that can be fixed.
  2. I think the features of Duncan’s conflict resolution methods that people find offputting are similarly things that Duncan is deliberately optimizing for or the results of principled commitments he’s made, so it’s not just a simple bug that can be fixed.
  3. I think both Said and Duncan a) contribute great stuff to the site and b) make some people like posting on LW less and it’s unclear what to do about that balance. This is one of the things that’s nice about clear rules that people are either following or not—it makes it easier for everyone to tell whether something is ‘allowed’ or ‘not allowed’, ‘fine’ or ‘not fine’, and so on, rather than making complicated judgments of whether or not you want someone around. I think the mod team does want to exercise some judgment and discernment beyond just rule-following, however.
2. How bad is it to state something that’s incorrect because it is too broad and then narrow it afterwards? Duncan has written about this in Ruling Out Everything Else, and I think Said did an adequate but not excellent job.
3. What’s the broader context of this discussion? Said has a commenting style that Duncan strongly dislikes, and Duncan seems to be in the midst of an escalating series of comments and posts pointing towards “the mods should ban Said”. My reckless speculation is that this comment looked to Duncan like the smoking gun that he could use to prove Said’s bad faith, and he tried to prosecute it accordingly. (Outside of context, I would be surprised by my reading not being raised to Duncan’s attention; in context, it seems obvious why he would not want (consciously or subconsciously) to raise that hypothesis.) My explanation is that Said’s picture of good faith is different than Duncan’s (and, as far as I can tell, both fit within the big tent of ‘rationality’).
  1. Incidentally, I should note that I view Duncan’s escalation as something of a bet, where if the mods had clearly agreed with Duncan, that probably would have been grounds for banning Said. If the mods clearly disagree with Duncan, then what does ‘losing the bet’ look like? What was staked here?
  2. The legal system sees a distinction between ‘false testimony’ (being wrong under oath) and ‘perjury’ (deliberately being wrong under oath), and it seems like a lot of this case hinges on “was Said deliberately wrong, or accidentally wrong?” and “was Duncan deliberately wrong, or accidentally wrong?”.
  3. I also don’t expect it to be uncontroversial “who started it”. Locally, my sense is Duncan started it, and yet when I inhabit Duncan’s perspective, this is all a response to Said and his style. I interpret a lot of Duncan’s complaints here thru the lens of imaginary injury that he writes about here.
4. I think also there’s something going on where Duncan is attempting to mimic Said’s style when interacting with Said, but in a way that wouldn’t pass Said’s ITT. Suppose my comment here had simply been a list of ways that Duncan behaved poorly in this exchange; then I think Duncan could take the approach of “well, but Said does the same thing in places A, B, and C!”. I think he overestimates how convincing I would find that, and Duncan did a number of things in this exchange that my model of Said would not do and has not done (according to my interpretation, but not my model of Duncan’s, in a mirror of the four examples above).
5. I think Said is trying to figure out which atomic actions are permissible or impermissible (in part because it is easier to do local validity checking on atomic actions), and Duncan is trying to suggest what is permissible or impermissible is more relational and deals with people’s attitudes towards each other (as suggested by gjm here). I feel sympathetic to both views here; I think Duncan often overestimates how familiar readers will be with his works / how much context he can assume, and yet also I think Said is undercounting how much people’s memory of past interactions colors their experience of comments. [Again, I think these are not simple bugs but deliberate choices—I think Duncan wants to build up a context in which people can hold each other accountable and build further work together, and I think Said views colorblindness of this sort as superior to being biased.]
1. ^
  Duncan has, I think, made it very clear that that a comment that just says “what are some examples of this claim?” is, in his view, unacceptable. That’s what I was talking about. I really do not think it’s controversial at all to ascribe this opinion to Duncan.
2. ^
  I note that my reasons for this are themselves perhaps white horses are not horses reasons, where I think Said’s original statement and follow-up are both imprecise, but they’re missing the additional features that would make them ‘blatant falsehood’s, while both imprecise statements and blatant falsehoods are ‘incorrect’.
What links here?
- Vaniver's comment on Moderation notes re: recent Said/Duncan threads by Raemon (15 Apr 2023 22:40 UTC; 4 points)