Adam Zerner
Oh that stuff is good to know. Thanks for those clarifications. I actually don’t see how it’s related to randomization though, so add that to the list of things I’m confused about. My question feels like a question of what to do with the data you got, regardless of whether you utilized randomization in order to get that data.
Meta: This seems like a 101-level question, so I ask it in the Open Thread rather than using the questions feature on LW.
Suppose you are designing an experiment for a blood pressure drug. I’ve heard stuff about how it is important to declare what metrics you are interested in up front, before collecting the data, not afterwards. Ie. you’d declare up front that you only care about measuring systolic blood pressure, diastolic blood pressure, HDL cholesterol, and LDL cholesterol. And then if you happen to see an effect on heart rate, you are supposed to ignore it.
Why is this? Surely just for social reasons, right? If we do the experiment and happen to see data on heart rate, The Way of Bayes says we are forced to update our beliefs. Why would we ignore those updated beliefs?
Maybe this is getting at the thing (I don’t know what it’s called) where, if you flip a coin 100 times every day for 10 years, some of those days are going to be extreme results, but that doesn’t mean that coin is weighted. Since you’re doing it so much, you’d expect it to happen. Or something like that. I don’t think my example is quite the same thing since it’s the same coin. A better example is probably when you do a blood test on someone, if you test 1000 different metrics, a few of them are bound to give “statistically significant results” just by chance. But this just seems like a phenomena that you need to adjust for, not that you should ignore data.
That sounds awesome! I have similar feelings. This is how I think about it. I don’t feel great about this as a way of explaining it, but perhaps it’d be useful.
Think about posts as forming some sort of spectrum. On one end (let’s say the right side) you’ve got something like a book. The ideas have been refined. The author spent a ton of time researching it, coming up with great examples, revising it, doing user testing on people, having professional editors look at it, etc. Next to a book maybe you’ve got something like an academic journal article. Next to that maybe an essay, or a blog post where a lot of effort has been put into it.
Then on the other end of the spectrum (left side) you’ve got maybe notes that are scribbled on the back of a napkin. Just the raw seeds of an idea. Then maybe after that you take that napkin home with you and expand a bit about those thoughts in a personal journal, but still very informal and unrefined. Then maybe you text a friend about it. Then maybe you email another friend. Then maybe posting on eg. the LW shortform. See, there’s a spectrum.
If you buy that there is this spectrum, which I think is pretty self-evident, it begs the question of how well we (LW? Rationality community? Society?) are doing at providing a platform for people at various points along that spectrum. I think that LW does a good job in the vicinity of “well researched blog post”, but for the sorts of things at the left side of the spectrum, I don’t really feel like LW addresses it. And I think that it is a cultural problem, not a technical one. We have things like Shortform, Open Thread, and various Slack and Discord groups. It’s just that, at least IME, people don’t use it for things that are on the left side of the spectrum, and thus it feels uncomfortable if you are doing things on left side of the spectrum, even if eg. the Personal Blog Posts are explicitly intended for “left side of the spectrum” types of thoughts.
So bringing this back full circle, seeing these sorts of not-fully-formed thoughts from you (Anna) is not only something I’d like to see for the more ground/object level value of those posts, but also because I think it’d push things in the right direction culturally.
Interesting, thanks for pointing that out! I’ll look into it.
But if I was saying that about a hundred pretty-uncorrelated agendas being pursued by two hundred people, I’d start to think that maybe the odds are in our favor.
Wait a minute − excuse my naïveté, but that doesn’t seem that hard!
I assume it is though and thus ask: why is it? Is it that hard to come up with such agendas? What if we had $100B to pay people and/or set bounties with?
Probably not as much as it should.
People remain surprisingly passive when faced with the prospect of death. Fear of public ridicule or losing one’s livelihood is more likely to drive men to extremes and the breaking of their customary habits.
It does make me quite anxious. I had already been anxious about it and now I’m maybe 30% more anxious than I had been.
My overall plan is to work my 9-5, make money, retire early, and then try to do something useful in AIS. I thought about forgetting the make money and retire early part due to the shorter timelines. But for whatever reason that sort of thing would make me anxious. I’d feel better knowing that I have the nest egg, so that seems like a good enough reason to pursue it. Plus I finally have a job that I enjoy, and I’ve been thinking that there might be some wisdom in what people were talking about in this thread regarding the sort of grounding that a normal job provides.
I’ve been a little bit more liberal with spending money on fun things. I’m planning a trip to Thailand right now that I’ve always wanted to take and never got around to before. I think the shorter timelines has influenced that somewhat, but not too significantly.
I don’t think much about long-term health stuff, a la Peter Attia. I’ve always gone back and forth about being a health nut, but now I feel confident that being a health nut is not worth it. I’ve even pushed things in the direction of not caring about eating unhealthy food, but I’m finding that eating too much unhealthy stuff isn’t worth it due to the short-term impact of how it makes me feel alone, physically and cognitively.
This one is probably unique to me, but I stopped caring about the risk of death due to things like Covid and cars. Previously I cared about that risk due to my having assigned a very high expected value to life, but that expected value has since gone down. Although given how things have developed with new variants being more mild and such, I probably wouldn’t be caring much about Covid anyway right now. Similar with the possibility of nuclear war.
Updates
I went to the dentist today. I have problems with teeth grinding that seem like they’ll cause TMJ and enamel erosion down the road (and have already started to). I could get a mouth guard. But mouth guards are uncomfortable for me to wear. So I have to weigh that discomfort against the down the road problems. The short timelines probably move me from “get the mouth guard” to “don’t get the mouth guard” here.
I had been spending time learning functional programming with Haskell and Clojure. I’d like to continue that—it’s fun and will be helpful to me in the long run—but now I feel like I don’t really have time for it and need to spend my time on higher priority things. Whereas previously I guess I figured that there’s time to get to the higher priority things down the road. Maybe this push towards focusing on higher priority things isn’t a bad thing. Maybe it is.
I think I didn’t do a very good job of being clear about my point and thoughts.
The big thing I was trying to say is that user research should be thought of as the ultimate barometer, not whether the code you wrote happens to adhere to various acronyms like DRY.
Perhaps most UX-style user testing is done on new UIs, but you can also do such user testing on older UIs. For example, maybe you’ve had the same navbar for a long time, people have been using it, but you want to see if mobile users understand how to use it to navigate.
You make a great point about how one has to be careful to not only optimize for new users, and that the Third User (as Said describes) is also important. But I see this point as supplementary to the points I was trying to make, not in contrast to them. For example, suppose you have a bunch of DRY code in your codebase with weird abstractions that are confusing to the team. In that case, the user testing results are negative, and, I argue, those user testing results are what matter at the end of the day, not heuristics like whether the code adheres to principles like DRY.
Although, to caveat that point, there is probably some wisdom in giving those principles some weight as well, since our ability to introspect and answer the question of how easy it is to work with code is imperfect. Damn, that wasn’t very clear. It’s hard to explain what I mean here. Maybe this is a good analogy. Psychologists who study happiness find that long commutes make people unhappy. Maybe you have a long commute and think that it’s fine. So the user testing shows that it’s fine. But maybe you’re failing to introspect and the commute is actually causing you a good deal of frustration. In that case the user testing has mislead us. Because of this risk, it’s probably a good idea to give some weight to what things like happiness research have shown. Hopefully that makes some sense.
In the case of software, UX-style user testing is going to lead straight into a well known, known bad attractor: brittle magic.
I agree with what you’re saying here.
Oh yes, in looking at Google Maps I see that you are correct. I was just going off of memory when I said two hours. Someone told me two hours at some point.
Yeah neither of those things apply to me, but they do make sense. Although re: dating, I have a perhaps unconventional take. I wouldn’t want to be with someone who’d view not having a car as an important signal, and so not having a car would provide me with good signal about someone who’d want to go on a date with me, while also helping me avoid dating someone I wouldn’t want to be with anyway.
Edit: Hm, actually I think I was being too uncharitable here. Not having a car does seem like a strong signal that someone is poor. And it does seem reasonable to not want to date someone who is poor.
Yeah, more generally I don’t understand places without AC either when it does get uncomfortably hot for a month or two a year. When I was in Culver City outside of LA I didn’t have AC and needed to get a portable unit in the summer. Iirc my apartment was something like 75 degrees at night, which makes it difficult for me to sleep. And probably for many others too. I recall research showing that something in the mid to high 60s is optimal sleeping temperature. And during the day it can be hot, so I’d have to go to the mall or something to get access to AC, which is inconvenient when you just want to relax at home. A window unit is only a couple hundred bucks, so the price seems more than low enough to justify it.
Haha yup! Thanks! I am not missing the 120 degree weather right now, that’s for sure.
The subway is as great as they say. It might give you permanent brain damage though. Until conclusive trustworthy information about that becomes publicly available, it’s probably worth it to wear a cheap P100 mask indoors and to eat outdoors.
My assessment of the risk is different. I do have a P100 that I’d wear at the airport, but I don’t think the risk is high enough right now to wear in subways. I do think it’s worth wearing a N95 though if it’s crowded, which I do.
The housing can be cheaper depending on the area. It’s worth thinking about.
Yeah that is true. I actually talked to a real estate agent actually to discuss the areas. It does seem like there are some more affordable areas. It’s just that they probably wouldn’t be the areas I’d want to hang out in as much, and so I’d be spending a good amount of the time on the subways commuting into the more central areas. Which wouldn’t be the worst thing in the world, to be clear.
I never noticed unusually cold weather in DC. Maybe compared to the west coast, sure. It’s the summer heat and humidity that’s the problem; if you get a cheaper place with no A/C, buying a window unit is necessary, and still saves a ton of money.
Oh ok. I actually was just assuming that it’d be similar to NY, where I grew up, and it gets cold there. But it also gets hot as well, yeah, heat and humidity are also a problem! Which makes me realize how much I love the moderate weather in Portland.
Jobs. You have to send out a ludicrously large amount of applications at the beginning, but once you’re in, a smart person with the most basic quant skills can get all kinds of bizarre, interesting jobs (e.g. python, R, even a couple college-level statistics classes)
Personally I work remotely as a programmer so my location doesn’t really matter, but that is cool to hear that about DC. I’ve experienced having to send out ludicrously large amounts of applications at times as well. Not fun at all.
Cars give a competitive advantage over other people. The social signalling alone more than pays for it, and the signalling value pales in comparison to the instrumental value. Some people prefer to be the change they want to see, but abstaining from having access to a car is like burning half of your money to help the government slow down inflation.
Hm, that is not matching my intuition. Do you have any examples in mind where having a car would provide that sort of useful social signaling? I’m genuinely curious about this.
As an anecdote, I do remember this one time when I was having trouble finding a job. A recruiter was seeing if I want to pursue this opportunity in Vegas that’d require me to have a car for. I said no, because I don’t have a car. The recruiter (unprompted) said I am making a bad life decision to not have a car and it’ll be very hard for me in my career without one. Fortunately, neither of those things turned out to be true.
Good point! I in fact did not really consider Canada. Although like you said, those two cities seem like they’re on the expensive side. In retrospect I think I should have looked harder at various options in Canada though, but right now I think we like Portland enough where it wouldn’t be worth the effort :)
Cool! There’s also the Portland Effective Altruism and Rationality group here in Portland. I haven’t had the chance to attend yet though, I always seem to be too busy, but I plan on it!
And I forgot to mention it in the post, but I like how Seattle is only a two hour drive away. Which would mean for us either a ~$20 bus ride or renting a car. Similar with the coast being two hours away if you want the ocean, but that I think you’d need to rent a car for.
Yeah I agree, there are definitely people who do this automatically. Well, it’s probably more like a spectrum. Some people do it a lot by default, some a medium amount, some a little. One of the claims I’m making is that that the spectrum leans too much towards “too little”.
Haha right, those debates can definitely tend to devolve and be unproductive. It’s a shame.
Why Portland
That’s great to hear!
I really like the content on https://www.painscience.com/. Perhaps it’d be a useful resource for you.
You could also try throwing money at it on https://www.crowdmed.com/.
Best wishes!
That’s really awesome to hear, I appreciate the compliment! I don’t think I ever got over the public speaking anxiety though. I say “don’t think” because it’s been such a long time since I’ve had to do public speaking, so it’s possible that it has gone away, but I doubt it.