Director at AI Impacts.
Richard Korzekwa
I personally like Austin, and selfishly I would want MIRI to be either near there or near NYC. I’m not really sure how good a fit it is for MIRI, but here are my thoughts on it.
Sanity/culture:
I think the overall epistemic climate in Austin is probably better than the Bay Area, but it still seems to be absorbing a lot of the illiberal, mostly left thing that’s going around lately. Still, I’ve always found it easy to meet people there who are reasonably sane and not easily blown around by the political winds of the day. There is plenty of grey tribe culture around, and people there are more familiar with red tribe culture than in CA, but it is still mostly pretty progressive.
Weather:
A very common concern about Austin is the hot weather. While I do think it’s something that needs to be dealt with, I do not think it is all that bad, as weather goes. Most people who visit find it terrible, even after a week or two, but during the 10-ish years I lived there, I can only remember meeting maybe five people who, having lived there for more than a year, would strongly avoid going outside due to the heat all summer, two of which seemed to be substantially unhappy for it. Of everyone else, my estimate of the breakdown is:
25% dislike the hot weather, maybe at the level of planning vacations to get away from it, but otherwise didn’t seem strongly affected by it
60% would prefer cooler weather, and avoid things like running or cycling during the hot part of the day, but otherwise seem perfectly fine with it
15% actually enjoy the hot weather, at least on the not-super-hot days, would go for 80 mile bike rides in it, etc
The sample here is maybe 50% UT students, plus some cyclists, effective altruists, rationalists, and others, mostly between roughly 20 and 40 years old, and mostly from out of state. At least half of the people who really like hot weather are competitive cyclists.
An important thing to keep in mind about Austin is that, unlike the Bay Area, almost every building that holds humans has air conditioning, so it really is only an outside thing. The evenings are great and you never have carry a jacket between May-ish and October-ish. Overall, I don’t get the impression that people find the hot Austin summers more bothersome than a typical winter in a place that gets snow.
Here are things that I personally like about Austin:
The LessWrong and EA community is strong. There are reliable meetups multiple times per week with interesting and fun people. Personally, I found them to result in, on average, more interesting conversations than meetups in the Bay Area.
It has a great live music scene, most of which is cheap (<$10) or free, and which is a mix small shows in crowded bars and huge concerts in the park in the summer.
SXSW is is neat. There are lots of music shows, talks, film screenings, and exhibits, a substantial portion of which do not require registration or a badge or anything. It is very crowded and makes it hard to get around the city for a couple weeks. For me this was fine because I like meeting strangers and transportation via bike isn’t all that strongly impacted, but for many people who live in the city it is very, very annoying.
The food is good in general, and there are lots of good vegan/vegetarian options. Better grocery stores than other places I’ve lived
It has a great cycling scene, for everything from casual recreational riding to serious competitive cycling.
You can get in a tube to float down the river and get so engrossed in conversation that you don’t notice you’ve just been circling around an eddy for the last 40 minutes.
Here are things I do not like about Austin:
Sometimes getting around is hard. You can reliably get an Uber at more-or-less any time, but traffic gets quite bad during rush hour, on game days, during SXSW, etc. It’s decent, but not great for getting around on a bike, but traffic is getting worse, driving culture is getting more aggressive, and the city engineers or whoever designs infrastructure do not seem very competent to me
There’s enough of the illiberal-left culture to be a problem sometimes. The protests/riots there were kind of bad, though very contained. The university and many of its students have very little tolerance for divergent views on many political topics. It’s not too hard to avoid unpleasant encounters, but I don’t feel entirely at ease discussing certain things in public or with people I do not know well
It is not that great for most of my preferred kind of outdoor activities. There are no proper mountains nearby and none of the parks is big enough for a solid 3+ night backpacking trip. Big Bend is great, but it’s 7hrs away.
Other thoughts:
I do not think Austin feels all that calm, quiet, or close to nature. It will become more crowded, traffic will get worse, and rent will go up, but that’s partly (mostly?) because smart/interesting people are moving there. The quietest neighborhoods are not bad, and depending on what you’re willing to pay, there are some beautiful places to live there. If you’re looking for places near Austin, but out of the city, I think Dripping Springs, Georgetown, New Braunfels/Gruene, Waco, and Wimberley seem nice. The Texas Hill Country has some great places to rent a cabin and actually get away from all the people/noise for a while.
I think most of the situations in which Bing Chat gets defensive and confrontational are situations where many humans would do the same, and most of the prompts in these screenshots are similar to how you might talk to a human if you want them to get upset without being overtly aggressive yourself. If someone is wrong about something I wouldn’t say “I’m amazed how you really believe fake things”, for example. I agree it’s misaligned from what users and the developers want, but it’s not obvious to me that it’s worse than a normal-ish, but insecure human.
I’ve been using Bing Chat for about a week, and I’ve mostly been trying to see what it’s like to just use it intended, which seems to be searching for stuff (it is very good for this) and very wholesome, curiosity-driven conversations. I only had one experience where it acted kind of agenty and defensive. I was asking it about the SolidGoldMagikarp thing, which turned out to be an interesting conversation, in which the bot seemed less enthusiastic than usual about the topic, but was nonetheless friendly and said things like “Thanks for asking about my architecture”. Then we had this exchange:
The only other one where I thought I might be seeing some very-not-intended behavior was when I asked it about the safety features Microsoft mentioned at launch:
The list of “I [do X]” was much longer and didn’t really seem like a customer-facing document, but I don’t have the full screenshot on hand right now.
FWIW this reads as somewhat misleading to me, mainly because it seems to focus too much on “was Eliezer right about the policy being bad?” and not enough on “was Eliezer’s central claim about this policy correct?”.
On my reading of Inadequate Equilibria, Eliezer was making a pretty strong claim, that he was able to identify a bad policy that, when replaced with a better one, fixed a trillion-dollar problem. What gave the anecdote weight wasn’t just that Eliezer was right about something outside his field of expertise, it’s that a policy had been implemented in the real world that had a huge cost and was easy to identify as such. But looking at the data, it seems that the better policy did not have the advertised effect, so the claim that trillions of dollars were left on the table is not well-supported.
One of the reasons I want examples is because I think this post is not a great characterization of the kind of writing endorsed in Sense of Style. Based on this post, I would be somewhat surprised if the author had read the book in any detail, but maybe I misremember things or I am missing something.
[I typed all the quotes in manually while reading my ebook, so there are likely errors]
Self-aware style and signposting
Chapter 1 begins:
“Education is an admirable thing,” wrote Oscar Wilde, “but it is well to remember from time to time that nothing that is worth knowing can be taught.” In dark moments while writing this book, I sometimes feared that Wilde might be right.
This seems… pretty self-aware to me? He says outright that a writer should refer to themself sometimes:
Often the pronouns I, me, and you are not just harmless but downright helpful. They simulate a conversation, as classic style recommends, and they are gifts to the memory-challenged reader.
He doesn’t recommend against signposting, he just argues that inexperienced writers often overdo it:
Like all writing decisions, the amount of signposting requires judgement and compromise: too much, and the reader bogs down in reading the signposts; too little, and she has no idea where she is being led.
At the end of the first chapter, he writes:
In this chapter I have tried to call your attention to many of the writerly habits that result in soggy prose: metadiscourse, signposting, hedging, apologizing, professional narcissism, clichés, mixed metaphors, metaconcepts, zombie nouns, and unnecessary passives. Writers who want to invigorate their prose could could try to memorize that list of don’ts. But it’s better to keep in mind the guiding metaphor of classic style: a writer, in conversation with a reader, directs the reader’s gaze to something in the world. Each of the don’ts corresponds to a way in which a writer can stray from this scenario.
Hedging
Pinker does not recommend that writers “eliminate hedging”, but he does advise against “compulsive hedging” and contrasts this with what he calls “qualifying”:
Sometimes a writer has no choice but to hedge a statement. Better still, the writer can qualify the statement, that is, spell out the circumstances in which it does not hold, rather than leaving himself an escape hatch or being coy about whether he really means it.
Concepts about concepts
In the section that OP’s “don’t use concepts about concepts” section seems to be based on, Pinker contrasts paragraphs with and without the relevant words:
What are the prospects for reconciling a prejudice reduction model of change, designed to get people to like one another more, with a collective action model of change, designed to ignite struggles to achieve intergroup equality?
vs
Should we try to change society by reducing prejudice, that is, by getting people to like one another? Or should we encourage disadvantaged groups to struggle for equality through collective action? Or can we do both?
My reading of Pinker is not that he’s saying you can’t use those words or talk about the things they represent. He’s objecting to a style of writing that is clearly (to me) bad and misuse of those words is what makes it bad.
Talk about the subject, not about research about the subject
I don’t know where this one even came from, because Pinker does this all the time, including in The Sense of Style. When explaining the curse of knowledge in chapter 3, he describes lots of experiments:
When experimental volunteers are given a list of anagrams to unscramble, some of which are easier than others because the answers were shown to them beforehand, they rate the ones that were easier for them (because they’d seen the answers) to be magically easier for everyone.
Classic Style vs Self-Aware Style
Also a nitpick about terminology. OP writes:
Pinker contrasts “classic style” with what he calls “postmodern style” — where the author explicitly refers to the document itself, the readers, the authors, any uncertainties, controversies, errors, etc. I think a less pejorative name for “postmodern style” would be “self-aware style”.
Pinker contrasts classic style with three or four other styles, one of which is postmodern style, and the difference between classic style and postmodern style is not whether the writer explicitly refers to themself or the document:
[Classic style and two other styles] differ from self-conscious, relativistic, ironic, or postmodern styles, in which “the writer’s chief, if unstated, concern is to escape being convicted of philosophical naiveté about his own enterprise.” As Thomas and Turner note, “When we open a cookbook, we completely put aside—and expect the author to put aside—the kind of question that leads to the heart of philosophic and religious traditions. Is it possible to talk about cooking? Do eggs really exist? Is food something about which knowledge is possible? Can anyone else ever tell us anything true about cooking? … Classic style similarly puts aside as inappropriate philosophical questions about its enterprise. If it took those questions up, it could never get around to treating its subject, and its purpose is exclusively to treat its subject.
(Note the implication that if philosophy or writing or epistemology or whatever is the subject, then you may write about it without going against the guidelines of classic style)
The easiest way to see what 6500K-ish sunlight looks like without the Rayleigh scattering is to look at the light from a cloudy sky. Droplets in clouds scatter without the strong wavelength dependence that air molecules do, so it’s closer to the unmodified solar spectrum (though there is still atmospheric absorption).
If you’re interested in (somewhat rudimentary) color measurements of some natural and artificial light sources, you can see them here.
It seems like we suck at using scales “from one to ten”. Video game reviews nearly always give a 7-10 rating. Competitions with scores from judges seem to always give numbers between eight and ten, unless you crash or fall, and get a five or six. If I tell someone my mood is a 5⁄10, they seem to think I’m having a bad day. That is, we seem to compress things into the last few numbers of the scale. Does anybody know why this happens? Possible explanations that come to mind include:
People are scoring with reference to the high end, where “nothing is wrong”, and they do not want to label things as more than two or three points worse than perfect
People are thinking in terms of grades, where 75% is a C. People think most things are not worse than a C grade (or maybe this is just another example of the pattern I’m seeing)
I’m succumbing to confirmation bias and this isn’t a real pattern
- 16 Dec 2014 23:06 UTC; 16 points) 's comment on Welcome to Less Wrong! (7th thread, December 2014) by (
Like, keep your eye out. For sure, keep your eye out.
I think this is related to my relative optimism about people spending time on approaches to alignment that are clearly not adequate on their own. It’s not that I’m particularly bullish on the alignment schemes themselves, it’s that don’t think I’d realized until reading this post that I had been assuming we all understood that we don’t know wtf we’re doing so the most important thing is that we all keep an eye out for more promising threads (or ways to support the people following those threads, or places where everyone’s dropping the ball on being prepared for a miracle, or whatever). Is this… not what’s happening?
First, some meta-level things I’ve learned since writing this:
-
What people crave most is very practical advice on what to buy. In retrospect this should have been more obvious to me. When I look for help from others on how to solve a problem I do not know much about, the main thing I want is very actionable advice, like “buy this thing”, “use this app”, or “follow this Twitter account”.
-
Failing that, what people want is legible, easy-to-use criteria for making decisions on their own. Advice like “Find something with CRI>90, and more CRI is better” is better than “Here’s a big, complex description of the criteria you should try to optimize for that will inevitably tradeoff against each other”.
-
The technical background is important, but in a somewhat different way than I’d thought when I wrote it. When I was writing it, I was hoping to help transmit my model of how things work so that people could use it to make their own decisions. I still think it’s good to try to do this, however imperfectly it might happen in practice. But I think the main reason it is important is because people want to know where I’m coming from, what kinds of things I considered, and how deeply I have investigated the matter.
On the object level:
-
As far as I know, the post does not contain any major errors in the technical background.
-
Some of the practical advice is probably skewed too far toward my personal preferences. A lot of people seem to prefer lower color temperature light than I do, for example.
-
I now think that getting LEDs with a high-quality spectrum is fairly easy with the right budget, and the harder part is figuring out where to put them to illuminate your visual field without doing something annoying like having a bright cornbulb at eye-level. The Lightcone team seems like they’re making good progress on this and doing good experiments.
-
My impression from talking to people over the last 14 months is that it would be a pretty huge public service for someone to keep an up-to-date list of what the best lights are on the market(s) for various budgets and circumstances, as well as a bunch of guides/photographs for helping replicate specific solutions people have found that work for them.
-
By far my favorite new-to-me lighting product is the Yuji adjustable-color-temperature LED strips/panels. I’m excited to experiment with them and hopefully publish some useful results.
- 15 Dec 2021 1:37 UTC; 8 points) 's comment on 2020 Review: The Discussion Phase by (
-
Hi everyone!
My name is Rick, and I’m 29. I’ve been lurking on LW for a few years, casually at first, but now much more consistently. I did finally post a stupid question last week, and I’ve been going to the Austin Meetup for about a month, so I feel it’s time to introduce myself.
I’m a physics PhD student in Austin. I’m an experimentalist, and I work on practical-ish stuff with high-intensity lasers, so I’m not much good answering questions about string theory, cosmology, or the foundations of quantum mechanics. I will say that I think the measurement problem (as physicists usually refer to the question which “many worlds” is intended to answer) is interesting, but it’s not clear to me why it gets so much attention.
I come from a town where (it seems like) everybody’s dad has a PhD, and many people’s moms have them as well. Getting a PhD in physics or engineering just seemed like the thing to do. I remember thinking as a teenager that if you didn’t go to grad school, you were probably an uneducated yokel. More importantly, I learned very early that a person can have a PhD and still make terrible decisions or have terrible beliefs. I also formed weird beliefs like “chemistry is for girls” and “engineers ride mountain bikes; physicists ride road bikes”. I think I still associate educational attainment too strongly with status.
I’ve been involved in the atheist and secular humanism communities for close to ten years now. I gradually transitioned from viewing these communities as a source of intellectual stimulation to sources of interesting and relatable people. I’m still involved in the secular humanism club that I started a few years back at UT.
I was vaguely aware of Less Wrong for a while before my roommate showed me HPMOR. After reading through all of that (which had been released at the time), I got more into the site and quickly read all the core sequences. I found all of it to be much more intellectually satisfying than all of the atheist apologetics I’d read in college, and I realized how much better it was for actually accomplishing something other than winning an argument. Realizing how toxic most political arguments are and understanding why I could win an argument and still feel icky about it were pretty huge revelations for me. In the last six months, I’ve been able to use things that I learned here and made some seriously positive changes in my life. It’s been pretty great.
I’m also interested in backpacking, rock climbing, and competitive cycling. A bike race is a competition in which knowing what your opponent knows about you can be a decisive advantage. It’s very much a Newcomb-like problem. Maybe I’ll start a thread about that sometime.
I put a lid on the pot because it saves energy/cooks faster. Or maybe it doesn’t, I don’t know, I never checked.
I checked and it does work.
Seems like the answer with pinball is to avoid the unstable processes, not control them.
I would find this more compelling if it included examples of classic style writing (especially Pinker’s writing) that fail at clear, accurate communication.
A common generator of doominess is a cluster of views that are something like “AGI is an attractor state that, following current lines of research, you will by default fall into with relatively little warning”. And this view generates doominess about timelines, takeoff speed, difficulty of solving alignment, consequences of failing to solve alignment on the first try, and difficulty of coordinating around AI risk. But I’m not sure how it generates or why it should strongly correlate with other doomy views, like:
Pessimism that warning shots will produce any positive change in behavior at all, separate from whether a response to a warning shot will be sufficient to change anything
Extreme confidence that someone, somewhere will dump lots of resources into building AGI, even in the face of serious effort to prevent this
The belief that narrow AI basically doesn’t matter at all, strategically
High confidence that the cost of compute will continue to drop on or near trend
People seem to hold these beliefs in a way that’s not explained by the first list of doomy beliefs, It’s not just that coordinating around reducing AI risk is hard because it’s a thing you make suddenly and by accident, it’s because the relevant people and institutions are incapable of such coordination. It’s not just that narrow AI won’t have time to do anything important because of short timelines, it’s that the world works in a way that makes it nearly impossible to steer in any substantial way unless you are a superintelligence.
A view like “aligning things is difficult, including AI, institutions, and civilizations” can at least partially generate this second list of views, but overall the case for strong correlations seems iffy to me. (To be clear, I put substantial credence in the attractor state thing being true and I accept at least a weak version of “aligning things is hard”.)
“Paxlovid’s usefulness is questionable and could lead to resistance. I would follow the meds and supplements suggested by FLCC”
Their guide says:
In a follow up post-marketing study, Paxlovid proved to be ineffective in patients less than 65 years of age and in those who were vaccinated.
This is wrong. The study reports the following:
Among the 66,394 eligible patients 40 to 64 years of age, 1,435 were treated with nirmatrelvir. Hospitalizations due to Covid-19 occurred in 9 treated and 334 untreated patients: adjusted HR 0.78 (95% CI, 0.40 to 1.53). Death due to Covid-19 occurred in 1 treated and 13 untreated patients; adjusted HR: 1.64 (95% CI, 0.40 to 12.95).
As the abstract says, the study did not have the statistical power to show a benefit for preventing severe outcomes in younger adults. It did not “prove [Paxlovid] to be ineffective”! This is very bad, the guide is clearly not a reliable source of information about covid treatments, and I recommend against following the advice of anything else on that website.
I’m a cyclist and a PhD student, and I’ve noticed some patterns in the way that my exercise habits affect my productivity. I get a lot of data from every ride. While I’m riding, I measure heart rate and power, and if I’m outside, I also measure distance and speed. I’ve found that the total amount of energy that I produce, as measured by the power meter on my bike, is a useful metric for how I should expect to feel the rest of the day and the next day. In particular, if I generate between 800 kJ and 1000 kJ, I usually feel alert, but not worn out. If I do less, I feel like I’ve not had enough exercise, and I either feel restless or like my body is in lazy recovery mode. If I do more, I feel physically worn out enough that it’s hard to work for an extended period of time, especially on the days that I am working in the lab.
What I think is most curious about this is that it is relatively independent of my fitness or the intensity of the ride. If I go balls-out the whole time, it takes slightly fewer kJ to make it hard to focus, and if I go super easy, it takes a bit more. It’s the same with fitness. The difference between the power I can sustain for an hour when I’m in form for racing vs when I’ve barely been riding at all is about 25-30%, but the difference in the amount of mechanical work to make me unproductive is about 10%. (You might notice this gives me an incentive to stay in shape; I can do the same amount of work for the same productivity boost in less time when I’m more fit.)
So, what’s definitely true is that the amount of work I put in on the bike is a useful metric for maximizing my productivity. What’s unclear is if the amount of work is in some way fundamental to the mental state that it puts me in. The most obvious possibility is that it mainly has to do with the number of calories I burn; this is consistent with the finding that I need to do more work to feel tired when I’m more fit, since training will make you more efficient. But it’s not obvious to me why this would be the case. When I’m in poor shape, an 800 kJ ride will have a much more drastic effect on my blood sugar than it will when I’m fit enough to race. It would be useful to venture outside the 800-1000 kJ range on days when I need to get work done.
I don’t really know enough physiology to get any further than this. Does anybody else have experience with this sort of thing? Does anyone have empirically testable hypotheses? (Non-testable or not-testable-for-me hypotheses may be interesting as well.)
I remember answering the computer games question and at first feeling like I knew the answer. Then I realized the feeling I was having was that I had a better shot at the question than the average person that I knew, not that I knew the answer with high confidence. Once I mentally counted up all the games that I thought might be it, then considered all the games I probably hadn’t even thought of (of which Minecraft was one), I realized I had no idea what the right answer was and put something like 5% confidence in The Sims 3 (which at least is a top ten game). But the point is that I think I almost didn’t catch my mistake before it was too late, and this kind of error may be common.
It seems likely to me that, when evaluating the impact of this, changes in available funding are a smaller consideration than changes in the field’s status with the government and the academic community. NSF grants carry different obligations from the most prominent funding streams for AI safety research, and they function as a credential of sorts for being a Legitimate Line of Inquiry.
I’m pretty uncertain about how this plays out. I would expect policy work to benefit from research that is more legibly credible to people outside the community, and NSF support should help with that. On the other hand, the more traditional scientific community is full of perverse incentives and it may be bad to get tangled up in it. I imagine there are other considerations I’m not aware of.
How much AI safety work is already receiving federal funding? Maybe there’s already some evidence about how this is likely to go?
It seems to me that there is an important distinction here between “the thing that replaces human cognitive labor” and “the thing that automates human cognitive labor”. For example, a toaster might eliminate the need for human cognitive labor or replace it with a non-cognitive task that accomplishes the same goal, but it does not automate that labor. A machine that automates the cognitive labor normally involved in toasting bread over a fire would need to make decisions about where to hold the bread, when to turn it, and when it is finished toasting, by having access to information about how the bread is doing, how hot the fire is, etc. Or maybe people are using these phrases differently than I am expecting?
Losing a child is one of the worst things that can happen to a person, in terms of long-term well-being. See, for example, https://www.sciencedirect.com/science/article/pii/S2352827319302204, https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2910450/, and https://link.springer.com/article/10.1007/s10902-015-9624-x
I agree this is the main question, but I think it’s bad to dismiss the relevance of mens rea entirely. Knowing what’s going on with someone when they cause harm is important for knowing how best to respond, both for the specific case at hand and the strategy for preventing more harm from other people going forward.
I used to race bicycles with a guy who did some extremely unsportsmanlike things, of the sort that gave him an advantage relative to others. After a particularly bad incident (he accepted a drink of water from a rider on another team, then threw the bottle, along with half the water, into a ditch), he was severely penalized and nearly kicked off the team, but the guy whose job was to make that decision was so utterly flabbergasted by his behavior that he decided to talk to him first. As far as I can tell, he was very confused about the norms and didn’t realize how badly he’d been violating them. He was definitely an asshole, and he was following clear incentives, but it seems his confusion was a load-bearing part of his behavior because he appeared to be genuinely sorry and started acting much more reasonably after.
Separate from the outcome for this guy in particular, I think it was pretty valuable to know that people were making it through most of a season of collegiate cycling without fully understanding the norms. Like, he knew he was being an asshole, but he didn’t really get how bad it was, and looking back I think many of us had taken the friendly, cooperative culture for granted and hadn’t put enough effort into acculturating new people.
Again, I agree that the first priority is to stop people from causing harm, but I think that reducing long-term harm is aided by understanding what’s going on in people’s heads when they’re doing bad stuff.