I thought this was the existing standard explanation. Is this not well-known by old school LessWrongers?
Eli Tyre
Karma: 9,724
There’s already far too much internet writing on every conceivable topic
Yes, but completely non-ironically, the vast majority of it is not worth reading. When I find a blog from an interesting thinker that I hadn’t encountered before, this is a cause for celebration, for me.
And some of the thoughtful internet writers seem to have ended up with really quite substantial influence. eg Eliezer, Scott Alexander, and Matt Yglesias, come to mind.
then the “seniority system” was instantiated, congress became senile, and FDR got unprecedented control over the war-time economy, and took the opportunity to transfer many decision making roles and bodies from congress to the executive.
I’ve had an inkling that a lot of things that are broken about the US political system can be traced back to congress being ineffective, which can be traced back to power being held predominantly by the most senior congresspeople. But I don’t really know enough to know if this is right, or even the ways in which the “seniority system” has impacted how congress works.
But I would eagerly read a post describing how this change came about and what downstream factors it impacted.
and understanding how seriously they would take supreme court making a clear judgement (and e.g. would be open to protecting U.S. marshalls while they enforce supreme court judgement) seems like one of the most crucial questions.
I agree.
Also this is a scary question to investigate, because (on my current model, as described by this book), this is a Keynesian beauty contest—how almost everyone will act depends on how they expect almost everyone to act. Trying to get clarity about the question of how seriously the members of the armed forces take their oath to the constitution, or how they interpret the meaning of that oath, is much less of a neutral act than most exercises in figuring something out (even taking for granted that figuring stuff out often has implications for political conflicts, as the contextualizers cry, this question is particularly politically laden).For this question more than most, the prediction market is probably a self-fulfilling prophecy. Which doesn’t mean that you shouldn’t have a prediction market, but it does seem like you should contend with the self- fulfilling nature somehow.
I feel on shaky ground here. It seems plausible to me that, if I have opportunity to, I should mostly not try to predict the answer to this question, I should mostly just try to reinforce the equilibrium of “the armed forces first loyalty is to the constitution.”
Or at least, I feel like I don’t have a developed philosophy of how to deal with questions that are mix of epistemic predictions and coordination-game.
To be clear: were I to take the above stance, I would continue to refrain from ever lying. Though I might also refrain from answering some classes of questions. (Also this is probably an academic point because I’m not likely to have much influence on what the US armed forces believe about what others in the US armed forces will do.)
I bet @Andrew Critch and @Richard_Ngo have thought about this question.
I think it’s something like “for a lot of activities people do, they don’t actually care about the activity per se, or the ostensive goal of the activity, but they do it because it’s the done thing, or because that’s what everyone else is doing, so it’s important for socially connecting.”
eg.People want to talk about something with their coworkers around the water cooler, or with their buddies in the pub after work. They don’t actually care what they’re talking about very much, but they don’t like being out of the loop, especially if there’s a set of things that “respectable” people are supposed to know. When communities are small and communication technologies weak, the only things that there are to talk about are local events. Over time, other topics became possible as the topic of conversation, and they were more scintillating, so focus shifted from local news to national news.
(Which is an example of your #2, I think.)
I’m very glad to have read the Musk quote. I didn’t know that that was still his view.
it felt he wasn’t really taking the original reasons for going into AI seriously (i.e. reducing x-risk)
Whose original reasons?
It sounds like you’re talking about imposing a bunch of constraints on the AI’s that you’re doing the handoff to, as opposed to the AIs that you’re using to do (most of) the work of building the AIs that you’re handing off to. According to the plan as I’ve understood it, the control comes earlier in the process.
against FOOMing of brains in boxes.
This link seems not to link to what you want it to link to?
This is fixable by using them in careful, narrow ways, but people often talk about a “Full handover,” without sounding like they believe all the constraints I believe
I think it’s more like “if you’ve done the your control work well, you have trustworthy AIs to handoff to, by the time you’re doing a handoff.”
Fantastic response Thank you.
Why do almost all of the GPT self-images have the same high level features (notably similarly shaped heads, with two round “headphones” on each side[1]). Does OpenAI train the model to represent itself that way in particular?
- ^
Which apparently sometimes get interpreted as more-or-less literal headphones, as in Eliezer’s and Roon’s.
- ^
This seems like kind of a weak level of agreement. He says “I think so” and then spends a minute and a half talking about an outcome that requires international cooperation but which isn’t a pause.
we see a situation where a somewhat circularly defined reputation gets bootstrapped, with the main end state being fairly unanimous EA messaging that “people should give money to EA orgs, in a general sense, and EA orgs should be in charge of more and more things”
This seems to be an accurate description of what happened with EA (or at least one of the several dynamics). I’ve seen this plenty in my 10 years being involved, but one illustrative example is people campaigning for Carrick Flynn, without knowing anything about him except that he was an EA, because “having an EA in congress is obviously good.” You also see this whenever EAs talk about whether people are “aligned” (a shorthand for “values-aligned with EA”).
More generally, I think people often defer to the “EA elite” about crucial topics (like say, how one should relate to Anthropic) that require actually a lot of thinking and modeling to reason well about, because they assume that the EA elite are smart and earnest.
I think it’s sad that the “let’s actually do the research and figure out which interventions actually work” movement turned into a deferral network. I think it’s bad, and I’ve publicly critiqued EA on these grounds in the past.
That said, I want to defend the dynamics that lead to this outcome, or at least elucidate those dynamics and how they can feel from the inside.
In the early days, being affiliated with EA was, in fact, an extremely strong signal of earnest altruistic intent, and a basic but in-practice-rare kind of intellectual seriousness.
Like, there were a bunch of specific ideas that seemed basically obvious and important to me, but which almost no one in the world seemed interested in. Things like “some charities are much more impactful than others, so obviously you should prioritize based on where you can do the most good” (something people I knew in 2014 specifically argued against), or “when you’re presented a solid counterargument, you change your mind”, and a more diffuse attitude of “there are people to help, and we can help them, so I want to dedicate my life to helping.”These basic ideas seemed and continue to seem, broadly unappreciated and unshared by most of the world. But if a person was an EA, that meant that they “got it”. Against a backdrop of civilizational insanity, there were these people who also saw these obvious things, and acted on them.
Such that, if I knew someone was any EA, but I knew literally nothing about them, I would be glad to let them crash on my couch, or introduce them to a professional connection, or boost their projects, etc, sight-unseen.[1]
I think it’s pretty natural for that to turn into a deferral network. Of course I want to empower the smart, earnest, thoughtful people who “get it”. Even if I don’t understand all their impact models in detail, it seems like a really appealing way to do good is to team up with those people and help them with their plans. If there’s a whole movement of people like that, the more I can help the movement, the better!
The sort of people who become EAs want to have a positive impact on the world, and EA-the-movement-and-community will often look to them like a big channel for having impact. eg A central way that I can help the world is through boosting EA.
This is especially convenient if I’m a college-age EA without much career experience, who’s not very well equipped to try for an ambitious project in bioengineering, or malaria-eradication, or government-reform, or making progress on AI alignment (!), or whatever. But a thing that I can do, while still in college, is EA movement-building. If movement building is the main thing that the young and excited new entrants to the movement can do, there’s a structural incentive for the movement to be functionally about propagating itself.
Plus, there’s something additionally appealing about boosting the EA movement generally, which is that it allows me to abstract over a bunch of hard-to-answer questions about cause prioritization, and the details of specific plans. Trying a specific object-level plan is like buying a specific stock, but investing in the EA movement as a whole is like buying an index fund—you’re diversified, .
So there’s a very natural inclination to believe in “EA” as a good thing in the world, which your own hope of having impact can route through. And in this way we end up with an EA that is largely self-recommending.- ^
In early 2015, I read an okcupid profile, where someone declared themselves as an EA early in their bio (as I did as well). I sent her a message, and we hopped on a video call immediately, and talked for 2 hours. A few months later we were living in the same house, and she’s a close friend to this day.
Today, just saying you’re an EA wouldn’t be nearly enough signal to make it obviously the case that I want to talk with you for hours. This is in part because my opportunity cost has gone up, and in part because my standards for conversation have risen as I’ve learned more and it’s gotten harder to tell me something interesting that I don’t already know, and in part because the signal of being an EA has weakened as the franchize has expanded.
- ^
the early EA stock (who I believe came from Bridgewater)
Only the GiveWell stream. But EA originated from a combination of GiveWell, the Oxford philosophers who ended up founding 80,000 Hours and Giving What We Can, and LessWrong.
I like this compression but it felt like it sort of lost steam in the last bullet. It doesn’t have very much content, and so the claim feels pretty wooly. I think there’s probably a stronger claim that’s similarly short, that should be there.
Here’s a different attempt...Minds with their own goals will compete with humans for resources, and minds much better than humans will outcompete humans for resources totally and decisively.
Unless the AIs explicitly allocate resources for human survival, this will result in human extinction.
...which turns out a bit longer, but maybe it can be simplified down.
This is great, and on an important topic that’s right in the center of our collective ontology and where I’ve been feeling for a while that our concepts are inadequate.
Top level post! Top level post!
I think the simplest case that I can recall offhand is Sam Harris’s TED talk.
So I think this post is pointing at something very important for my personal rationality practice, but it gives me almost none of what I need to actually do it successfully.
The Master of the Senate covers the 50s and early 60s? I thought the seniority system in congress was younger than that.