This month (and a half), I dropped out of community college, raised money as investment in what I’ll do in the future, moved to Berkeley, got very involved in the rationalist community here, smashed a bunch of impostor syndrome, wrote a bunch of code, got into several extremely promising and potentially impactful projects, read several MIRI papers and kept being urged to involve myself with their research further.
I took several levels of agency.
Replied above. There’s a strong chilling effect on bringing up that you don’t want children at events.
For context, calling her out specifically is extremely rare, people try to be very diplomatic, and there is definitely a major communcation failure Elo is trying to address.
It was not an exaggeration.
So, to my understanding, doing this in 2015 instead of 2018 is more or less exactly the sort of thing that gets talked about when people refer to a large-scale necessity to “get there first”. This is what it looks like to push for the sort of first-mover advantage everyone knows MIRI needs to succeed.
It seems like a few people I’ve talked to missed that connection, but they support the requirement for having a first-mover advantage. They support a MIRI-influenced value alignment research community, but then they perceive you asking for more money than you need! Making an effort to remind people more explicitly why MIRI needs to grow quickly may be valuable. Link the effect of ‘fundraiser’ to the cause of ‘value learning first-mover’.
Unrelated to this particular post, I’ve seen a couple people mention that all your ideas as of late are somewhat scattered and unorganized, and in need of some unification. You’ve put out a lot of content here, but I think people would definitely appreciate some synthesis work, as well as directly addressing established ideas about these subproblems as a way of grounding your ideas a bit more. “Sixteen main ideas” is probably in need of synthesis or merger.
This post puts me maybe 50% the way to thinking this is a good idea from my previous position.
My largest qualm about this is well-represented by a pattern you seem to show, which starts with saying “Taking care of yourself always comes first, respect yourself”, then getting people to actually act on that in simple, low-risk low-involvement contexts, and assuming that means they’ll actually be able to do it when it matters. People can show all the signs of accepting a constructed social norm when that norm is introduced, without that meaningfully implying that they’ll use it when push comes to shove. Think about how people act when actual conflicts with large fight/flight/freeze responses interact with self-care norms. I suspect some typical-mind, as my model of you is better at that than most people. I think it depends on what “running on spite” cashes out to. This is kind of a known skull, but I think the proposed solution of check-ins is probably insufficient.
My other big concern is what comments like your reply to Peter here imply about your models and implicit relationship to the project. In this comment, you say you’ll revise something, but I pretty strongly anticipate you still wanting people to do the thing the original wording implied. This seems to defuse criticism in dangerous ways, by giving other people the impression that you’re updating not just the charter, but your aesthetics. Frankly, you don’t seem at all likely to revise your aesthetics. And those, ultimately, determine the true rules.
To summarize the nature of my issues here in a few words: aesthetic intuitions have huge amounts of inertia and can’t be treated like normal policy positions, and people’s self-care abilities (and stress-noticing abiities) cannot be trusted in high-stress environments, even under light to moderate testing.
-Olivia
Determining the language to use is a classic case of premature optimization. No matter what the case, it will have to be provably free of ambiguities, which leaves us programming languages. In addition, in terms of the math of FAI, we’re still at the “is this Turing complete” sort of stage in development. So it doesn’t really matter yet. I guess one consideration is that the algorithm design is going to take way more time and effort than the programming, and the program has essentially no room for bugs (Corrigibility is an effort to make it easier to test an AI without it resisting). So in that sense, it could be argued that the lower level the language, the better.
Directly programming human values into an AI has always been the worst option, partially for your reason. In addition, the religious concept you gave can be trivially broken by two different beings having different or conflicting utility functions, and so acting as if they were the same is a bad outcome. A better option is to construct a scheme so that the smarter the AI gets, the better it approximates human values, by using its own intelligence to determine them, as in coherent extrapolated volition.
This was great. I appreciate that it exists, and I want more stories like it to exist.
As a model for what I’d actually want myself, the world felt kind of unsatisfying, though the bar I’m holding it to is exceptionally high—total coverage of my utility-satisfaction-fun-variety function. I think I care about doing things in base reality without help or subconscious knowledge of safety. Also, I see a clinging to human mindspace even when unnecessary. Mainly an adherence to certain basic metaphors of living in a physical reality. Things like space and direction and talking and sound and light and places. It seems kind of quaintly skeuomorphic. I realize that it’s hard to write outside those metaphors though.
Creating arbitrary animals that are barely alive, don’t need food, water, air, or movement, and made of easily workable material which is also good as armor seems like a good place to start, and also within the bounds of magic. This isn’t as absurd as it seems. Essentially living armor plates. You’d want them to be thin so you could have multiple layers, and to fall off when they die, and various similar things. Or maybe on a different scale, like scale or lamellar armor.
I logged in just to downvote this.
Several months late, but that Mission Impossible movie had real world effects, because Joe Biden watched it. https://www.the-independent.com/news/world/americas/us-politics/joe-biden-ai-mission-impossible-b2440365.html
Something very similar happened to Reagan with a depiction of nuclear war.
I think you have the causality flipped around. Jonah is suggesting that something about Berkeley contributes to the prevalence of low conscientiousness among rationalists.
I would not call this rudimentary! This is excellent. I’ll be using this.
Didn’t someone also do this for each post in the sequences a while back?
Is anyone willing to share an Anki deck with me? I’m trying to start using it. I’m running into a problem likely derived from having never, uh, learned how to learn. I look through a book or a paper or an article, and I find it informative, and I have no idea what parts of it I want to turn into cards. It just strikes me as generically informative. I think that learning this by example is going to be by far the easiest method.
You could construct an argument about needing to reinforce explicitly using system-2 ethics on common situations to make sure that you associate those ethics implicitly with normal situations, and not just contrived edge cases. But that seems to be even a bit too charitable. And also easily fixed if so.
A point I think others missed here is that in the TV example, there’s more data than the situations the OP talks about, so mscottveach can say there’s a disparity instead of just having the hatemail. Maybe more situations should involve anonymous polling.
I think people tend to need a decent amount of evidence before they start talking about someone looking potentially abusive. Then the crux is “does this behavior seem normal or like a predictive red flag?”. In those cases, your lived experience directly influences your perception. Someone’s actions can seem perfectly fine to most people. But if some others experience spooky hair-raising flashes of their questionably abusive father or a bad ex, that’s evidence. The people who didn’t think anything was weird brush off the others as oversensitive, risk averse, or paranoid. Then those raising alarms think of everyone else as callous, imperceptive, or malicious. It’s not just people who don’t alieve the correct base rates. Certainly those people exist, though they’re much more plentiful on Tumblr than in person or on LW. It’s very non-obvious whether a strong reaction is correct.
Neither side can truly accept the other’s arguments. It’s a bad situation when both sides consider the other’s reasoning compromised beyond repair. That brings politics and accusations of bad faith on all sides. But there is a fact of the matter, and the truth is actually unclear. Anyone thinking at enough of a distance from the issue should have honest uncertainty. I suspect you’re particularly prone to refusing to let the conflicting experience of others be seen by your deep internal world-models, to strongly underestimating the validity and reliability of that type of evidence. That would cause what you say to be parsed as bad faith, which other people then respond to in kind. That would cause a positive feedback loop where your prior shifts even further away from them having useful things to say. Then you’d end up a frog boiled in a pot of drama nobody else is experiencing. I’m not sure this is what’s happening, but it looks plausible.
To correct one thing here, the Bussard ramjet has drag effects. It can only get you to about 0.2c, making it pretty pointless to bother if you have that kind of command over fusion power.
From what I’ve seen, it’s not rare at all. I count… myself and at least 7 other people who’ve expressed the sentiment in private across both this year and last year (it happened last year too). It is, however, something that is very difficult for people to speak up about. I think what’s going on is that different people care about differing portious of the solstice (community, message, aesthetics, etc) to surprisingly differing degrees, may have sensory sensitivites or difficulty with multiple audio input streams, and may or may not find children positive to be around in principle. I think this community has far more people for whom noisy children destroy the experience than the base rate of other communities.
From what I’ve observed, the degree to which children ruin events for certain people is almost completely lost on many others. It’s difficult to speak up largely because of sentiments like yours, which make it feel like people will think that I’m going against the idea of the community. For me, and I don’t think I’m exceptionally sensitive, I think it removes between a third and half of the value of going to the event.