I think that losing your faith in civilization adequacy does feel a bit more like a deconversion experience. All your safety nets are falling, and I cannot promise you that we’ll replace them all. The power that ‘made things okay’ is gone from the world.
Ben Pace
Karma: 13,497
Hm, what caused them? I’m not sure exactly, but I will riff on it for a bit anyway.
Why was I uninterested in hanging out with most people? There was something I cared about quite deeply, and it felt feasible that I could get it, but it seemed transparent that these people couldn’t recognise it or help me get it and I was just humouring them to pretend otherwise. I felt kinda lost at sea, and so trying to understand and really integrate others’ worldviews when my own felt unstable was… it felt like failure. Nowadays I feel stable in my ability to think and figure out what I believe about the world, and so I’m able to use other people as valuable hypothesis generation, and play with ideas together safely. I feel comfortable adding ideas to my wheelhouse that aren’t perfectly vetted, because I trust overall I’m heading in a good direction and will be able to recognise their issues later.
I think that giving friends a life-presentation and then later noticing a clear hole in it felt really good, it felt like thinking for myself, putting in work, and getting out some real self-knowledge about my own cognitive processes. I think that gave me more confidence to interact with others’ ideas and yet trust I’d stay on the right track. I think writing my ideas down into blogposts also helped a lot with this.
Generally building up an understanding of the world that seemed to actually be right, and work for making stuff, and people I respected trusted, helped a lot.
That’s what I got right now.
Oh, there was another key thing tied up with the above: feeling like I was in control of my future. I was terrible at being a ‘good student’, yet I thought that my career depended on doing well at university. This lead to a lot of motivated reasoning and a perpetual fear that made it hard to explore, and gave me a lot of tunnel vision throughout my life at the time. Only when I realised I could get work that didn’t rely on good grades at university, but instead on trust I had built in the rationality and EA networks, and I could do things I cared about like work on LessWrong, did I feel more relaxed about considering exploring other big changes I wanted in how I lived my life, and doing things I enjoyed.
A lot of these worries felt like I was waiting to fix a problem—a problem whose solution I could reach, at least in principle—and then the worry would go away. This is why I said ‘transitional’. I felt like the problems could be overcome.
You know,
About 2-3 months earlier, I chatted with Eliezer at a party. Afterward, on the drive home, I said to my friend
“Gosh, Eliezer looks awfully spindly. It looks like he’s lost weight, but it’s all gone from his face and his arms.”
I was starting to make all these updates about how it doesn’t look good to lose weight when you’re hitting 40, and that it’s important to lose weight early, and so on.
I told this to Eliezer later. He said I got points for noticing my confusion, which I was pleased about.
I’m reading Jameson as just saying that, from an editing standpoint, the wording was sufficiently confusing and had to stop for a few seconds to figure out that this wasn’t what Zvi was saying. Like, he didn’t believe Zvi believed it, but it nonetheless read like that for a minute.
(Either way, I don’t care about it very much.)
I’ll be the first to admit that Singularity is a better song than Five Thousand Years.
I agree it’s more fun musically speaking, but the line about entropy in Five Thousand Years gets me every time.
This is such an interesting use of a spoiler tags. I might try it myself sometime.
(These last two comments were very helpful for me, thanks.)
That’s close.
Engaging with CFAR and LW’s ideas about redesigning my mind and focusing on important goals for humanity (e.g. x-risk reduction), has primarily—not partially—majorly improved my general competence, and how meaningful my life is. I’m a much better person, more honest and true, because of it. It directly made my life better, not just my abstract beliefs about the future.
The difficulties above were transitional problems, not the main effects.
+1
I see. I guess that framing feels slightly off to me—maybe this is what you meant or maybe we have a disagreement—but I would say “Helping people not have worse lives after interacting with <a weird but true idea>”.
Like I think that similar disorienting things would happen if someone really tried to incorporate PG’s “Black Swan Farming” into your action space, and indeed many good startup founders have weird lives with weird tradeoffs relative to normal people that often leads to burnout. “Interacting with x-risk” or “Interacting with the heavy-tailed nature of reality” or “Interacting with AGI” or whatever. Oftentimes stuff humans have only been interacting with in the last 300 years, or in some cases 50 years.
I find the structure of this post very clear, but I’m confused about which are the ‘reality-masking’ problems that you say you spent a while puzzling. You list three bullets in that section, let me rephrase them as problems.
How to not throw things out just because they seem absurd
How to update on bayesian evidence even if it isn’t ‘legible, socially approved evidence’
How to cause beliefs to propagate through one’s model of the world
I guess this generally connects with my confusion around the ontology of the post. I think it would make sense for the post to be ‘here are some problems where puzzling at them helped me understand reality’ and ‘here are some problems where puzzling at them caused me to hide parts of reality from myself’, but you seem to think it’s an attribute of the puzzle, not the way one approaches it, and I don’t have a compelling sense of why you think that.
You give an example of teaching people math, and finding that you were training particular bad patterns of thought in yourself (and the students). That’s valid, and I expect a widespread experience. I personally have done some math tutoring that I don’t think had that property, due to background factors that affected how I approached it. In particular, I wasn’t getting paid, my mum told me I had to do it (she’s a private english teacher who also offers maths, but knows I grok maths better than her), and so I didn’t have much incentive to achieve results. I mostly just spoke with kids about what they understood, drew diagrams, etc, and had a fun time. I wasn’t too results-driven, mostly just having fun, and this effect didn’t occur.
More generally, many problems will teach you bad things if you locally hill-climb or optimise in a very short-sighted way. I remember as a 14 year old, I read Thinking Physics, spent about 5 mins per question, and learned nothing from repeatedly just reading the answers. Nowadays I do Thinking Physics problems weekly, and I spend like 2-3 hours per question. This seems more like a fact about how I approached it than a fact about the thing itself.
Looking up at the three bullets I pointed to, all three of them are important things to get right, that most people could be doing better on. I can imagine healthy and unhealthy ways of approaching them, but I’m not sure what an ‘unhealthy puzzle’ looks like.
Thank you, mingyuan, Nat and Chelsea, for organising the Solstice. It’s one of the most meaningful events I go to each year, that makes me feel like I care about the same things as so many other people I know.
As a second point, this retrospective is really detailed and I feel like I can get a lot of your knowledge from it, and I’m really glad something like this will be around for future solstice organisers to learn from.
Fair, but I expect I’ve also read those comments buried in random threads. Like, Nate said it here three years ago on the EA Forum.
The main case for [the problems we tackle in MIRI’s agent foundations research] is that we expect them to help in a gestalt way with many different known failure modes (and, plausibly, unknown ones). E.g., ‘developing a basic understanding of counterfactual reasoning improves our ability to understand the first AGI systems in a general way, and if we understand AGI better it’s likelier we can build systems to address deception, edge instantiation, goal instability, and a number of other problems’.
I have a mental model of directly working on problems. But before Eliezer’s post, I didn’t have an alternative mental model to move probability mass toward. I just funnelled probability mass away from “MIRI is working on direct problems they foresee in AI systems” to “I don’t understand why MIRI is doing what it’s doing”. Nowadays I have a clearer pointer to what technical research looks like when you’re trying to get less confused and get better concepts.
This sounds weirdly dumb to say in retrospect, because ‘get less confused and get better concepts’ is one of the primary ways I think about trying to understand the world these days. I guess the general concepts have permeated a lot of LW/rationality discussion. But at the time I guess I had a concept shaped whole in my discussion of AI alignment research, and after reading this post I had a much clearer sense of that concept.
I experienced a bunch of those disorientation patterns during my university years. For example:
I would only spend time with people who cared about x-risk as well, because other people seemed unimportant and dull, and I thought I wouldn’t want to be close to them in the long run. I would choose to spend time with people even if I didn’t connect with very much, hoping that opportunities to do useful things would show up (most of the time they didn’t). And yet I wasn’t able to hang out with these people. I went through maybe a 6 month period where when I met up with someone, the first thing I’d do was list out like 10-15 topics we could discuss, and try to figure out which were the most useful to talk about and in what order we should talk. I definitely also turned many of these people off hanging out with me because it was so taxing. I was confused about this at the time. I though I was not doing it well enough or something, because I wasn’t providing enough value to them such that they were clearly having a good time.
I became very uninterested in talking with people whose words didn’t cache out into a gears level model of the situation based in things I could confirm or understand. I went through a long period of not being able to talk to my mum about politics at all. She’s very opinionated and has a lot of tribal feels and affiliations, and seemed to me to not be thinking about it in the way I wanted to think about it, which was a more first-principles fashion. Nowadays I find it interesting to put engage with how she sees the world, argue with it, feel what she feels. It’s not the “truth” that I wanted, I can’t take in the explicit content of her words and just input them into my beliefs, but this isn’t the only way to learn from her. She has a valuable perspective on human coordination, that’s tied up with important parts of her character and life story, that a lot of people share.
Relatedly, I went through a period of not being able to engage with aphorisms or short phrases that sounded insightful. Now I feel more trusting of my taste in what things mean and which things to take with me.
I generally wasn’t able to connect with my family about what I cared about in life / in the big picture. I’d always try to be open and honest, and so I’d say something like “I think the world might end and I should do something about it” and they’d think that sounded mad and just ignore it. My Dad would talk about how he just cares that I’m happy. Nowadays I realise we have a lot of shared reference points for people who do things, not because they make you happy or because they help you be socially secure, but because they’re right, because they’re meaningful and fulfilling, and because it feels like it’s your purpose. And they get that, and they know they make decisions like that, and they understand me when I talk about my decisions through that frame.
I remember on my 20th birthday, I had 10 of my friends round and gave a half-hour power-point presentation on my life plan. Their feedback wasn’t that useful, but I realised like a week later, that the talk only contained info about how to evaluate whether a plan was good, and not how to generate plans to be evaluated. I’d just picked the one thing that people talked about that sounded okay under my evaluation process (publishing papers in ML, which was a terrible choice for me, I interacted very badly with academia). It took me a week to notice that I’d not said how to come up with plans. I then realised that I’d been thinking in a very narrow and evaluative way, and not been open to exploring interesting ideas before I could evaluate whether they worked.
I should say, these shifts have not been anything like an unmitigated failure, and I don’t now believe were worth it just because they caused me to be more socially connected to x-risk things or because they were worth it in some pascal’s mugging kind of way. Like, riffing off that last example, the birthday party was followed by us doing a bunch of other things I really liked—my friends and I read a bunch of dialogues from GEB after that (the voices people did were very funny) and ate cake, and I remember it fondly. The whole event was slightly outside my comfort zone, but everyone had a great time, and it was also in the general pattern of me trying to more explicitly optimise for what I cared about. A bunch of the stuff above has lead me to form the strongest friendships I had, much stronger than I think I expected I could have. And many other things I won’t detail here.
Overall the effects on me personally, on my general fulfilment and happiness and connection to people I care about, has been strongly positive, and I’m glad about this. I take more small social risks, and they pay off bigger. I’m better at getting what I want, getting sh*t done, etc. Here, I’m mostly just listing some of the awkward things I did while at university.
Huh, I’m surprised to hear you say you already knew it. I did not know this already. This is the post where I properly understood that Eliezer et al are interested in decision theory and tiling agents and so on, not because they’re direct failures that they expect of future systems, but because they highlight confusions that are in want of basic theory to describe them, and that this basic theory will hopefully help make AGI alignable. Like I think I’d heard the words once or twice before then, but I didn’t really get it.
(Its important that Embedded Agenyou came out too, which was entirely framed around this “list of confusions in want of better concepts / basic theory” so I has some more concrete things to pin this to.)
(Reminder that you can subscribe to a post to get notified of comments on that post.)
Huh? A lot of these points about evolution register to me as straightforwardly false. Understanding the theory of evolution moved us from “Why are there all these weird living things? Why do they exist? What is going on?” to “Each part of these organisms has been designed by a local hill-climbing process to maximise reproduction.” If I looked into it, I expect I’d find out that early medicine found it very helpful to understand how the system was built. This is like me handing you a massive amount of code that has a bunch of weird outputs and telling you to make it work better and more efficiently, and the same thing but where I tell you what company made the code, why they made it, and how they made it, and loads of examples of other pieces of code they made in this fashion.
If I knew how to operationalise it I would take a pretty strong bet that the theory of natural selection has been revolutionary in the history of medicine.
10 posts I like in the 2018 Review
Hey, actually no, we’re currently reviewing 2018′s posts. We’ve waited a year in order to give everyone the power of hindsight to figure out what was actually good.
Btw, I think you might be the same user as user taryneast. That account is eligible to vote. I suggest logging in with that account, or contacting us via intercom (bottom right corner of the screen) if you’d like to reset your password.
Woop! I did the same yesterday.