Driving at night is not just about your own tiredness/circadian rhythm, there are other people driving tired and drunk.
In my college town, there’s a 1⁄4 mile long plastic-fenced in road, leading to rental houses. Every Thursday-Sunday night, someone will run their car into this fence, leaving broken fence marks the next morning.
Conspiracy theories are usually represented with a large amount of connections (and a distrust of those in power). Notably, I love Scott Alexander’s many self-created non-sense connections (see all of Unsong), which still end up evoking this sense of importance even though I know it’s fiction.
I’m glad you honed in on coherence, existential mattering, and purpose because there are an infinite amount of connections between things that feel unmeaningful (i.e. the grass and my mouse pad are both green, it is hot outside my door and also hot outside my door to a few feet to the right, etc.). Honing in on what specific properties makes a connection feel meaningful seems interesting (as well as looking at the existing literature and listing specific, real-life examples but that’s just my personal preference).
The strong emotion causing meaning (as opposed to a connection evoking meaning) was interesting, though couldn’t you say that specific connections cause strong emotions? For example, someone making fun of something I strongly identify with (“All your actions are selfish!”) as opposed to something I don’t really care about (“You’re a bad tuba player!”) affects me differently; I could describe each activity as weaker and stronger “connections” to myself.
A specific strong emotion that’s doesn’t quite fit is experiencing jhana, which I could describe as a meditative flow state that feels really good. It felt important and meaningful, though part of that is I had a pre-existing model of what “jhana” was and what it may mean. Specifically, I thought it meant that the rest of the crazy-sounding meditation claims like infinite happiness and willpower are way more likely to be true.
I’m currently interested in the idea of “the physical sensation correlation of different mental states”, like becoming intimately aware of the visceral, physical felt sense of being stressed or triggered, or relaxed and open, or having a strong sense of self or a small sense of identity, or a strong emotion in physical sensations only or a strong emotion with a story and sense of self attached or...
Specifically practicing this would look like paying attention to your body’s felt sense while doing [thing] in different ways (like interacting with your emotions using different system’s techniques). Building this skill will create higher quality feedback from your body’s felt-sense, allowing a greater ability to identify different states in the wild. This post’s idea of hijacked values and your comment point to a specific feeling attached to hijacked values.
This better bodily intuition may be a more natural, long term solution to these types of problems than what I would naively come up with (like TAPs or denying the part of me that actually wants the “bad” thing)
There is something here along the lines of “becoming skilled at a thing helps you better understand the appeal (and costs) of being skilled at other things”. It’s definitely not the only thing you need because I’ve been highly skilled at improv piano, but still desired these other things.
What I want to point out in the post is the disconnect between becoming highly skilled and what you actually value. It’s like eating food because it’s popular as opposed to actually tasting it and seeing if you like that taste (there was an old story here on LW about this, I think).
Making the cost explicit does help (“it would take decades to become a grandmaster”), but there can be a lack of feedback on why becoming a national master sounds appealing to you. Like the idea of being [cool title] sounds appealing, but is the actual, visceral, moment-to-moment experience of it undeniably enjoyable to you? (in this case, you can only give an educated guess until you become it, but an educated guess can be good enough!)
Oh! That makes sense as a post on it’s own.
Listing pros and cons of current rationalist techniques could then be compared to your ideal version of rationality to see what’s lacking (or points out holes in the “ideal version”). Also, “current rationality techniques” is ill-defined in my head and the closest I can imagine is the CFAR manual, though that is not the list I would’ve made.
No, which is part of the point.
I don’t know what point you’re referring to here. Do you mean that listing specific skills of rationality is bad for systematized winning?
I also want to wrangle more specifics from you, but I can just wait for your post:)
Regarding “problems we don’t understand”, you pointed out an important meta-systematic skill: figuring out when different systems apply and don’t apply (by applying new systems learned to a list of 20 or so big problems).
The new post you’re eluding to sounds interesting, but rationality is a loaded term. Do you have specific skills of rationality in mind for that post?
Your bulleted self-inquiries are very useful. These seem like more playful questions that I would feel comfortable asking someone else if I felt they were being hijacked by a metric/scaling (where a more naive approach could come across as judgmental and untactful).
Not all of your questions fit every situation of course, but that’s not the point! Actually, I want to try out a few examples:
What would it be like to be very skilled? I would be much fitter than I am now!, so less winded when doing other things. I feel like there’s a bragging angle, but who likes a bragger?
What would suck? The long practice hours, I will likely be more prone to injuries and joint problems.
What’s the good part of training to be a skilled runner? Consistently being outside would be nice. I think I would feel better after training.
What would be the bad part of training? That out-of-breath feeling and burning muscles is uncomfortable.
Are there people who aren’t skilled long distance runners, but are still better in a meaningful way? Swimmers are very fit, have greater upper body strength, and aren’t as prone to injuries (though looking it up, they do suffer shoulder injuries)
What would it look like to be successful? Being paid to do research full time. Making meaningful contributions that reduce x-risk. Having lots of smart people who will listen and give you feedback. Have a good understanding of lots of different, interesting topics.
What would suck about it? Maybe being in an official position will cause counter-productive pressure/responsibility to make meaningful contributions. I will be open to more criticism. I may feel responsible and slightly helpless regarding people who want to work on alignment, but have trouble finding funding.
What would be great about the process of becoming successful? Learning interesting subjects and becoming better at working through ideas. Gaining new frames to view problems. Meeting new people to discuss interesting ideas and “iron sharpening iron”. Knowing I’m working on something that feels legitimately important.
What would suck about the process? The context-loading of math texts is something to get used to. There’s a chance of failure due to lack of skill or not knowing the right people. There is no road map to guarantee success, so there is a lot of uncertainty on what to do specifically.
Any people who also are great but not successful Alignment researchers? There’s people who are good at communicating these ideas with others (for persuasion and distillation), or work at machine learning jobs and will be in good positions of power for AI safety concerns. There are also other x-risks to work on out there and EA fields that also viscerally feel important.
I’ll leave it here due to time, but I think I would add “How could I make the process of getting good more enjoyable?” and making explicit what goals I actually care about.
“Rationality” was a vague metric for me when I first started reading the sequences. Breaking it down into clear skills (taking ideas seriously, noticing confusion, “truth” as predictive accuracy, etc) with explicit benefits and common pitfalls would be useful.
Once you nail down what metrics you’re talking about when you say “rationality”, I believe the costs and benefits of investing in becoming more rational will be clearer.
Feel free to brainstorm as replies to this comment, I would enjoy a full post on the subject.
If the details are available within you
Maybe! One framing is: I expected “great accomplishments that people I admire say is good” to make me very happy or very liked, but reality was not as great, even negative sometimes. This pattern was hidden because:
I wasn’t explicit with my expectations—if I was clear with how happy all A’s would make me and paid attention when I did get an A, I would realize the disconnect sooner.
Related: making explicit the goals that all A’s helps me with (seriously considering why it matters in fine-grained details) would’ve been much more aligned with my goals than the proxy “get all A’s”. This serious analysis is not something I really did, but rationality skills of thinking an idea through while noticing confusions helped (I include focusing here)
I was in love with the idea of e.g. running a marathon and didn’t pay attention to how accomplishing it actually made me feel in physical sensations, or how the process I went about achieving that goal made me feel in physical sensations. This even happened with food! I used to eat a box of Zebra cakes (processed pastry cakes) on my drive home, but one time I decided to actually taste it instead of eating while distracted (inspired by mindful eating meditation). It was actually kind of gross and waxy and weirdly sweet, and I haven’t eaten more than a few of them these past several years.
I liked that you provided a lot of examples!
Thanks! Real life examples keep me honest. I was even thinking of your post, specifically the image of you scrambling to maintain and improve 10+ skills. How would you answer your own question?
For the gamification one, they tend to involve a bunch of open loops that leave you wanting to resolve (where the cliff hanger is a great example). This causes thoughts regarding the loop to come up spontaneously. In context, the loops aren’t that important, but locally, they may appear more important (like being angry at a loved one for interrupting you or preventing you from finishing a show/chapter/etc). I think being triggered in general here counts. Typical antidotes is the traditional “take a walk” and regarding meditation, better awareness and capacity to let go (not arguing that meditation works here, but may write a post on that)
This is different than cults and abusive relationships, where there is a strong motivation to leave your normal environment (the type of abusive relationship I have in mind is “you can’t see your friends anymore”), making the local rewards and punishments more salient as time goes by. I may even include drugs w/ withdrawals here. The usual solution is leaving those environments for healthier ones to compare against, though this happens in transitions due to ideas coupling (bucket errors). [This feels unsubstantiated to me and would benefit from more specific examples]. The gamification one had two answers: “change environment” or “change your relationship to the environment”. There may be some situations where you’re forced in a horrible environment and your only choice is to change your relationship to your environment, but this would require some high-level meditation insights in my opinion. “Leaving” seems the most actionable response. Maybe “recognizing” you’re in a cult is an important vein for future thought.
The identity based will cause ignoring/flinching from incompatible thoughts. This may benefit from becoming more sensitive to subtle thoughts you typically ignore (noticing confusions was a similar process for me). I feel like meditating relates, but I’m unsure on the mechanism. It’s mumble mumble everything is empty mumble.
There’s also a thread on “horrible events cause you to realize what’s important” to look into.
Thanks! Changed to “social appraisals”. Someone’s opinion is definitely a loaded term which may lead to pattern matching. I’m also fine with more novel phrasing since it’s explained immediately after.
I may have been misleading, but my point is not about tradeoffs, but about not pursuing things that you don’t actually care about upon reflection.
Thanks for bringing this up. I believe explicitly stating tradeoffs is important because you may then realize that you actually don’t care about them. For example, I don’t actually care about being “enlightened” or reaching stage 10 in TMI (though I thought I did). I would have come to a better conclusion and had better meditation sessions earlier if I made the metrics I care about explicit.
[Though, this isn’t true for looking cool dancing or eating new foods because I don’t know if I like them until it happens]
This post’s purpose is only to point out the pattern and nudge basic self-reflection, and that it is sometimes enough to solve the problem. It doesn’t solve all problems regarding hijacked values, which is what I’m currently trying to find a good-enough solution to (or create a good-enough taxonomy of types of hijacked values and heuristics).
For example, some of these are identity based. I saw myself as a hard worker, so I worked hard at every school assignment, even when it wasn’t at all necessary.
Others are gamificiation-y (like a video game): 100%-ing a game, reaching a certain rank/level, daily rewards!, or recommended videos/articles, or a cliff-hanger in a story with the next chapter available.
Others are extreme social incentive-y, such as cults, abusive relationships, and multi-level marketing, where local rewards and punishments become more salient than what you used to value (or would value if you left that environment for a few years).
I’m currently not in love with these divisions. A better framing (according to the metric of achieving your goals better) would make it clear what action to take in each situation to better know your true values.
Happy belated birthday, brother. My grandfather got married yesterday, so I was away from my laptop. You’ve missed a lot. I think it would’ve been real fun to discuss Gamestop stocks with you; I think you would’ve invested!
I’ve almost finished grad school as well and intend to study math for a bit on my own. You got to experience life after college for a bit; I wonder how your business and overall life would’ve gone since now. I think we would’ve argued vaccines, lockdowns, and mask when we saw each other. I remember arguing with you about politics and religion, but I don’t remember us ever getting mad at each other (except freshman year, but that was over a girl, haha).
I don’t even know where your grave is, but I’ll continue to talk to you here.
Looking back, this all seems mostly correct, but missing a couple, assumed steps.
I’ve talked to one person since about their mild anxiety talking to certain types of people; I found two additional steps that helped them.
Actually trying to become better
Understanding that their reaction is appropriate for some situations (like the original trauma), but it’s overgeneralized to actually safe situations.
These steps are assumed in this post because, in my case, it’s obvious I’m overreacting (there’s no drone) and I understand PTSD is common and treatable. Step 2 is very much Kaj Sotala’s internal family system’s post while this post is mainly about accessing lower-level sensory information about the trauma-reaction.
This post (or sequence of posts) not only gave me a better handle on impact and what that means for agents, but it also is a concrete example of de-confusion work. The execution of the explanations gives an “obvious in hindsight” feeling, with “5-minute timer”-like questions which pushed me to actually try and solve the open question of an impact measure. It’s even inspired me to apply this approach to other topics in my life that had previously confused me; it gave me the tools and a model to follow.
And, the illustrations are pretty fun and engaging, too.
I’m expecting either (1) A future GPT’s meta-learning combined with better prompt engineering will be able to learn the correct distribution and find the correct distribution, respectively. Or (2) curating enough examples will be good enough (though I’m not sure if GPT-3 could do it even then).
I also expect it to be harder as well, and 10-30% chance that it will require some new insight seems reasonable.