Found on Twitter: https://mobile.twitter.com/paulg/status/1172767548363935744
Personal answer:
1. This WaitButWhy article on artificial superintelligence started my journey down the rabbit hole of AI alignment and actually, as a matter of fact, changed the course of my life
2. Runner-up: Meditations on Moloch—unsurprising choice, but it did have a profound impact on my worldview
The criteria for choosing your answer should be the impact the thing you read had on your life, not whether you agree with it, learned from it or would recommend it to others, though these things might be correlated.
This post contains my answer.
My answer from 2017 is here.
Significant Digits, the unofficial sequel to HPMOR. A certain section (alas, I don’t recall which) singlehandedly leveled me up as human being.
The text immediately and irrevocably rewired my brain, to the degree that I stopped reading and thought to myself, “I don’t know the full consequences yet, but whatever just happened in my head is staggeringly important to my personal development.” I was right.
I’ve been trying to succinctly describe my insight for about three years. The best I have so far is this: people are the person who arrived at a moment, not the person in the moment. For the more mathematically inclined, individuals are the integral of their experiences, not the sum.
In practice, it was like suddenly developing empathy and a theory of mind in the span of 60 seconds. While reading fan fiction of fan faction.
Huh – do you remember any clues at all about where/what-the-context was for the passage in question? Super curious.
“Zen and the Art of Motorcycle Maintenance,” by Robert Pirsig.
I read it when it first came out in 1974. I was a 22-year old paratrooper with less than a year to go before getting out of the Army. I knew I was going to go to college, but I really didn’t have any idea of what I wanted to study. Having gone to Woodstock and been highly influenced by the counterculture zeitgeist, I suppose I would have been an English major or something along those lines.
But ZAMM was a complete head slap. It opened my eyes to some stuff I had never thought about, and I ended up majoring in mathematics and philosophy when I finally got to college.
Although a degree in math and philosophy did not have any impact whatsoever in my future work or career paths or any other matter of consequence, it gave me a way of thinking and a foundation for exploring a variety of other intellectual paths in later years.
Many books influenced my life, but all things considered, ZAMM had the biggest influence.
Creating Friendly AI
Runner up: GEB
Blindsight, by Peter Watts, helped a lot with my depression (though I am very much an outlier here, and nobody should expect the same result) and was sort of my “rabbithole” moment for thinking about brains and consciousness and “What makes me, ‘me’?” and so on, which continues to pay dividends insofar as a hugely useful tactic for dealing with depression is realizing that “you” are, at best, just one brain-part of a whole system of brain-parts, some of which are malfunctioning, and that e.g. you shouldn’t necessarily trust the brain-part which is saying “You are awful and need to die.”
(It also helped my depression because there’s just a certain element of “horrifying beauty” in Blindsight, but again I don’t expect other people to agree.)
http://www.overcomingbias.com/2007/05/policy_tugowar.html
Made me realize that the things that are optimized for attention aren’t the things optimized for accomplishing anything useful. I’ve since pursued many contrarian rabbit holes, some of which were of enormous benefit.
Rationality, from AI to Zombies. I went into it knowing it would change how I thought forever (the path I took to RAZ explicitly warned me of this), but I had no idea how much.
It’s tempting to reply with a book of philosophy or mathematics or fiction here. However, if I look at actual impact on my life, the book that did the most to change my future was The C Programming Language, by Dennis Ritchie and Brian Kernighan. If it hadn’t been for that book, it’s very well possible that I’d never have gotten into computers or programming (I’d probably have become a doctor instead).
deleted
“On being a happy, healthy, and ethical member of an unhappy, unhealthy, and unethical profession” by Patrick J. Schiltz - http://faculty.law.miami.edu/mcoombs/Schlitz.htm
The article has helped change the course of my professional life. It’s specific to law, but has been relevant to how I look at other industries as well.