One of the things I miss most about the old world was that I still felt like I had a reason to be learning new things about science and engineering. When I was an undergrad I would dream up these wacky learning projects like trying to learn both Chinese and operating systems at once, by reading a Chinese translation of a book that did a line by line breakdown of an early linux kernel. I of course didn’t complete most of these projects, but I felt energized enough to attempt them, and to engage in real ways with ideas and background knowledge about software design. I felt like if I just worked hard enough I could at least become a really good engineer, and take pride in my craft even if it didn’t lead to money or status.
LLMs can probably teach me about linear algebra more effectively than whatever textbook I was reading in 2020, but all I can think now is, what is even the point? I’ll probably never even get to use it.
Alignment isn’t just about safety, it’s about your ability to get the LLMs to output stuff that actually fits your vision. If you don’t understand the thing you’re asking for, you won’t specify properly and you’ll end up with something you didn’t want.
I’ve noticed that as they’ve gotten more accurate, they’ve gotten less creative. So if everyone asks the same question, they get the same answer. Seeing that there’s an opportunity to do something different is probably going to be more, not less valuable in the future, and the only way to spot those opportunities will be to have better understanding yourself.
The returns on investment for understanding are probably going up not down. Figuring out where to focus your effort is probably getting harder not easier.
This sounds pretty motivational, but I notice myself spending significantly less time reading textbooks, so it is clearly not in line with my expressed preferences.
one hypothesis is that LLMs just provide a faster route to building knowledge, but I typically use them in a much more targeted way (than broad-based study like textbook reading).
I guess broad-based study also just pays off less in the short term, and maybe I’m planning for the short term (for reasons beyond just impending automation of knowledge work, such as impending extinction).
TBH, I’m still pretty hyped to learn differential geometry. I was reading Loring Tu’s fantastic Introduction to Manifolds earlier today, and sure, part of my reason is to be able to show off in front of others. But not all. Part of my identity is “guy who understands (theoretical) physics”, so I take actions in accord with that. The AIs knowing more physics than I do doesn’t change that. Many humans already know more physics than I do.
In fact, I’m getting more hyped over time to learn more. Learning unlocks new paths of seeing the world, and hence moving through the world. I am reminded of the old story of the thief who viewed a wall as no barrier to entry, since he knows how to get through. I am learning that is what all learning is like. It’s great, 10⁄10, would recommend.
funny, I feel like the opposite. learning is so cheap now. the scope of what i can do has been expanded humongously. I feel very personally uplifted and self actualised even though I have updated upwards on existential risk. It’s a weird place to be in emotionally
For context, how long are your timelines and how sure are you of them? / How sure are you that you’ll never get to use some knowledge? (Did you care very much in the past about getting to “use” the skills you learned in some direct sense?)
Checking if a solution to alignment (AI-generated or human-created) is the correct one is an alignment-complete problem. Meaning that for you to trust an AI with checking alignment solution, this AI has to be aligned already.
In light of this, I see my current quest as to upskill myself enough to be a useful person in the project of checking the AI-generated or AI-assisted solution to alignment. LLMs can be of use with that upskilling.
Spending time and effort to better understand, explore and interact with the many facets of one’s own mind could still be worthwhile in an automated world where most skills become useless, assuming individual human minds will still exist for a while in a similar form as today. I have a quite debilitating chronic pain condition that over the past years led me into exploring techniques like autogenic training, progressive muscle relaxation, self-hypnosis, mindfulness and nondual mediation, pain neuroscience education, cognitive behavioral therapy, psychedelics (in appropriate settings) and a few other things, which altered my view of what the human condition is and what states of human consciousness are accessible by various means. In particular, nondual meditation turned out to be surprisingly helpful when I’m hit by an extreme pain flair-up, I suspect due to a form of willfully induced and controlled dissociation that gets better the more you train it under ordinary conditions. I can now imagine less vaguely that monks who spend decades on meditation practice can endure apparently extreme situations up to self-immolation without actually suffering from it. Weaker forms of that are accessible by much less practice, perhaps comparable to semi-regularly going to a gym to build some muscles. Among the many resources out there, I’ve found Sam Harris’ meditation app particularly useful (not intended as a product ad, they also provide free access upon request without asking questions), which contains courses on various such topics by a wide range of experienced teachers. If anyone has other recommendations from personal experience I’d be quite interested in taking a look at new resources adjacent to that semantic cluster.
One of the things I miss most about the old world was that I still felt like I had a reason to be learning new things about science and engineering. When I was an undergrad I would dream up these wacky learning projects like trying to learn both Chinese and operating systems at once, by reading a Chinese translation of a book that did a line by line breakdown of an early linux kernel. I of course didn’t complete most of these projects, but I felt energized enough to attempt them, and to engage in real ways with ideas and background knowledge about software design. I felt like if I just worked hard enough I could at least become a really good engineer, and take pride in my craft even if it didn’t lead to money or status.
LLMs can probably teach me about linear algebra more effectively than whatever textbook I was reading in 2020, but all I can think now is, what is even the point? I’ll probably never even get to use it.
Alignment isn’t just about safety, it’s about your ability to get the LLMs to output stuff that actually fits your vision. If you don’t understand the thing you’re asking for, you won’t specify properly and you’ll end up with something you didn’t want.
I’ve noticed that as they’ve gotten more accurate, they’ve gotten less creative. So if everyone asks the same question, they get the same answer. Seeing that there’s an opportunity to do something different is probably going to be more, not less valuable in the future, and the only way to spot those opportunities will be to have better understanding yourself.
The returns on investment for understanding are probably going up not down. Figuring out where to focus your effort is probably getting harder not easier.
This sounds pretty motivational, but I notice myself spending significantly less time reading textbooks, so it is clearly not in line with my expressed preferences.
one hypothesis is that LLMs just provide a faster route to building knowledge, but I typically use them in a much more targeted way (than broad-based study like textbook reading).
I guess broad-based study also just pays off less in the short term, and maybe I’m planning for the short term (for reasons beyond just impending automation of knowledge work, such as impending extinction).
TBH, I’m still pretty hyped to learn differential geometry. I was reading Loring Tu’s fantastic Introduction to Manifolds earlier today, and sure, part of my reason is to be able to show off in front of others. But not all. Part of my identity is “guy who understands (theoretical) physics”, so I take actions in accord with that. The AIs knowing more physics than I do doesn’t change that. Many humans already know more physics than I do.
In fact, I’m getting more hyped over time to learn more. Learning unlocks new paths of seeing the world, and hence moving through the world. I am reminded of the old story of the thief who viewed a wall as no barrier to entry, since he knows how to get through. I am learning that is what all learning is like. It’s great, 10⁄10, would recommend.
funny, I feel like the opposite. learning is so cheap now. the scope of what i can do has been expanded humongously. I feel very personally uplifted and self actualised even though I have updated upwards on existential risk. It’s a weird place to be in emotionally
For context, how long are your timelines and how sure are you of them? / How sure are you that you’ll never get to use some knowledge? (Did you care very much in the past about getting to “use” the skills you learned in some direct sense?)
Checking if a solution to alignment (AI-generated or human-created) is the correct one is an alignment-complete problem. Meaning that for you to trust an AI with checking alignment solution, this AI has to be aligned already.
In light of this, I see my current quest as to upskill myself enough to be a useful person in the project of checking the AI-generated or AI-assisted solution to alignment. LLMs can be of use with that upskilling.
Spending time and effort to better understand, explore and interact with the many facets of one’s own mind could still be worthwhile in an automated world where most skills become useless, assuming individual human minds will still exist for a while in a similar form as today. I have a quite debilitating chronic pain condition that over the past years led me into exploring techniques like autogenic training, progressive muscle relaxation, self-hypnosis, mindfulness and nondual mediation, pain neuroscience education, cognitive behavioral therapy, psychedelics (in appropriate settings) and a few other things, which altered my view of what the human condition is and what states of human consciousness are accessible by various means. In particular, nondual meditation turned out to be surprisingly helpful when I’m hit by an extreme pain flair-up, I suspect due to a form of willfully induced and controlled dissociation that gets better the more you train it under ordinary conditions. I can now imagine less vaguely that monks who spend decades on meditation practice can endure apparently extreme situations up to self-immolation without actually suffering from it. Weaker forms of that are accessible by much less practice, perhaps comparable to semi-regularly going to a gym to build some muscles. Among the many resources out there, I’ve found Sam Harris’ meditation app particularly useful (not intended as a product ad, they also provide free access upon request without asking questions), which contains courses on various such topics by a wide range of experienced teachers. If anyone has other recommendations from personal experience I’d be quite interested in taking a look at new resources adjacent to that semantic cluster.
Edit: Typo