Occasionally, I will ask someone who is very skilled in a certain subject how they became skilled in that subject so that I can copy their expertise. A common response is that I should read a textbook in the subject.
Eight years ago, Luke Muehlhauser wrote,
For years, my self-education was stupid and wasteful. I learned by consuming blog posts, Wikipedia articles, classic texts, podcast episodes, popular books, video lectures, peer-reviewed papers, Teaching Company courses, and Cliff’s Notes. How inefficient!
I’ve since discovered that textbooks are usually the quickest and best way to learn new material.
However, I have repeatedly found that this is not good advice for me.
I want to briefly list the reasons why I don’t find sitting down and reading a textbook that helpful for learning. Perhaps, in doing so, someone else might appear and say, “I agree completely. I feel exactly the same way” or someone might appear to say, “I used to feel that way, but then I tried this...” This is what I have discovered:
When I sit down to read a long textbook, I find myself subconsciously constantly checking how many pages I have read. For instance, if I have been sitting down for over an hour and I find that I have barely made a dent in the first chapter, much less the book, I have a feeling of hopelessness that I’ll ever be able to “make it through” the whole thing.
When I try to read a textbook cover to cover, I find myself much more concerned with finishing rather than understanding. I want the satisfaction of being able to say I read the whole thing, every page. This means that I will sometimes cut corners in my understanding just to make it through a difficult part. This ends in disaster once the next chapter requires a solid understanding of the last.
Reading a long book feels less like I’m slowly building insights and it feels more like I’m doing homework. By contrast, when I read blog posts it feels like there’s no finish line, and I can quit at any time. When I do read a good blog post, I often end up thinking about its thesis for hours afterwards even after I’m done reading it, solidifying the content in my mind. I cannot replicate this feeling with a textbook.
Textbooks seem overly formal at points. And they often do not repeat information, instead putting the burden on the reader to re-read things rather than repeating information. This makes it difficult to read in a linear fashion, which is straining.
If I don’t understand a concept I can get “stuck” on the textbook, disincentivizing me from finishing. By contrast, if I just learned as Muehlhauser described, by “consuming blog posts, Wikipedia articles, classic texts, podcast episodes, popular books, video lectures, peer-reviewed papers, Teaching Company courses, and Cliff’s Notes” I feel much less stuck since I can always just move from one source to the next without feeling like I have an obligation to finish.
I get the feeling that for AI safety, some people believe that it’s crucially important to be an expert in a whole bunch of fields of math in order to make any progress. In the past I took this advice and tried to deeply study computability theory, set theory, type theory—with the hopes of it someday giving me greater insight into AI safety.
Now, I think I was taking a wrong approach. To be fair, I still think being an expert in a whole bunch of fields of math is probably useful, especially if you want very strong abilities to reason about complicated systems. But, my model for the way I frame my learning is much different now.
I think my main model which describes my current perspective is that I think employing a lazy style of learning is superior for AI safety work. Lazy is meant in the computer science sense of only learning something when it seems like you need to know it in order to understand something important. I will contrast this with the model that one should learn a set of solid foundations first before going any further.
Obviously neither model can be absolutely correct in an extreme sense. I don’t, as a silly example, think that people who can’t do basic arithmetic should go into AI safety before building a foundation in math. And on the other side of the spectrum, I think it would be absurd to think that one should become a world renowned mathematician before reading their first AI safety paper. That said, even though both models are wrong, I think my current preference is for the lazy model rather than the foundation model.
Here are some points in favor of both, informed by my first-person experience.
Points in favor of the foundations model:
If you don’t have solid foundations in mathematics, you may not even be aware of things that you are missing.
Having solid foundations in mathematics will help you to think rigorously about things rather than having a vague non-reductionistic view of AI concepts.
Subpoint: MIRI work is motivated by coming up with new mathematics that can describe error-tolerant agents without relying on fuzzy statements like “machine learning relies on heuristics so we need to study heuristics rather than hard math to do alignment.”
We should try to learn the math that will be useful for AI safety in the future, rather than what is being used for machine learning papers right now. If your view of AI is that it is at least a few decades away, then it’s possible that learning the foundations of mathematics will be more robustly useful no matter where the field shifts.
Points in favor of the lazy model:
Time is limited and it usually takes several years to become proficient in the foundations of mathematics. This is time that could have been spent reading actual research directly related to AI safety.
The lazy model is better for my motivation, since it makes me feel like I am actually learning about what’s important, rather than doing homework.
Learning foundational math often looks a lot like just taking a shotgun and learning everything that seems vaguely relevant to agent foundations. Unless you have a very strong passion for this type of mathematics, it would seem outright strange that this type of learning is fun.
It’s not clear that the MIRI approach is correct. I don’t have a strong opinion on this, however
Even if the MIRI approach was correct, I don’t think it’s my comparative advantage to do foundational mathematics.
The lazy model will naturally force you to learn the things that are actually relevant, as measured by how much you come in contact with them. By contrast, the foundational model forces you to learn things which might not be relevant at all. Obviously, we won’t know what is and isn’t relevant beforehand, but I currently err on the side of saying that some things won’t be relevant if they don’t have a current direct input to machine learning.
Even if AI is many decades away, machine learning has been around for a long time, and it seems like the math useful for machine learning hasn’t changed much. So, it seems like a safe bet that foundational math won’t be relevant for understanding normal machine learning research any time soon.
I think there are some serious low hanging fruits for making people productive that I haven’t seen anyone write about (not that I’ve looked very hard). Let me just introduce a proof of concept:
Final exams in university are typically about 3 hours long. And many people are able to do multiple finals in a single day, performing well on all of them. During a final exam, I notice that I am substantially more productive than usual. I make sure that every minute counts: I double check everything and think deeply about each problem, making sure not to cut corners unless absolutely required because of time constraints. Also, if I start daydreaming, then I am able to immediately notice that I’m doing so and cut it out. I also believe that this is the experience of most other students in university who care even a little bit about their grade.
Therefore, it seems like we have an example of an activity that can just automatically produce deep work. I can think of a few reasons why final exams would bring out the best of our productivity:
1. We care about our grade in the course, and the few hours in that room are the most impactful to our grade.
2. We are in an environment where distractions are explicitly prohibited, so we can’t make excuses to ourselves about why we need to check Facebook or whatever.
3. There is a clock at the front of the room which makes us feel like time is limited. We can’t just sit there doing nothing because then time will just slip away.
4. Every problem you do well on benefits you by a little bit, meaning that there’s a gradient of success rather than a binary pass or fail (though sometimes it’s binary). This means that we care a lot about optimizing every second because we can always do slightly better.
If we wanted to do deep work for some other desired task, all four of these reasons seem like they could be replicable. Here is one idea (related to my own studying), although I’m sure I can come up with a better one if I thought deeply about this for longer:
Set up a room where you are given a limited amount of resources (say, a few academic papers, a computer without an internet connection, and a textbook). Set aside a four hour window where you’re not allowed to leave the room except to go to the bathroom (and some person explicitly checks in on you like twice to see whether you are doing what you say you are doing). Make it your goal to write a blog post explaining some technical concept. Afterwards, the blog post gets posted to Lesswrong (conditional on it being at least minimal quality). You set some goal, like it must acheive 30 upvote reputation after 3 days. Commit to paying $1 to a friend for each upvote you score below the target reputation. So, if your blog post is at +15, you must pay $15 to your friend.
I can see a few problems with this design:
1. You are optimizing for upvotes, not clarity or understanding. The two might be correlated but at the very least there’s a Goodhart effect.
2. Your “friend” could downvote the post. It can easily be hacked by other people who are interested, and it encourages vote manipulation etc.
Still, I think that I might be on the right track towards something that boosts productivity by a lot.
Related to: The Lottery of Fascinations, other posts probably
When you are older, you will learn that the first and foremost thing which any ordinary person does is nothing.
Professor Quirrell in HPMOR Ch. 73
I will occasionally come across someone who I consider to be extraordinarily productive, and yet when I ask what they did on a particular day they will respond, “Oh I basically did nothing.” This is particularly frustrating. If they did nothing, then what was all that work that I saw!
I think this comes down to what we mean by doing nothing. There’s a literal meaning to doing nothing. It could mean sitting in a chair, staring blankly at a wall, without moving a muscle.
More practically, what people mean by doing nothing is that they are doing something unrelated to their stated task, such as checking Facebook, chatting with friends, browsing Reddit etc.
When productive people say that they are “doing nothing” it could just be that they are modest, and don’t want to signal how productive they really are. On the other hand, I think that there is a real sense in which these productive people truly believe that they are doing nothing. Even if their “doing nothing” was your “doing work”, to them it’s still a “doing nothing” because they weren’t doing the thing they explicitly set out to do.
I think, therefore, there is something of a “do nothing” differential, which helps explain why some people are more productive than others. For some people who are less productive than me, their “doing nothing” might just be playing video games. For me, my “doing nothing” is watching people debate the headline of a Reddit news article (and I’m not proud of this).
For those more productive than me, perhaps their “doing nothing” is reading blog posts that are tangentially related to what they are working on. For people more productive still, it might be obsessively re-reading articles directly applicable to their work. And for Terence Tao, his “doing nothing” might be reading math papers in fields other than the one he is supposed to be currently working in.
There are two questions which I think are important to distinguish:
Is AI x-risk the top priority for humanity?
Is AI x-risk the top priority of some individual?
The first question is perhaps extremely important in a general sense. However, the second question is, I think, more useful since it provides actionable information to specific people. Of course, the difficulty of answering the second question is that it depends heavily on individual factors, such as
The ethical system of the individual which they are using the evaluate the question.
The specific talents, and time-constraints of the individual.
I also partially object to placing AI x-risk into one entire bundle. There are many ways that people can influence the development of artificial intelligence:
Social research to predict and intervene on governance for AI
AI forecasting to help predict which type of AI will end up existing and what their impact will be
Even within technical research, it is generally considered that there are different approaches:
Machine learning research with an emphasis on creating systems that could scale to superhuman capabilities while remaining aligned. This would include, but would not be limited to
Paul Christiano-style research, such as expanding iterated distillation and amplification
ML robustness to distributional shifts
Fundamental mathematical research which could help dissolve confusion about AI capabilities and alignment. This includes
Uncovering insights into decision theory
Discovering the necessary conditions for a system to be value aligned
Examining how systems could be stable upon reflection, such as after self-modification
I understand that other people have different feelings about in-group identification, and so I sympathize with the idea. That being said, I have a pretty strong negative reaction to identity symbols. It would essentially be an explicit gesture that some people are part of the tribe and other people aren’t. It’s hard to explain exactly why, but I would probably be less likely to interact with the community by a non-trivial amount if this because widespread.
I’ve often wished that conversation norms shifted towards making things more consensual. The problem is that when two people are talking, it’s often the case that one party brings up a new topic without realizing that the other party didn’t want to talk about that, or doesn’t want to hear it.
Let me provide an example: Person A and person B are having a conversation about the exam that they just took. Person A bombed the exam, so they are pretty bummed. Person B, however, did great and wants to tell everyone. So then person B comes up to person A and asks “How did you do?” fully expecting to brag the second person A answers. On it’s own, this question is benign. This happens frequently without question. On the other hand, if person B had said, “Do you want to talk about the exam?” person A might have said “No.”
This problem can be alleviated by simply asking people whether they want to talk about certain things. For sensitive topics, like politics and religion, this is already the norm in some places. I think it can be taken further. I suggest the following boundaries, and could probably think of more if pressed:
Ask someone before sharing something that puts you in a positive light. Make it explicit that you are bragging. For example, ask “Can I brag about something?” before doing so.
Ask someone before talking about something that you know there’s a high variance of difficulty and success. This applies to a lot of things: school, jobs, marathon running times.
It’s unclear to me how this is different from other boxing designs which merely trade some usefulness for safety. Therefore, like the other boxing designs, I don’t think this is a long term solution. There isn’t an obvious question that, if we could just ask an Oracle AI, the world would be saved. For sure, we should focus on making the first AGIs safe, and boxing methods may be a good way to do this. But creating AI’s with epistemic design flaws seems like a risky solution. There are potentially many ways that, if the AI ever got out of the box, we would see malignant instantiations due to its flawed understanding of the world.