Please don’t overhype or oversell AI Safety as a professional field, and in particular how easy it is to work in AI Safety. You ~have to be either quite smart or hard working/motivated to get a stable career, sometimes even both. AI Safety is generally attention and mentoring and management bottlenecked1, there is much less pipeline for “normal people” to make it, contrary to most standardized fields. When teaching and mentoring people, not all will be able to transition fast or upskill fast or get strong positions, some would benefit more from keeping their careers or donating or upskilling over longer timespans.
More smart wise well motivated people caring and working on good AI futures will keep mattering until the end, and the total number of people who can meaningfully contribute is probably in the millions, so let’s not stop field building
What does it mean to teach AI Safety?
There are many facts about AI, and theories about how it will go under what conditions. It means getting to learn and understand these well. Teaching AI Safety well should not over-determine someone to come out of the pipeline with a MIRI-view, nor an Anthropic-view, nor an EA-view, but should allow them to pass the basic Ideological Turing Test for all of these.
A central example of AI Safety fundamental knowledge is the AI Safety Atlas
On teaching AI Safety
AI safety changes all the time. Update your slides ~everytime you teach something.
If you’re new to teaching, find existing resources to build on. Be careful of AI slop. Better much less content than irrelevant or unsupported. Test yourself on your own knowledge, imagining questions one might ask and answering them. Traditional “how to teach/mentor/tutor” advice/resources are helpful.
Understand peoples’ motivations asap
For a given cohort, some will be motivated by “Do Good” EA or EA-adjacent reasons. Some by trends. Some by interest. Broadly all are valid in that there’s a place for a skilled someone with any of these motivations (but certain positions are only fit for certain motivations).
For people who want to have a lot of good impact, contributing towards good futures, context matters a lot
I recommend they seriously upskill in understand AI progress and trendlines. Read most EpochAI work and peter wildeford stuff, and AI threat models and scenarios, in particular thinking through AI 2027, contrasting with all other threat models listed in Deepmind’s lit review.
I recommend reading the AI Safety Atlas and understanding everything in it, as basic context for what the gameboard looks like, what areas exist in AIS.
And they’d benefit from building a pipeline for new info. Being involved in AIS community, having friends and colleagues, following the right people on twitter. Checking LessWrong from time to time.
For people who want to have a career or do fun research, they can just focus on that. They should find a context where someone else is thinking about why stuff matters and happy to have collaborators/employees executing on stuff. They should get good at whatever particular job or tasks they wanna do.
Most people interested in AI safety nowadays don’t know the old stuff, haven’t read the sequences. It’d be too much to ask them to read it all, but I do broadly recommend reading the best of LessWrong from most years. Being good at thinking and life is instrumental for being good at AI Safety. Different people need different advice so I can’t really put a best-of here (I really like Please don’t throw your mind away). There is wisdom in many different communities. If you don’t have the “rationalist community” wisdom then invest in it, otherwise keep branching out. Go to burning man or a local burn, understand the basics of meditation and different embodiment practices (eg. yoga), how therapy and self actualization work, read this blog and others. Keep asking questions about how to live the good life and have good community, actively pursue your best understanding of how to do that at any given moment.
Learnings and ramblings from teaching AI Safety for 3 years
On the field on AI Safety
Please don’t overhype or oversell AI Safety as a professional field, and in particular how easy it is to work in AI Safety. You ~have to be either quite smart or hard working/motivated to get a stable career, sometimes even both. AI Safety is generally attention and mentoring and management bottlenecked1, there is much less pipeline for “normal people” to make it, contrary to most standardized fields. When teaching and mentoring people, not all will be able to transition fast or upskill fast or get strong positions, some would benefit more from keeping their careers or donating or upskilling over longer timespans.
More smart wise well motivated people caring and working on good AI futures will keep mattering until the end, and the total number of people who can meaningfully contribute is probably in the millions, so let’s not stop field building
What does it mean to teach AI Safety?
There are many facts about AI, and theories about how it will go under what conditions. It means getting to learn and understand these well. Teaching AI Safety well should not over-determine someone to come out of the pipeline with a MIRI-view, nor an Anthropic-view, nor an EA-view, but should allow them to pass the basic Ideological Turing Test for all of these.
A central example of AI Safety fundamental knowledge is the AI Safety Atlas
On teaching AI Safety
AI safety changes all the time. Update your slides ~everytime you teach something.
If you’re new to teaching, find existing resources to build on. Be careful of AI slop. Better much less content than irrelevant or unsupported. Test yourself on your own knowledge, imagining questions one might ask and answering them. Traditional “how to teach/mentor/tutor” advice/resources are helpful.
Understand peoples’ motivations asap
For a given cohort, some will be motivated by “Do Good” EA or EA-adjacent reasons. Some by trends. Some by interest. Broadly all are valid in that there’s a place for a skilled someone with any of these motivations (but certain positions are only fit for certain motivations).
For people who want to have a lot of good impact, contributing towards good futures, context matters a lot
I recommend they seriously upskill in understand AI progress and trendlines. Read most EpochAI work and peter wildeford stuff, and AI threat models and scenarios, in particular thinking through AI 2027, contrasting with all other threat models listed in Deepmind’s lit review.
I recommend reading the AI Safety Atlas and understanding everything in it, as basic context for what the gameboard looks like, what areas exist in AIS.
And they’d benefit from building a pipeline for new info. Being involved in AIS community, having friends and colleagues, following the right people on twitter. Checking LessWrong from time to time.
For people who want to have a career or do fun research, they can just focus on that. They should find a context where someone else is thinking about why stuff matters and happy to have collaborators/employees executing on stuff. They should get good at whatever particular job or tasks they wanna do.
Most people interested in AI safety nowadays don’t know the old stuff, haven’t read the sequences. It’d be too much to ask them to read it all, but I do broadly recommend reading the best of LessWrong from most years. Being good at thinking and life is instrumental for being good at AI Safety. Different people need different advice so I can’t really put a best-of here (I really like Please don’t throw your mind away). There is wisdom in many different communities. If you don’t have the “rationalist community” wisdom then invest in it, otherwise keep branching out. Go to burning man or a local burn, understand the basics of meditation and different embodiment practices (eg. yoga), how therapy and self actualization work, read this blog and others. Keep asking questions about how to live the good life and have good community, actively pursue your best understanding of how to do that at any given moment.