Yup that tracks. (Mainly thinking out loud to see what I can do with my org:) I wonder if some sort of proof-of-work / proof-of-understanding incentive structure might help. I think one reason people focus on more concrete things in their upskilling journey is that it’s more legible to outsiders / offers more rewards/prestige/status. E.g. if you do some fellowship, that’s good for your resume and good for concrete artefacts. Deep (abstract) thinking doesn’t have similar legibility (yet), i.e. “I thought hard for 100h” probably doesn’t do much on a resume.
Luc Brinkman
Karma: 390
It’s really hard to get people to think hard about the core of the problem. So thanks for highlighting this and sharing your journey!
It’s not even just beginners that don’t like to go deep on this; MATS has tried running a strategy curriculum but fellows were apparently like ‘but we’re here now so we want to do the (empirical) research we came here to do’.
I think such thinking is important and neglected, so to get more people to do it, I recently co-created a new advanced AI Safety strategy course with my colleagues at Lens Academy; Forecasting, Modeling, and Shaping AI Futures.
Our motto with this course is something like: “What should people know that they wouldn’t otherwise come to know (because the topics aren’t sexy enough)?”
I might include this article in one of our courses. It seems pretty nice at conveying some important considerations to people.
Your first point is true to some extent but I think one of the points of Tsvi’s first article commented below exposes a problem with it:
”Deferral-based opinions don’t contain the detailed content that generated the opinions, and therefore can’t direct action effectively or update on new evidence correctly.”
Another problem is that if people defer, they will never get to that expert level.
But yeah, overall, I agree with the sentiment of your comment and I notice similar patterns in myself.
Yup I think we want to cater to power users if it’s not too hard, and this one seems relatively easy to implement (at least when remaining in the Anthropic ecosystem).
There’s potentially a world in which the in-app browser of the Claude app can be used, and then maybe maybe it can use its own subscription? Heard someone suggesting that some time but I didn’t look into it and I plausibly misunderstood it, and it’s probably pretty bad UX.
Hey I was thinking back of your comments, and I’d love to see if I can help you ideate for your journey. I’m the founder of an org (Lens Academy) that offfers AI Safety courses and ongoing AI guidance, and as part of that it also helps me to do some coaching by myself. Not super sure I can help but if you want, I’ll give it my best for a 30 minute session. You can book here https://www.temporary-url.com/648BC
Building a network and creating artefacts seem particularly important. It’s not just about being smart. No idea what you’ve already done but I feel like I see plenty of decent people get in and cracked people not get in, so I don’t think the bar is impossibly high. Higher for technical research than some other parts though
Stop Chasing Views: How to Reduce x-Risk as an AI Safety Content Creator
Hmm, sth like doing ML research or creating tooling that’s dual use without really attending to the idea that it might be dual use and without a clear theory of change. Sorry I don’t have something more specific for you here.
Thank you, this was very useful to me
I agree with the prior (at least in the case of CG) but not the latter. CG says they’re bottlenecked by funding allocation. Thus, distrubuting funding to ‘AI Automation’ likely takes away funding-allocating-resources from other domains, and thereby reduces funding for those other domains.
Conversely, if they do get more more human resources to allocate funding, they initially still have to choose whether to spend those marginally on ‘classical funding’ or on ‘Safety Automation’
I think so too. Part of an upcoming post in this sequence :)
Does this cartoon basically define impact as “affecting something”? Because then I’d say that’s not what I’m referring to with impact in the context of AI Safety. I mean something like “reduce probability of existentially bad outcomes”
Can you tell me more about that non-obvious relationship? At first I thought the main difference you were pointing at is that alignment is not a one-off thing but more of a continuous/recurring state. But I think you’re pointing at something else (too)
Yup, I agree that’s one factor working in favor of making nonprofit easier than forprofit
Meaning in for-profit there’s more competition and existing solutions are more efficient, thus you need to be very good to be marginally better?
Why Even Experts Don’t Know What to Do About AI Risk
Lack of conceptual understanding of the basics seems to me like a major reason why people keep on doing activities that sound like “AI Safety” but are likely making things worse.
It’s also what we’re trying to change with Lens Academy.
Interesting hypothesis that such basics might be less understood because they’re inherently less trained in practice while doing research. Seems plausible.
I do also think the AI Safety education space has a share in this, focusing too much on prosaic empirical methods at the cost of strategy and conceptual fundamentals.
Shout outs to AFFINE and Iliad Intensive for also emphasising the basics (as far as I can see)
As someone with a decent amount of ideas but not a lot of published writings, I notice I have a few things blocking me from writing more:
Writing taking (me with my current skillset and behaviors) a long time, and being very busy
Good writing taking a long time; not having internalized various methods of making writing good; needing to look up and reflect on such methods while writing, which is a very slow process until those methods are internalized
not wanting to waste people’s time with low quality writing
noticing that if I do iterate on a draft, I do (at least feel like) I am learning about writing and making progress in my abilities. So spending a long time on a draft doesn’t feel like wasted time. But it also doesn’t produce output, so reward is low.
not wanting to “dilute” my account with mediocre posts compared to a few posts that have seen much much more effort and with which I’m much happier / more proud.
I do enjoy the few times I have simply taken 20min to write a draft without the intention of publishing it. Those have usually been helpful in shaping my ideas.
This doesn’t really relate to your posts but it came to mind and I guess your post incentivized me to write more, resulting in this comment
Can you say more about what motivated you to write this? I like the points your make in and of themselves but I feel they’re left hanging without a conclusion or reason.
Hmm that’s tough.. And I imagine you’re not the only one to have this reaction. This does seem to be a legitimate downside of thinking through the high-level. It does feel somewhat important (for a good chunk of people) to go through that, though, I think. Like you say, it makes you a better researcher.
Sure, not everyone. Though more people should probably have better strategic takes than is the case now. One reason is that AI Safety is pretty pre-paradigmatic. Another one is that impact is heavy-tailed and I expect people with good strategic takes to be strongly overrepresented in that heavy tail.