This is one of those posts, like “pain is not the unit of effort,” that combines a memorable and informative and very useful and important slogan with a bunch of argumentation and examples to back up that slogan. I think this type of post is great for the LW review.
When I first read this post, I thought it was boring and unimportant: trivially, there will be some circumstances where knowledge is the bottleneck, because for pretty much all X there will be some circumstances where X is the bottleneck.
However, since then I’ve ended up saying the slogan “when money is abundant, knowledge is the real wealth” probably about a dozen separate times when explaining my career decisions, arguing with others at CLR about what our strategy should be, and even when deliberating to myself about what to do next. I guess longtermist EAs right now do have a surplus of money and a shortage of knowledge (relative to how much knowledge is needed to solve the problems we are trying to solve...) so in retrospect it’s not surprising that this slogan was practically applicable to my life so often.
I do think there are ways the post could be expanded and improved. Come to think of it, I’ll make a mini-comment right here to gesture at the stuff I would add to it if I could:
1. List of other ideas for how to invest in knowledge. For example, building a community with good epistemic norms. Or paying a bunch of people to collect data / info about various world developments and report on them to you. Or paying a bunch of people to write textbooks and summaries and explainer videos and make diagrams illustrating cutting-edge knowledge (yours and others’).
2. Arguments that in fact, right now, longtermist EAs and/or AI-risk-reducers are bottlenecked on knowledge (rather than money, or power/status)
--My own experience doing cost-benefit analyses is that interventions/plans vary in EV by OOMs and that it’s common to find new considerations or updated models that flip the sign entirely, or add or subtract a few OOMs, for a given intervention/plan. This sure seems like a situation in which more knowledge is really helpful compared to just having more money & ability to execute on plans.
--Everyone I’ve talked to in government/policy says that the bottleneck is knowledge. Nobody knows what to advocate for right now because everything is so uncertain and backfirey. (See previous point, lol)
--One might counterargue that it is precisely for the above reasons that we shouldn’t invest in knowledge; we aren’t getting much knowledge out of our research and we still won’t in the future. Instead, our best hope is to accumulate lots of power and resources and then when the crucial crunch time period comes, hopefully it’ll be clear what to do. Because the world might hand us knowledge on a silver platter, so to speak, in the form of new evidence. No need to deduce it in advance.
--(I have a couple responses to the above counterargument, but I take it seriously and think that others should too)
This is one of those posts, like “pain is not the unit of effort,” that combines a memorable and informative and very useful and important slogan with a bunch of argumentation and examples to back up that slogan. I think this type of post is great for the LW review.
When I first read this post, I thought it was boring and unimportant: trivially, there will be some circumstances where knowledge is the bottleneck, because for pretty much all X there will be some circumstances where X is the bottleneck.
However, since then I’ve ended up saying the slogan “when money is abundant, knowledge is the real wealth” probably about a dozen separate times when explaining my career decisions, arguing with others at CLR about what our strategy should be, and even when deliberating to myself about what to do next. I guess longtermist EAs right now do have a surplus of money and a shortage of knowledge (relative to how much knowledge is needed to solve the problems we are trying to solve...) so in retrospect it’s not surprising that this slogan was practically applicable to my life so often.
I do think there are ways the post could be expanded and improved. Come to think of it, I’ll make a mini-comment right here to gesture at the stuff I would add to it if I could:
1. List of other ideas for how to invest in knowledge. For example, building a community with good epistemic norms. Or paying a bunch of people to collect data / info about various world developments and report on them to you. Or paying a bunch of people to write textbooks and summaries and explainer videos and make diagrams illustrating cutting-edge knowledge (yours and others’).
2. Arguments that in fact, right now, longtermist EAs and/or AI-risk-reducers are bottlenecked on knowledge (rather than money, or power/status)
--My own experience doing cost-benefit analyses is that interventions/plans vary in EV by OOMs and that it’s common to find new considerations or updated models that flip the sign entirely, or add or subtract a few OOMs, for a given intervention/plan. This sure seems like a situation in which more knowledge is really helpful compared to just having more money & ability to execute on plans.
--Everyone I’ve talked to in government/policy says that the bottleneck is knowledge. Nobody knows what to advocate for right now because everything is so uncertain and backfirey. (See previous point, lol)
--One might counterargue that it is precisely for the above reasons that we shouldn’t invest in knowledge; we aren’t getting much knowledge out of our research and we still won’t in the future. Instead, our best hope is to accumulate lots of power and resources and then when the crucial crunch time period comes, hopefully it’ll be clear what to do. Because the world might hand us knowledge on a silver platter, so to speak, in the form of new evidence. No need to deduce it in advance.
--(I have a couple responses to the above counterargument, but I take it seriously and think that others should too)
--Reasoning by analogy, making AI go well sure seems like the sort of problem where knowledge is the bottleneck rather than money or power. It seems a lot more like figuring out the laws of physics and building a safe rocket before anyone gets killed in an unsafe rocket, or building a secure merchant drone operating system before anyone else builds an unsecure one, than e.g. preventing malaria or reforming US animal welfare laws.