Morality must scale to be useful. A common failure mode for people is to engage in “performative morality” where they choose low-cost signaling over high-cost action. What’s easier to do—starting a sustainable energy company (and scaling it) or donating $10 to climate change activism and tweeting about the seas rising 24/7?
But “easy” doesn’t mean “effective”. A sustainable energy company has 1000x the impact of a $10 donation on reversing climate change. Sure, not everyone can—or should—build a company; but then it shouldn’t be the case that the person donating $10 feels some mysterious aura of morality and feels like they’re just doing just as much as the person running a sustainable energy company.
People should feel good about doing good. I think the problem starts when we start to reason qualitatively about actions instead of using a quantitative reasoning framework. This is where we start to judge the intent of an action instead of focusing on the outcomes.
This idealist vs. pragmatist argument has been repeated ad nauseam, but I think it’s worth bringing up again. Why? The world is getting more complex and the nature of problems we’re dealing with reflects that complexity. We cannot afford to think simplistically about those problems and go for solutions that feel good on the surface, but don’t actually move the needle a lot.
We all want to feel agentic and feel like we can contribute to making the world a better place. But we’re only effective at doing this when we’re intellectually honest about our limitations and know where our efforts fit into the overall plan.
AI alignment might be one of the areas that could benefit from this type of thinking. I haven’t followed the AI conversation all too closely, but it seems you have two groups. Group #1 is people who have been doing AI safety before LLMs went mainstream saying an aligned superintelligence is impossible to build. Group #2 is AI safety people who emerged post-LLM boom and believe building a safe superintelligence is a resource problem—one we can solve with the right combination of capital, hardware, software, and talent.
It’s common to see people from group #1 claim the moral high ground and show they’re doing more to stop the extinction of the human race. They might berate AI companies for building software, tweet about AI risk and sign petitions to “pause AI”, and make dollar donations to AI safety research. Often, people in this group will claim moral superiority on matters concerning AI development and acquire status in the process.
But the line of thinking I’ve previously should show that group #2 should get more airtime and recognition because they’re doing harder things to solve the problem of AI misalignment. Raising capital, building hardware, attracting talent—these are all extremely difficult things to do at scale. If someone working on AI alignment does them well, then they deserve more attention in my opinion.
It’s not that the OG AI safety researchers are wrong, or the newfangled AI safety researchers raising billions cannot be wrong. My view is based on the idea that many problems in the world are in fact resource coordination and allocation problem.
I think the history of technology suggests that capital formation and technological innovation go hand in hand; we have built technology to destroy each other (missiles) and technology to defend ourselves (Israel’s Iron Dome). My weakly held belief is that AI alignment will follow the same pattern.
In a world where AI alignment is a resource problem, I expect that people serious about solving the problem work harder on acquiring resources. It’s okay to fail at this and admit failure. What is not right is refusing to do the hard work of resource acquisition while claiming specific problems are unsolvable. This, in my opinion, is intellectual hubris (failing to acknowledge that many solutions are emergent) and fails to understand the pattern of technological breakthroughs.
More to the point, we should be wary of awarding points for intention and assigning status to the person who does a better job of signaling that they care about a particular problem and doesn’t do enough to effectively to solve the problem. This is difficult to do for many reasons (e.g., assessing what counts as “effective action” in a particular problem-solving domain), but if done correctly, can raise the baseline of people who work harder to actually solve problems facing the world today.
By saying morality must scale, we’re essentially saying that the extent to which someone cares about solving a problem can be deduced from how much impact they plan to have with their action. The hope is that people learn to do more useful things and avoid motivated stopping when moving through the solution space related to a particular problem.
Morality must scale to be useful. A common failure mode for people is to engage in “performative morality” where they choose low-cost signaling over high-cost action. What’s easier to do—starting a sustainable energy company (and scaling it) or donating $10 to climate change activism and tweeting about the seas rising 24/7?
But “easy” doesn’t mean “effective”. A sustainable energy company has 1000x the impact of a $10 donation on reversing climate change. Sure, not everyone can—or should—build a company; but then it shouldn’t be the case that the person donating $10 feels some mysterious aura of morality and feels like they’re just doing just as much as the person running a sustainable energy company.
People should feel good about doing good. I think the problem starts when we start to reason qualitatively about actions instead of using a quantitative reasoning framework. This is where we start to judge the intent of an action instead of focusing on the outcomes.
This idealist vs. pragmatist argument has been repeated ad nauseam, but I think it’s worth bringing up again. Why? The world is getting more complex and the nature of problems we’re dealing with reflects that complexity. We cannot afford to think simplistically about those problems and go for solutions that feel good on the surface, but don’t actually move the needle a lot.
We all want to feel agentic and feel like we can contribute to making the world a better place. But we’re only effective at doing this when we’re intellectually honest about our limitations and know where our efforts fit into the overall plan.
AI alignment might be one of the areas that could benefit from this type of thinking. I haven’t followed the AI conversation all too closely, but it seems you have two groups. Group #1 is people who have been doing AI safety before LLMs went mainstream saying an aligned superintelligence is impossible to build. Group #2 is AI safety people who emerged post-LLM boom and believe building a safe superintelligence is a resource problem—one we can solve with the right combination of capital, hardware, software, and talent.
It’s common to see people from group #1 claim the moral high ground and show they’re doing more to stop the extinction of the human race. They might berate AI companies for building software, tweet about AI risk and sign petitions to “pause AI”, and make dollar donations to AI safety research. Often, people in this group will claim moral superiority on matters concerning AI development and acquire status in the process.
But the line of thinking I’ve previously should show that group #2 should get more airtime and recognition because they’re doing harder things to solve the problem of AI misalignment. Raising capital, building hardware, attracting talent—these are all extremely difficult things to do at scale. If someone working on AI alignment does them well, then they deserve more attention in my opinion.
It’s not that the OG AI safety researchers are wrong, or the newfangled AI safety researchers raising billions cannot be wrong. My view is based on the idea that many problems in the world are in fact resource coordination and allocation problem.
I think the history of technology suggests that capital formation and technological innovation go hand in hand; we have built technology to destroy each other (missiles) and technology to defend ourselves (Israel’s Iron Dome). My weakly held belief is that AI alignment will follow the same pattern.
In a world where AI alignment is a resource problem, I expect that people serious about solving the problem work harder on acquiring resources. It’s okay to fail at this and admit failure. What is not right is refusing to do the hard work of resource acquisition while claiming specific problems are unsolvable. This, in my opinion, is intellectual hubris (failing to acknowledge that many solutions are emergent) and fails to understand the pattern of technological breakthroughs.
More to the point, we should be wary of awarding points for intention and assigning status to the person who does a better job of signaling that they care about a particular problem and doesn’t do enough to effectively to solve the problem. This is difficult to do for many reasons (e.g., assessing what counts as “effective action” in a particular problem-solving domain), but if done correctly, can raise the baseline of people who work harder to actually solve problems facing the world today.
By saying morality must scale, we’re essentially saying that the extent to which someone cares about solving a problem can be deduced from how much impact they plan to have with their action. The hope is that people learn to do more useful things and avoid motivated stopping when moving through the solution space related to a particular problem.