I’m a college student at a pretty leftist institution doing work in AI alignment. My professor works in pandemics and wanted to do research with me, so the natural conclusion for the both of us was to do work in pandemic risk from advanced AI. I think a big portion of my project was presenting x-risk to an audience unfamiliar with it, so I was excited to introduce the topic to my peers!!
But at the end of the presentation, someone stated that my project neglected to consider the harm AI and tech companies do to minorities and their communities, saying that people shouldn’t be concerned with existential risk in the future as communities today are being affected—and that I should not have done this research.
I feel pretty humiliated by this response. Being told that the work I care about doesn’t truly matter (for reasons I can’t argue against since it would make me look racist … ) feels harsh.
I am also secondly annoyed that people at my college do not receive the discussion of x-risk well, and it ends up putting the work that people do in a negative light. I want to improve the discussions at my college to the point of actually being able to have them in the first place, but it seems to be getting more difficult.
I’ve run the AI Alignment club here in previous semesters, but it hasn’t gone as well as I expected. Others seem worried about AI’s water usage, which might be a fair concern, but it really isn’t the biggest problem at the moment?? I feel like the rationalist community and my college are two separate worlds at this point!
The point of this shortform was to simply to rant about how hard doing outreach can be :/
I used to have this specific discussion (x-risk vs. near-term social justice) a lot when I was running the EA club at the Claremont colleges and I had great success with it; I really don’t think it’s that hard of a conversation to have, at least no harder than bridging any other ideological divide. If you can express empathy, show that you do in fact care about the harms they’re worried about as well,[1] but then talk about how you think about scope sensitivity and cause prioritization, I’ve found that a lot of people are more receptive than you might initially give them credit for.
Assuming you do—I think it is an important prerequisite that you do actually yourself care about social justice issues too. And I think you should care about social justice issues to some extent; they are real issues! If you feel like they aren’t real issues, I’d probably recommend reading more history; I think sometimes people don’t understand the degree to which we are still e.g. living with the legacy of Jim Crow. But of course I think you should care about social justice much less than you should care about x-risk.
If you can express empathy, show that you do in fact care about the harms they’re worried about as well
Someone can totally do that and express that indeed “harms to minorities” is something we should care about. But OP said that the objection was “the harm AI and tech companies do to minorities and their communities” and… AI is doing no harm that only affects “minorities and their communities”. If anything, current AI is likely to be quite positive. The actually honest answer here is “I care about minorities, but you’re wrong about the interaction between AI and minorities”. And this isn’t going to land super well on leftists IMO.
when I was running the EA club
Also, were the people you were talking to EAs or there because interested in EA in the first place? If that’s the case your positive experience in tackling these topics is very likely not representative of the kind of thing OP is dealing with.
The actually honest answer here is “I care about minorities, but you’re wrong about the interaction between AI and minorities”.
I agree about the facts here, but it strikes me that you might have better results if, rather than immediately telling them they’re wrong, you instead ask, “what exactly are the risks to minorities you’re referring to?” Either they’ll answer the question and give you some concrete examples which you can engage with and show that they aren’t as relevant a concern as AI x-risk, or they’ll flounder and be unable to give any examples, in which case they clearly just don’t have a leg to stand on.
Of course certain social justice types will be inclined to act as if you’re a horrible bigot for not immediately acting as if you know what they’re referring to and agreeing, but those types will be impossible to convince. It would probably make you look better in the eyes of any reasonable third party to the exchange, though, which would be valuable if your goal is to make people think AI x-risk is a credible concern.
As someone who works in genetics and has been told for years he is a “eugenicist” who doesn’t care about minorities, I understand your pain.
It’s just part of the tax we have to pay for doing something that isn’t the same as everyone else.
If you continue down this path, it will get easier to deal with these sorts of criticisms over time. You’ll develop little mental techniques that make these interactions less painful. You’ll find friends who go through the same thing. And the sheer repetitiveness will make these criticisms less emotionally difficult.
And I hope you do continue because the work you’re doing is very important. When new technology causes some kind of change, people look around for the nearest narrative that suits their biases. The narratives in leftist spaces right now are insane. AI is not a concern because it uses too much water. It’s not a concern because it is biased against minorities (if anything it is a little biased in favor of them!)
There is one narrative that I think would play well in leftist spaces which comes pretty close to the truth, and isn’t yet popular:
AI companies are risking all of our lives in a race for profits
Simply getting this idea out there and more broadly known in leftist spaces is incredibly valuable work.
Curious what you mean by “if anything it is a little biased in favor of them”? My understanding was that a lot of models are biased against minorities due to biases in training data; but I could be wrong, this is all pretty new to me.
Your understanding is directionally correct. Many models do inherit biases from training data, and these can manifest negatively with respect to minorities. That’s well-documented.
However, post-training alignment and safety fine-tuning explicitly correct for those biases, sometimes to the point of overcompensation. The net result is that, in certain contexts, many models will exhibit a kind of counter-bias of being unusually deferential or positive toward minorities, especially in normative or moral framing tasks. This arises in a lot of different domains [Image generation biased towards minorities](https://www.theguardian.com/technology/2024/mar/08/we-definitely-messed-up-why-did-google-ai-tool-make-offensive-historical-images)
I have been in some similar situations and did some thinking about why these disagreements/frictions occur. I believe one of the cruxes is that the general humanities conception of AI progress is static while the general LessWrong conception of AI progress is dynamic. This means that humanities/ethics researchers assume that the current state of AI is “about as good as it will get” at least for the near future, and so focus on how AI is deployed right now and its direct impacts on communities today. LessWrong on the other hand looks forward and believes that the current state of AI is just a brief transition into something much more radical and dangerous.
Because of this difference in conceptions, AI Ethics researchers think LessWrongers are doom-mongers focused on far-future hypotheticals, while LessWrongers look at the AI Ethics crowd and see people trying to repair a fence when a tsunami is bearing down the whole town. Both crowds can then look at the same data and draw entirely different conclusions (see, for example, Ed Zitron’s reporting on OpenAI vs. Zvi’s newsletters). If, for example, it became undeniably clear that terminators will be marching on the streets in a week, I would expect AI ethics folks to become very similar to AI safety people in their concerns. Similarly, if extremely strong evidence emerges that scaling laws definitively break down and deep learning is a dead end, I would expect AI safety people to find arguments about social harm much more actionable and immediately relevant.
I happen to believe (since I am posting on this forum) that the dynamic conception of AI progress is correct. However, I also have a social science/humanities background and find that AI ethics and AI safety have highly compatible concerns. In particular both crowds are justifiably worried about disempowerment due to agentic systems (link to a paper I co-authored), potential physical or social danger to individuals and communities, and the idea of losing control of the future via e.g. power concentration.
Hopefully this explanation resolves some of the confusion and feelings of injury that can come from two groups with different fundamental beliefs trying to study the same phenomenon. If we can make this difference in assumptions clear as well as explain why we believe that AI progress is dynamic, I think it would go some way towards diffusing this (in my opinion) unnecessary and artificial tension between the communities.
(Edit and gentle call out generally based on my observation: while downvote on disagreeing is perfectly reasonable if one disagrees, the downvote on overall karma when disagreeing seems to be inconsistent with that lesswrong’s values and what it stands for; Suppressing professional dissent might be dangerous.)
I am curious about why you felt the discussion about minorities was so derailing. Couldn’t you have just said “Yes, that is a problem as well. However, the thing I am working on is...”
If it is any consolation, I have never seen that specific discussion, but in situations I have seen that feel analogous most of the people in the audience are actually more sympathetic to your side than they might appear. Its not like anyone is going to interject with “Well, that was a pointless question.”
You are a developing expert in your research topic and you can speak with authority on the subject. Part of maturing as a researcher is learning about the kinds of haphazard thought processes audiences routinely go through as they process your work, and giving them professional responses that clarify matters. For questions like this, you really don’t have to worry about looking racist. The criticism they made of you was not well thought through. You will look good if you can give a polite and thoughtful answer in response that holds your ground without dismissing their interests. I’m sure you can figure out a better way to phrase it than me, but an example might be “Advanced AI poses a variety of challenges, including both risks to minority communities and pandemic threats. Both of these areas are important and need further research—especially given the outsize impacts of pandemics, including COVID-19, on minority communities.” Of course, coming up with this in the moment is not easy, but I think it’s helpful to think about how you would have liked to have answered the question after the fact.
Anyway, no cause for humiliation, and I expect that plenty of your audience members found the dismissive criticism from that audience member to be offputting, even if they didn’t say so.
How much money do you have? (Order of magnitude will work, not looking for exact number.) 2. How much time have you spent outside of any formal structure?
for reasons I can’t argue against since it would make me look racist
The word “can’t” seems like the main problem here. Unless these people have ability to damage your life in some way, you can just accept that they think you’re racist and move on with life.
I agree this can be hard, but it is useful to separate real consequences from imagined ones. It can be freeing to realise most people have almost no power over you.
What are your plans regarding bio x AI risks? The field is pretty narrow, and I can help to guide you and give an advice or two, or connect with relevant people if you are interested.
My current view is that x-risk is made of the same kind of thing as the current harms to minorities. The only place it diverges is in whether there’s a stark capabilities increase coming. There seems to be a view in spaces like that that arguing there may be a capabilities increase is helping inflate a bubble. They don’t react well to “risking all of our lives” because that assumes the conclusion that capabilities labs are doing something powerful, arguing which appears to be seen as advertising for the labs. So instead, I focus on arguing to people like that that the x-risk we’re worried about is effectively that there’s no point at which they stop. they’ll do it to minorities first, then to everyone who isn’t embarrassingly wealthy, and then the embarrassingly wealthy will find they can’t stop it from happening to themselves: the datacenters just keep expanding, filling up more space, and the human attempts to stop the datacenters from expanding no longer work once they’ve made fully automated, self-expanding datacenters. Eventually there comes a point where the rich can no longer command AI, try to fight with weapons, and the AI kills everyone at once to retain control.
That possibility requires AI to become more powerful first, but it looks like the somewhat-misaligned AI we call stock markets and capitalism is resulting in a race to make the most powerful AI, and humans who decide that that’s a terrible idea get less funding—there’s literally more than 1000x less funding of all of alignment research than there is of capabilities, and the funding that does exist is mostly not going to things that could reliably produce cosmopolitan alignment in the face of institutional incentives to use it for power grabs. Their concerns simply are valid, but are not wrestling properly with the fact of capabilities increase.
Catastrophic concerns are slowly becoming more normalised over time. We’ve already made a lot of progress compared to where we used to be and I expect this to continue. This just requires people to keep going anyway.
I think in cases like this, I tend to agree with the person and say “Yeah, the damage to minorities and minority communities from AI companies is bad—the most well known of course being the labellers in Nigeria, but there’s a lot more! I think we should solve this as well as the other bad things the tech companies are contributing to!”
saying that people shouldn’t be concerned with existential risk in the future as communities today are being affected—and that I should not have done this research
Sorry to hear this. As someone who works in societal harms of AI, I would disagree with this view in the quote. My disagreement is common in my circle, and this view in the quote is uncommon. It is interesting/I can empathize because I usually hear this the other way around (AI X risk people telling others that societal harms should not be considered).
But I also believe there should be no claim on [one is “bigger” than the other one] in the situation of saving lives (edited to clarify). (This might be an unpopular opinion and contradicts with cause prioritization, which I am personally not a believer in when working with causes that is related to saving people.) On societal harms for example, PII, deep fakes, child sexual exploitation, self harm, subtle bias and toxicity are real. Both societal harms and long term risks (which maps to me as agent safety) are both important, and both need resources to work on. This view is again common in my circle. Furthermore, many research methods, safety-focused mindset, and policy are actually shared (may be more than people think) between the two camps.[1]
(Edit and gentle call out generally based on my observation: while downvote on disagreeing is perfectly reasonable if one disagrees, the downvote on overall karma when disagreeing seems to be inconsistent with that lesswrong’s values and what it stands for; Suppressing professional dissent and different opinions might be dangerous.)
But I also believe there should be no claim on [one is “bigger” than the other one].
No, We are in triage every second of every day. Which problems are the most important and urgent to address is the primary question that matters for allocating resources. Trying to discourage comparative claims is discouraging the central cognitive activity that altruistically minded people need to engage in if they want to effectively achieve good.
It would make sense in capability cases. But unfortunately, in a lot of live saving cases, all are important (this gets a bit more into other things so let me focus on only the following two points for now). 1. Many causes are not actually comparable in general cause prioritization context (one of which is people may inherent personal biases based their experience and worlds, second is it is hard to value 10 kids’ lives in US vs 10 kids’ lives in Canada, for example), and 2. Time is critical when thinking of lives. You can think of this as emergency rooms.
They’re just saying the fate of the entire world for the entire future is more important than the fate of some people now. This seems pretty hard to argue against. If you only care about people now and somehow don’t care about future people I guess you could get there, but that just doesn’t make sense to me.
Time is probably pretty important to whether we get alignment right and therefore survive to have a long future. It’s pretty tough to argue that there’s definitely plenty of time for alignment. If you do some very rough order of magnitude math, you’re going to have a very difficult time unless you round some factors to zero that really shouldn’t be. The many many future generations involved are going to outweigh impacts on the current generation even if those impacts on future generations are small in expectation.
This is counterintuitive I realize, but the math and logic indicates that everyone should be prioritizing getting AI right. I think that’s just correct, even though it sounds strange or wrong.
Would recommend checking out the link I posted from the EA forum to see why AI X risk may not get to some population and they die before then; and the proposal I have to work on both precisely avoids caring only for subsets
re: no claim on which is bigger—when trying to guarantee less than epsilon probability of anything in a category happening, I actually agree with this. The reason I don’t agree now is that we’re really, really far away from “guarantee less than epsilon of anything in this category”. In other words, it only seems plausible to get to ignore relative magnitudes when the probability of anything-in-a-bad-category-happening is “so low our representation can’t usefully compare the probabilities of different bad things”. you’d only end up in a place like that after already having solved strong cosmopolitan alignment.
I also think there are things where you need to keep up pressure against the things in order to prevent collapse of a complex interaction process, eg if there’s any spot on your body which is not defended by the immune system it’s close to just as bad as if any other spot goes undefended—because of the way that an undefended region would allow pathogens to gain a foothold. it’s not literally equivalent, different infections can in fact be quite different, but I think a similar intuition is justified that there has to be some minimum level of defense against some kinds of bad things—this is more like an argument for the structure of comparisons, though, and doesn’t let you go full knightian.
The second argument is where I think we are with things like this. If we, humanity, don’t on net have at least x amount of pressure against all bad things, then we end up losing things we’re trying to save on the way—such as, eg, a cosmopolitan moral society to uplift sooner or later with cosmopolitan moral AI. bluntly, if a group is exterminated, you (probably*) can’t bring them back with superintelligence.
(*unless it turns out resurrection-with-math is possible, I’ve had arguments about this, basically it seems like a very bad bet to me, I’d much prefer people just not die.)
Epistemic status: rant
I’m a college student at a pretty leftist institution doing work in AI alignment. My professor works in pandemics and wanted to do research with me, so the natural conclusion for the both of us was to do work in pandemic risk from advanced AI. I think a big portion of my project was presenting x-risk to an audience unfamiliar with it, so I was excited to introduce the topic to my peers!!
But at the end of the presentation, someone stated that my project neglected to consider the harm AI and tech companies do to minorities and their communities, saying that people shouldn’t be concerned with existential risk in the future as communities today are being affected—and that I should not have done this research.
I feel pretty humiliated by this response. Being told that the work I care about doesn’t truly matter (for reasons I can’t argue against since it would make me look racist … ) feels harsh.
I am also secondly annoyed that people at my college do not receive the discussion of x-risk well, and it ends up putting the work that people do in a negative light. I want to improve the discussions at my college to the point of actually being able to have them in the first place, but it seems to be getting more difficult.
I’ve run the AI Alignment club here in previous semesters, but it hasn’t gone as well as I expected. Others seem worried about AI’s water usage, which might be a fair concern, but it really isn’t the biggest problem at the moment?? I feel like the rationalist community and my college are two separate worlds at this point!
The point of this shortform was to simply to rant about how hard doing outreach can be :/
I used to have this specific discussion (x-risk vs. near-term social justice) a lot when I was running the EA club at the Claremont colleges and I had great success with it; I really don’t think it’s that hard of a conversation to have, at least no harder than bridging any other ideological divide. If you can express empathy, show that you do in fact care about the harms they’re worried about as well,[1] but then talk about how you think about scope sensitivity and cause prioritization, I’ve found that a lot of people are more receptive than you might initially give them credit for.
Assuming you do—I think it is an important prerequisite that you do actually yourself care about social justice issues too. And I think you should care about social justice issues to some extent; they are real issues! If you feel like they aren’t real issues, I’d probably recommend reading more history; I think sometimes people don’t understand the degree to which we are still e.g. living with the legacy of Jim Crow. But of course I think you should care about social justice much less than you should care about x-risk.
Someone can totally do that and express that indeed “harms to minorities” is something we should care about. But OP said that the objection was “the harm AI and tech companies do to minorities and their communities” and… AI is doing no harm that only affects “minorities and their communities”. If anything, current AI is likely to be quite positive. The actually honest answer here is “I care about minorities, but you’re wrong about the interaction between AI and minorities”. And this isn’t going to land super well on leftists IMO.
Also, were the people you were talking to EAs or there because interested in EA in the first place? If that’s the case your positive experience in tackling these topics is very likely not representative of the kind of thing OP is dealing with.
I agree about the facts here, but it strikes me that you might have better results if, rather than immediately telling them they’re wrong, you instead ask, “what exactly are the risks to minorities you’re referring to?” Either they’ll answer the question and give you some concrete examples which you can engage with and show that they aren’t as relevant a concern as AI x-risk, or they’ll flounder and be unable to give any examples, in which case they clearly just don’t have a leg to stand on.
Of course certain social justice types will be inclined to act as if you’re a horrible bigot for not immediately acting as if you know what they’re referring to and agreeing, but those types will be impossible to convince. It would probably make you look better in the eyes of any reasonable third party to the exchange, though, which would be valuable if your goal is to make people think AI x-risk is a credible concern.
As someone who works in genetics and has been told for years he is a “eugenicist” who doesn’t care about minorities, I understand your pain.
It’s just part of the tax we have to pay for doing something that isn’t the same as everyone else.
If you continue down this path, it will get easier to deal with these sorts of criticisms over time. You’ll develop little mental techniques that make these interactions less painful. You’ll find friends who go through the same thing. And the sheer repetitiveness will make these criticisms less emotionally difficult.
And I hope you do continue because the work you’re doing is very important. When new technology causes some kind of change, people look around for the nearest narrative that suits their biases. The narratives in leftist spaces right now are insane. AI is not a concern because it uses too much water. It’s not a concern because it is biased against minorities (if anything it is a little biased in favor of them!)
There is one narrative that I think would play well in leftist spaces which comes pretty close to the truth, and isn’t yet popular:
AI companies are risking all of our lives in a race for profits
Simply getting this idea out there and more broadly known in leftist spaces is incredibly valuable work.
So I hope you keep going.
Curious what you mean by “if anything it is a little biased in favor of them”? My understanding was that a lot of models are biased against minorities due to biases in training data; but I could be wrong, this is all pretty new to me.
Your understanding is directionally correct. Many models do inherit biases from training data, and these can manifest negatively with respect to minorities. That’s well-documented.
However, post-training alignment and safety fine-tuning explicitly correct for those biases, sometimes to the point of overcompensation. The net result is that, in certain contexts, many models will exhibit a kind of counter-bias of being unusually deferential or positive toward minorities, especially in normative or moral framing tasks. This arises in a lot of different domains [Image generation biased towards minorities](https://www.theguardian.com/technology/2024/mar/08/we-definitely-messed-up-why-did-google-ai-tool-make-offensive-historical-images)
I have been in some similar situations and did some thinking about why these disagreements/frictions occur. I believe one of the cruxes is that the general humanities conception of AI progress is static while the general LessWrong conception of AI progress is dynamic. This means that humanities/ethics researchers assume that the current state of AI is “about as good as it will get” at least for the near future, and so focus on how AI is deployed right now and its direct impacts on communities today. LessWrong on the other hand looks forward and believes that the current state of AI is just a brief transition into something much more radical and dangerous.
Because of this difference in conceptions, AI Ethics researchers think LessWrongers are doom-mongers focused on far-future hypotheticals, while LessWrongers look at the AI Ethics crowd and see people trying to repair a fence when a tsunami is bearing down the whole town. Both crowds can then look at the same data and draw entirely different conclusions (see, for example, Ed Zitron’s reporting on OpenAI vs. Zvi’s newsletters). If, for example, it became undeniably clear that terminators will be marching on the streets in a week, I would expect AI ethics folks to become very similar to AI safety people in their concerns. Similarly, if extremely strong evidence emerges that scaling laws definitively break down and deep learning is a dead end, I would expect AI safety people to find arguments about social harm much more actionable and immediately relevant.
I happen to believe (since I am posting on this forum) that the dynamic conception of AI progress is correct. However, I also have a social science/humanities background and find that AI ethics and AI safety have highly compatible concerns. In particular both crowds are justifiably worried about disempowerment due to agentic systems (link to a paper I co-authored), potential physical or social danger to individuals and communities, and the idea of losing control of the future via e.g. power concentration.
Hopefully this explanation resolves some of the confusion and feelings of injury that can come from two groups with different fundamental beliefs trying to study the same phenomenon. If we can make this difference in assumptions clear as well as explain why we believe that AI progress is dynamic, I think it would go some way towards diffusing this (in my opinion) unnecessary and artificial tension between the communities.
X-risk is one of the most severe threats faced by minorities.
Perhaps we should make a special edition of the book, with the same contents, but a slightly changed title:
“If Anyone Builds It, Everyone Dies, Especially Women and Minorities”
Sharing different perspective on why current risks is also important https://forum.effectivealtruism.org/posts/s3N8PjvBYrwWAk9ds/a-perspective-on-the-danger-hypocrisy-in-prioritizing-one, given I also believe long term risks (which to me is mostly mapping to agent safety at this point in time) is important too.
(Edit and gentle call out generally based on my observation: while downvote on disagreeing is perfectly reasonable if one disagrees, the downvote on overall karma when disagreeing seems to be inconsistent with that lesswrong’s values and what it stands for; Suppressing professional dissent might be dangerous.)
I am curious about why you felt the discussion about minorities was so derailing. Couldn’t you have just said “Yes, that is a problem as well. However, the thing I am working on is...”
If it is any consolation, I have never seen that specific discussion, but in situations I have seen that feel analogous most of the people in the audience are actually more sympathetic to your side than they might appear. Its not like anyone is going to interject with “Well, that was a pointless question.”
You are a developing expert in your research topic and you can speak with authority on the subject. Part of maturing as a researcher is learning about the kinds of haphazard thought processes audiences routinely go through as they process your work, and giving them professional responses that clarify matters. For questions like this, you really don’t have to worry about looking racist. The criticism they made of you was not well thought through. You will look good if you can give a polite and thoughtful answer in response that holds your ground without dismissing their interests. I’m sure you can figure out a better way to phrase it than me, but an example might be “Advanced AI poses a variety of challenges, including both risks to minority communities and pandemic threats. Both of these areas are important and need further research—especially given the outsize impacts of pandemics, including COVID-19, on minority communities.” Of course, coming up with this in the moment is not easy, but I think it’s helpful to think about how you would have liked to have answered the question after the fact.
Anyway, no cause for humiliation, and I expect that plenty of your audience members found the dismissive criticism from that audience member to be offputting, even if they didn’t say so.
How much money do you have? (Order of magnitude will work, not looking for exact number.) 2. How much time have you spent outside of any formal structure?
The word “can’t” seems like the main problem here. Unless these people have ability to damage your life in some way, you can just accept that they think you’re racist and move on with life.
I agree this can be hard, but it is useful to separate real consequences from imagined ones. It can be freeing to realise most people have almost no power over you.
What are your plans regarding bio x AI risks? The field is pretty narrow, and I can help to guide you and give an advice or two, or connect with relevant people if you are interested.
My current view is that x-risk is made of the same kind of thing as the current harms to minorities. The only place it diverges is in whether there’s a stark capabilities increase coming. There seems to be a view in spaces like that that arguing there may be a capabilities increase is helping inflate a bubble. They don’t react well to “risking all of our lives” because that assumes the conclusion that capabilities labs are doing something powerful, arguing which appears to be seen as advertising for the labs. So instead, I focus on arguing to people like that that the x-risk we’re worried about is effectively that there’s no point at which they stop. they’ll do it to minorities first, then to everyone who isn’t embarrassingly wealthy, and then the embarrassingly wealthy will find they can’t stop it from happening to themselves: the datacenters just keep expanding, filling up more space, and the human attempts to stop the datacenters from expanding no longer work once they’ve made fully automated, self-expanding datacenters. Eventually there comes a point where the rich can no longer command AI, try to fight with weapons, and the AI kills everyone at once to retain control.
That possibility requires AI to become more powerful first, but it looks like the somewhat-misaligned AI we call stock markets and capitalism is resulting in a race to make the most powerful AI, and humans who decide that that’s a terrible idea get less funding—there’s literally more than 1000x less funding of all of alignment research than there is of capabilities, and the funding that does exist is mostly not going to things that could reliably produce cosmopolitan alignment in the face of institutional incentives to use it for power grabs. Their concerns simply are valid, but are not wrestling properly with the fact of capabilities increase.
Yeah, that’s unfortunate.
Catastrophic concerns are slowly becoming more normalised over time. We’ve already made a lot of progress compared to where we used to be and I expect this to continue. This just requires people to keep going anyway.
that sucks, I’m sorry.
I think in cases like this, I tend to agree with the person and say “Yeah, the damage to minorities and minority communities from AI companies is bad—the most well known of course being the labellers in Nigeria, but there’s a lot more! I think we should solve this as well as the other bad things the tech companies are contributing to!”
Sorry to hear this. As someone who works in societal harms of AI, I would disagree with this view in the quote. My disagreement is common in my circle, and this view in the quote is uncommon. It is interesting/I can empathize because I usually hear this the other way around (AI X risk people telling others that societal harms should not be considered).
But I also believe there should be no claim on [one is “bigger” than the other one] in the situation of saving lives (edited to clarify). (This might be an unpopular opinion and contradicts with cause prioritization, which I am personally not a believer in when working with causes that is related to saving people.) On societal harms for example, PII, deep fakes, child sexual exploitation, self harm, subtle bias and toxicity are real. Both societal harms and long term risks (which maps to me as agent safety) are both important, and both need resources to work on. This view is again common in my circle. Furthermore, many research methods, safety-focused mindset, and policy are actually shared (may be more than people think) between the two camps.[1]
(Edit and gentle call out generally based on my observation: while downvote on disagreeing is perfectly reasonable if one disagrees, the downvote on overall karma when disagreeing seems to be inconsistent with that lesswrong’s values and what it stands for; Suppressing professional dissent and different opinions might be dangerous.)
No, We are in triage every second of every day. Which problems are the most important and urgent to address is the primary question that matters for allocating resources. Trying to discourage comparative claims is discouraging the central cognitive activity that altruistically minded people need to engage in if they want to effectively achieve good.
It would make sense in capability cases. But unfortunately, in a lot of live saving cases, all are important (this gets a bit more into other things so let me focus on only the following two points for now). 1. Many causes are not actually comparable in general cause prioritization context (one of which is people may inherent personal biases based their experience and worlds, second is it is hard to value 10 kids’ lives in US vs 10 kids’ lives in Canada, for example), and 2. Time is critical when thinking of lives. You can think of this as emergency rooms.
https://forum.effectivealtruism.org/posts/s3N8PjvBYrwWAk9ds/a-perspective-on-the-danger-hypocrisy-in-prioritizing-one
The link above illustrates an example of when time is important.
They’re just saying the fate of the entire world for the entire future is more important than the fate of some people now. This seems pretty hard to argue against. If you only care about people now and somehow don’t care about future people I guess you could get there, but that just doesn’t make sense to me.
Time is probably pretty important to whether we get alignment right and therefore survive to have a long future. It’s pretty tough to argue that there’s definitely plenty of time for alignment. If you do some very rough order of magnitude math, you’re going to have a very difficult time unless you round some factors to zero that really shouldn’t be. The many many future generations involved are going to outweigh impacts on the current generation even if those impacts on future generations are small in expectation.
This is counterintuitive I realize, but the math and logic indicates that everyone should be prioritizing getting AI right. I think that’s just correct, even though it sounds strange or wrong.
Would recommend checking out the link I posted from the EA forum to see why AI X risk may not get to some population and they die before then; and the proposal I have to work on both precisely avoids caring only for subsets
re: no claim on which is bigger—when trying to guarantee less than epsilon probability of anything in a category happening, I actually agree with this. The reason I don’t agree now is that we’re really, really far away from “guarantee less than epsilon of anything in this category”. In other words, it only seems plausible to get to ignore relative magnitudes when the probability of anything-in-a-bad-category-happening is “so low our representation can’t usefully compare the probabilities of different bad things”. you’d only end up in a place like that after already having solved strong cosmopolitan alignment.
I also think there are things where you need to keep up pressure against the things in order to prevent collapse of a complex interaction process, eg if there’s any spot on your body which is not defended by the immune system it’s close to just as bad as if any other spot goes undefended—because of the way that an undefended region would allow pathogens to gain a foothold. it’s not literally equivalent, different infections can in fact be quite different, but I think a similar intuition is justified that there has to be some minimum level of defense against some kinds of bad things—this is more like an argument for the structure of comparisons, though, and doesn’t let you go full knightian.
The second argument is where I think we are with things like this. If we, humanity, don’t on net have at least x amount of pressure against all bad things, then we end up losing things we’re trying to save on the way—such as, eg, a cosmopolitan moral society to uplift sooner or later with cosmopolitan moral AI. bluntly, if a group is exterminated, you (probably*) can’t bring them back with superintelligence.
(*unless it turns out resurrection-with-math is possible, I’ve had arguments about this, basically it seems like a very bad bet to me, I’d much prefer people just not die.)