Although HIA will result in Humanity being better able to tackle AGI, I think it may also result in an adverse effect if we do not get down the problem of Human Alignment correctly. Now, if there are more people working to reduce existential risk than there are people who are working to advance AGI as fast as possible, HIA is helpful. And the more people that are working on existential risk compared to AGI, the more percentage points of existential risk gets reduced per IQ increased. However, we should consider the alternative. If there are currently a bunch of people working to accelerate AGI, and fewer people than that working on existential risk, then HIA has a negative effect on existential risk.
This I expect would be lessened somewhat as people with higher intelligence figure out that AGI might, possibly, kill us all if done wrong. But that lessening assumes everyone is working with an altruist mindset, and it might be the case that despite the altruistic motivations, personal greed/ambition/etc. overcomes it and has them work on AGI anyway. It might also be the case that greater intelligence results in greater rationalization without proper rationalist training, (although research suggests higher IQ leads to a naturally higher level of rational thinking, they are still independent) which would amplify the effects of unwholesome motivations.
The TL;DR here is that if we have more people “against us” (working on AGI) than “with us” (working on existential risk) more intelligence works for “them” and not for “us”.
Although HIA will result in Humanity being better able to tackle AGI, I think it may also result in an adverse effect if we do not get down the problem of Human Alignment correctly. Now, if there are more people working to reduce existential risk than there are people who are working to advance AGI as fast as possible, HIA is helpful. And the more people that are working on existential risk compared to AGI, the more percentage points of existential risk gets reduced per IQ increased. However, we should consider the alternative. If there are currently a bunch of people working to accelerate AGI, and fewer people than that working on existential risk, then HIA has a negative effect on existential risk.
This I expect would be lessened somewhat as people with higher intelligence figure out that AGI might, possibly, kill us all if done wrong. But that lessening assumes everyone is working with an altruist mindset, and it might be the case that despite the altruistic motivations, personal greed/ambition/etc. overcomes it and has them work on AGI anyway. It might also be the case that greater intelligence results in greater rationalization without proper rationalist training, (although research suggests higher IQ leads to a naturally higher level of rational thinking, they are still independent) which would amplify the effects of unwholesome motivations.
The TL;DR here is that if we have more people “against us” (working on AGI) than “with us” (working on existential risk) more intelligence works for “them” and not for “us”.