I don’t agree that targeting 100 IQ individuals is an effective strategy for slowing down AI development, because 100 IQ people generally don’t decide policy. Public opinion tends to matter very little in politics, especially in areas like AI policy that have little relation to everyday life.
Meanwhile, surprisingly enough, it turned out that regular “100 IQ individuals” with no prospect to become absurdly rich and powerful actually do not want an apocalypse! Too bad that we have already stained our reputation quite a bit, while appearing as bootlickers to the tech-billionares for all these years, but better late than never.
There is a lesson about naivity/cynycism and personal bias here. How it’s much more pleasant to persuade influential elites than common masses. The former feels like respectable intellectual activity, while the latter like—gah! - politics, something what crazy activists would do. And it’s good that we’ve managed to learn it, diversifying our activity and trying to appeal to common people more. Would’ve been even better if managed to win initially, instead of making this kind of fascinating mistake, but sadly we are not that good at rationality yet.
At least in democracies, convincing the people of something is an effective way to get politicians to pay attention to it—their job depends on getting these people to vote for them.
Notably in the UK, David Cameron gave the people a vote on whether to leave the EU because this was an idea that was gaining popularity. He did this despite not himself believing in the idea.
Naturally, plenty of legislation also gets passed without most people noticing, and in this respect we are better off convincing lawmakers. But I think that if we are able to convince a significant portion of the public, we will by extension convince a substantial number of lawmakers through their interaction with the public.
I have not read through the whole of the blogpost that you linked, but I disagreed with the “two important facts” used as a premise (1. People’s opinions are mostly genetic and 2. Most people’s opinions are completely random unless they’re smart.), and did not therefore trust any conclusions that might come from them.
Equally I get the impression that given the scale of the challenge, even if we were to cede that convincing the public is less important than convincing politicians, we will most likely need to do both to have a reasonable shot at passing anything that looks like good legislation.
I don’t agree that targeting 100 IQ individuals is an effective strategy for slowing down AI development, because 100 IQ people generally don’t decide policy. Public opinion tends to matter very little in politics, especially in areas like AI policy that have little relation to everyday life.
Convincing a few dozen influential people in tech, politics, and media is likely to have a vastly larger impact than winning over hundreds of millions of ordinary people. This blog post might help outline why: https://www.cremieux.xyz/p/the-cultural-power-of-high-skilled?utm_source=publication-search
That has been the default strategy for many years and it failed dramatically.
All the “convinced influential people in tech”, started making their own AI start-ups, while comming up with galaxy-brained rationalizations why everything will be okay with their idea in particular. We tried to be nice to them in order not to lose our influence with them. Turned out we didn’t have any. While we carefully and respectfully showed the problems with their reasoning, they likewise respectfully nodded their heads and continued to burn the AI timelines. Who could’ve though that people who have a real chance to become incredibly rich and important at the cost of dooming human civilization a bit later, are going to take this awesome opportunity?
Meanwhile, surprisingly enough, it turned out that regular “100 IQ individuals” with no prospect to become absurdly rich and powerful actually do not want an apocalypse! Too bad that we have already stained our reputation quite a bit, while appearing as bootlickers to the tech-billionares for all these years, but better late than never.
There is a lesson about naivity/cynycism and personal bias here. How it’s much more pleasant to persuade influential elites than common masses. The former feels like respectable intellectual activity, while the latter like—gah! - politics, something what crazy activists would do. And it’s good that we’ve managed to learn it, diversifying our activity and trying to appeal to common people more. Would’ve been even better if managed to win initially, instead of making this kind of fascinating mistake, but sadly we are not that good at rationality yet.
At least in democracies, convincing the people of something is an effective way to get politicians to pay attention to it—their job depends on getting these people to vote for them.
Notably in the UK, David Cameron gave the people a vote on whether to leave the EU because this was an idea that was gaining popularity. He did this despite not himself believing in the idea.
Naturally, plenty of legislation also gets passed without most people noticing, and in this respect we are better off convincing lawmakers. But I think that if we are able to convince a significant portion of the public, we will by extension convince a substantial number of lawmakers through their interaction with the public.
I have not read through the whole of the blogpost that you linked, but I disagreed with the “two important facts” used as a premise (1. People’s opinions are mostly genetic and 2. Most people’s opinions are completely random unless they’re smart.), and did not therefore trust any conclusions that might come from them.
Equally I get the impression that given the scale of the challenge, even if we were to cede that convincing the public is less important than convincing politicians, we will most likely need to do both to have a reasonable shot at passing anything that looks like good legislation.