It sounds like your view is that (say) a House with 5 legislators who are amazing on AI X-risk, 15 who seem like they’re kinda pretty good, and 415 others is actively worse than one with 5 amazing legislators and 430 others?
I’m not sure why you think this. I’d think that most of the ways in which the pretty good legislators could be disappointing would make them more similar to the 415 others, or less influential, rather than actively worse. And often it would still be somewhat helpful to have them in Congress, e.g. they’d generally be more likely than random legislators to vote for a good AI bill that has a chance at becoming law.
One big way it could backfire to have a pretty-good-seeming legislator in the house is if they become a leading voice on AI while having misguided views on AI. But the concern about candidates who have a combination of prioritizing AI, being very competent, and having misguided views on AI feels different than just having extremely high standards for amazingness on AI X-risk.
It sounds like your view is that (say) a House with 5 legislators who are amazing on AI X-risk, 15 who seem like they’re kinda pretty good, and 415 others is actively worse than one with 5 amazing legislators and 430 others?
I think it’s quite possible 1 great /15 maybes is worse than 1⁄0, depending on how you define “seem like kinda pretty good”. Or put another way, I don’t trust the ecosystem to distinguish kinda pretty good from mildly moderately bad. Here are some ways someone who was nominally an AI safety advocate could end up being net harmful:
suck up resources better spent on other people. Money, airtime, staff...
Be offputting in a way that ends up tarring AI safety (I’m pretty worried that Scott Weiner’s woke reputation will pass on to AI safety).
Make the coordination harder. If you have 5 very smart people whose top priority is AI, you can pivot pretty quickly. If you have those 5 people, plus 15 pretty smart people who are invested enough to feel offended if not included but not enough to put in the necessary time, pivoting is much harder.
Pass mediocre or counterproductive legislation/regulation that eats up the public’s appetite for AI safety work.
I’m especially worried about regulatory capture masquerading as safety.
This is pretty sensitive to current conditions. If donors are inexhaustible, I care less about suboptimal distribution of money. Once you have a core that’s working productively (5 might be enough) you can support a second ring where the pretty good people can go without risk of them trying to steer.
On the other hand, we might want a policy of automatically supporting anyone opposing someone the pro-AI PACs support, since the counterfactual is worse.
It sounds like your view is that (say) a House with 5 legislators who are amazing on AI X-risk, 15 who seem like they’re kinda pretty good, and 415 others is actively worse than one with 5 amazing legislators and 430 others?
I’m not sure why you think this. I’d think that most of the ways in which the pretty good legislators could be disappointing would make them more similar to the 415 others, or less influential, rather than actively worse. And often it would still be somewhat helpful to have them in Congress, e.g. they’d generally be more likely than random legislators to vote for a good AI bill that has a chance at becoming law.
One big way it could backfire to have a pretty-good-seeming legislator in the house is if they become a leading voice on AI while having misguided views on AI. But the concern about candidates who have a combination of prioritizing AI, being very competent, and having misguided views on AI feels different than just having extremely high standards for amazingness on AI X-risk.
I think it’s quite possible 1 great /15 maybes is worse than 1⁄0, depending on how you define “seem like kinda pretty good”. Or put another way, I don’t trust the ecosystem to distinguish kinda pretty good from
mildlymoderately bad. Here are some ways someone who was nominally an AI safety advocate could end up being net harmful:suck up resources better spent on other people. Money, airtime, staff...
Be offputting in a way that ends up tarring AI safety (I’m pretty worried that Scott Weiner’s woke reputation will pass on to AI safety).
Make the coordination harder. If you have 5 very smart people whose top priority is AI, you can pivot pretty quickly. If you have those 5 people, plus 15 pretty smart people who are invested enough to feel offended if not included but not enough to put in the necessary time, pivoting is much harder.
Pass mediocre or counterproductive legislation/regulation that eats up the public’s appetite for AI safety work.
I’m especially worried about regulatory capture masquerading as safety.
This is pretty sensitive to current conditions. If donors are inexhaustible, I care less about suboptimal distribution of money. Once you have a core that’s working productively (5 might be enough) you can support a second ring where the pretty good people can go without risk of them trying to steer.
On the other hand, we might want a policy of automatically supporting anyone opposing someone the pro-AI PACs support, since the counterfactual is worse.