Work done by Lesswrongians could decrease the workload of such an FAI while providing immediate results. If it takes twenty years for such a thing to be developed, that’s twenty years in either direction on the good/bad scale civilization could go. This could be the difference of an entire year that it takes an FAI to implement whatever changes to make society better.
This could be the difference of an entire year that it takes an FAI to implement whatever changes to make society better.
You are not taking AI seriously. Is this intentional?
A superintelligence could likely take over the world in a matter of days, no matter what people thought. (They would think it was great, because the AI could manipulate them better than the best current marketing tactics, even if it couldn’t just rewrite their brains with nano.)
It may not do this, for the sake of our comfort, but if anything was urgent, it would be done.
A superintelligence could likely take over the world in a matter of days, no matter what people thought. (They would think it was great, because the AI could manipulate them better than the best current marketing tactics, even if it couldn’t just rewrite their brains with nano.)
While I wouldn’t dismiss this possibility at all you seem a little overconfident. The best current marketing tactics can shift market share a percentage point or two or maybe make a half-percentage-point difference in a political campaign. Obviously better than the best is better. But assuming ethical limitations on persuasion tactics and general human suspicion of new things “days” seems pretty optimistic (and twenty-years pessimistic). No good reason to think the persuasive power of marketing is at all linear with the intelligence of the creator. We ought to have very large error bars on this kind of thing and while the focus on these fast take-over scenarios makes sense for emphasizing risk that focus will make them appear more likely to us than they actually are.
Work done by Lesswrongians could decrease the workload of such an FAI while providing immediate results. If it takes twenty years for such a thing to be developed, that’s twenty years in either direction on the good/bad scale civilization could go. This could be the difference of an entire year that it takes an FAI to implement whatever changes to make society better.
You are not taking AI seriously. Is this intentional?
A superintelligence could likely take over the world in a matter of days, no matter what people thought. (They would think it was great, because the AI could manipulate them better than the best current marketing tactics, even if it couldn’t just rewrite their brains with nano.)
It may not do this, for the sake of our comfort, but if anything was urgent, it would be done.
While I wouldn’t dismiss this possibility at all you seem a little overconfident. The best current marketing tactics can shift market share a percentage point or two or maybe make a half-percentage-point difference in a political campaign. Obviously better than the best is better. But assuming ethical limitations on persuasion tactics and general human suspicion of new things “days” seems pretty optimistic (and twenty-years pessimistic). No good reason to think the persuasive power of marketing is at all linear with the intelligence of the creator. We ought to have very large error bars on this kind of thing and while the focus on these fast take-over scenarios makes sense for emphasizing risk that focus will make them appear more likely to us than they actually are.