Sometimes I despair that our current world seems like it lacks the “civilizational adequacy” to handle many of the deployment issues raised here, like implementing competent global monitoring, or even just navigating around our own antitrust laws to allow AI developers to avoid races… or even just building common knowledge that AI misalignment is a problem in the first place!
I think some other rationalists share this pessimistic inclination, which leads them to think that we had better just get AI right early on, when it is still under the control of a single tech company and we aren’t forced to deal with the pandora’s box of geopolitical/coordination issues around deployment. (I think this is the wrong attitude—getting AI right early on would be great, but we still need to think about all these problems as a “Plan B” in case not everything goes swimmingly.)
One major drawback of this hope is that the timelines might not match up—advanced AI might be developed soon, while these speculative ideas might take decades to make their way from small-scale experiments to the level of maturity where they can powerfully influence national decisionmaking. Nevertheless, it still seems like a promising strategy to have in one’s portfolio, if only to help in scenarios where AI is developed in the second half of this century or later.
How do you think about this issue? Is the idea of creating new experimental institutions and leveling up civilizational adequacy too indirect/dilute/meta compared to trying to directly influence existing institutions / decisionmakers? Too slow-acting, as mentioned above? Maybe prediction markets (and other ideas) just aren’t promising enough, or are too intractable because of political opposition?
(Apologies for the late reply!) I think working on improved institutions is a good goal that could potentially help, and I’m excited about some of the work going on in general categories you mentioned. It’s not my focus because (a) I do think the “timelines don’t match up” problem is big; (b) I think it’s really hard to identify specific interventions that would improve all decision-making—it’s really hard to predict the long-run effects of any given reform (e.g., a new voting system) as the context changes. Accordingly, what feels most pressing to me is getting more clarity on specific measures that can be taken to reduce the biggest risks to humanity, and then looking specifically at which institutional changes would make the world better-positioned to evaluate and act on those types of measures. Hence my interest in AI strategy “nearcasting” and in AI safety standards.
Sometimes I despair that our current world seems like it lacks the “civilizational adequacy” to handle many of the deployment issues raised here, like implementing competent global monitoring, or even just navigating around our own antitrust laws to allow AI developers to avoid races… or even just building common knowledge that AI misalignment is a problem in the first place!
I think some other rationalists share this pessimistic inclination, which leads them to think that we had better just get AI right early on, when it is still under the control of a single tech company and we aren’t forced to deal with the pandora’s box of geopolitical/coordination issues around deployment. (I think this is the wrong attitude—getting AI right early on would be great, but we still need to think about all these problems as a “Plan B” in case not everything goes swimmingly.)
Since the cause of my despair is the lack of “civilizational adequacy”, I find myself drawn to the idea of new institutions (like prediction markets, charter cities, improved voting systems, etc) which might be able to help our society make better decisions. (For instance, if prediction markets were more widely used, society might be quicker to build common knowledge about the danger of misalignment risk. As a stretch goal, maybe prediction markets could actually help us evaluate and quickly implement good policies in response to the danger, preventing us from flailing around a la the covid-19 response! For more detail along these lines, see my winning entry in the Future of Life Institute’s “A.I. Worldbuilding Competition”, which was all about my hope that improved institutional designs could help create a wiser civilization better able to safely develop AI.)
One major drawback of this hope is that the timelines might not match up—advanced AI might be developed soon, while these speculative ideas might take decades to make their way from small-scale experiments to the level of maturity where they can powerfully influence national decisionmaking. Nevertheless, it still seems like a promising strategy to have in one’s portfolio, if only to help in scenarios where AI is developed in the second half of this century or later.
How do you think about this issue? Is the idea of creating new experimental institutions and leveling up civilizational adequacy too indirect/dilute/meta compared to trying to directly influence existing institutions / decisionmakers? Too slow-acting, as mentioned above? Maybe prediction markets (and other ideas) just aren’t promising enough, or are too intractable because of political opposition?
(Apologies for the late reply!) I think working on improved institutions is a good goal that could potentially help, and I’m excited about some of the work going on in general categories you mentioned. It’s not my focus because (a) I do think the “timelines don’t match up” problem is big; (b) I think it’s really hard to identify specific interventions that would improve all decision-making—it’s really hard to predict the long-run effects of any given reform (e.g., a new voting system) as the context changes. Accordingly, what feels most pressing to me is getting more clarity on specific measures that can be taken to reduce the biggest risks to humanity, and then looking specifically at which institutional changes would make the world better-positioned to evaluate and act on those types of measures. Hence my interest in AI strategy “nearcasting” and in AI safety standards.