This is not limited to AI, it applies to everything. The gap grows with power, technology, competence, intelligence, and everything of the sort. Society is currently trying to solve it through absolute surveillance and the disempowerment of citizens. The former is the aim for complete information of all citizens actions, and the second is laws forbidding people from having tools unless they need them (and even then, to restrict them with permits and such). In short, the government will either lower both the green and the red line for citizens, or it will make very sure that you get punished for anything under the red curve (e.g. trying to print money with your own printer). Companies are a bigger danger. I think the incentives will become strong enough to kill people, that people will start falling to Moloch with nobody specific to blame (as it’s all n-order effects for high n)
I think you’re also right that the red is worse than the green is good. While the gap is symmetrical, the asymmetry in the state space keeps growing. For instance, knives have shields as their defensive equivalence, but nuclear weapons don’t really have a defensive equivalence. Destruction is simply easier than construction, because there’s more bad states than good states. More complex things have a higher ratio of bad states—you can still use a deformed hammer as a hammer, but a CPU can take very little damage before it stops working. And it goes without saying that things are getting more complex over time.
I’m very fond of freedom personally, but I think we’re heading towards either tyranny (the state managing to control tech) or anarchy (the state failing to control tech), and it will be a wild ride either way. Neither extreme is appealing. The healthy balance, if one even exists (can exist), seems narrow and difficult to hit.
This is not limited to AI, it applies to everything. The gap grows with power, technology, competence, intelligence, and everything of the sort. Society is currently trying to solve it through absolute surveillance and the disempowerment of citizens. The former is the aim for complete information of all citizens actions, and the second is laws forbidding people from having tools unless they need them (and even then, to restrict them with permits and such). In short, the government will either lower both the green and the red line for citizens, or it will make very sure that you get punished for anything under the red curve (e.g. trying to print money with your own printer). Companies are a bigger danger. I think the incentives will become strong enough to kill people, that people will start falling to Moloch with nobody specific to blame (as it’s all n-order effects for high n)
I think you’re also right that the red is worse than the green is good. While the gap is symmetrical, the asymmetry in the state space keeps growing. For instance, knives have shields as their defensive equivalence, but nuclear weapons don’t really have a defensive equivalence. Destruction is simply easier than construction, because there’s more bad states than good states. More complex things have a higher ratio of bad states—you can still use a deformed hammer as a hammer, but a CPU can take very little damage before it stops working. And it goes without saying that things are getting more complex over time.
I’m very fond of freedom personally, but I think we’re heading towards either tyranny (the state managing to control tech) or anarchy (the state failing to control tech), and it will be a wild ride either way. Neither extreme is appealing. The healthy balance, if one even exists (can exist), seems narrow and difficult to hit.