I’m skeptical of strategies which look like “steer the paradigm away from AI agents + modern generative AI paradigm to something else which is safer”. Seems really hard to make this competitive enough and I have other hopes that seem to help a bunch while being more likely to be doable.
(This isn’t to say I expect that the powerful AI systems will necessarily be trained with the most basic extrapolation of the current paradigm, just that I think steering this ultimate paradigm to be something which is quite different and safer is very difficult.)
I agree and I would add I am generally suspicious of a general pattern here. Let’s call it the ′ Steering Towards Safe Paradigms’-strategy.
It goes something like this:
>>> There is a technology X causing public harm Z. The problem is that X is very competitive—individual actors benefit greatly from using X.
Instead of working to (i) [develop technologies to] mitigate the harms of technology X, (ii) put in existing safeguards or develop new safeguards to prevent harm Z, (iii) or ban/govern technology X
we should instead purposely and deliberately develop a technology Y that is fundamentally different than X that does the same role as X without harm Z but slightly less competitively. >>>
A good example could be the development of renewable energy to replace fossil fuels to prevent climate change.
I am generally skeptical of strategies that look like ′ steer the paradigm’ as I think it overstates in general how much influence ′ we’ (individuals, governments, human society as a whole etc) have and falls prey to a social desirability bias. In so far as it is based on uncalibrated hopes and beliefs about the actual action space it can be net harmful.
Note that this is distinct from humanity automatically developing a technology Y’ that is more competitive than X without harm Z because Y’ is actually just straight up the next step in the tech tree that is more competitive for mundane market reasons.
An example could be the benefit of plastic billard balls instead of ivory billard balls saving the elephant. Although saving the elephant was iirc part of the reason to develop plastic billard balls the reason it worked was more mundane: plastic billard balls are far cheaper than ivory billard balls. In other words, standard market forces were likely the prime cause, not deliberate and purposeful policy to develop a less competitive but safer technology.
Let’s zoom in to the renewable energy example. Are these examples where the ′ Steering towards Safe Paradigms’-strategy was succesful?
The new tech (fusion, fission, solar, wind) is based on fundamental principles than the old tech (oil and gas).
Fusion would be an example but perpetually thirty years away. Fission works but wasnt purposely develloped to fight climate change. Wind is deliberatily and purposefully developed to combat climate change but unfortunately it is not competitive without large subsidies and most likely never will [mostly because of low energy density and off-peak load balance concerns].
Solar is at least somewhat competitive with fossil fuels [except because of load balancing it may not be able to replace fossil fuels completely] and purposely developped out of environmental concerns and would be the best example.
I think my main question marks here is: solar energy is still a promise. It hasnt even begun to make a dent in total energy consumption ( a quick perplexity search reveals only 2 percent of global energy is solar-generated). Despite the hype it is not clear climate change will be solved by solar energy.
Moreover, the real question is to what degree the development of competitive solar energy was the result of a purposeful policy. People like to believe that tech development subsidies have a large counterfactual but imho this needs to be explicitly proved and my prior is that the effect is probably small compared to overall general development of technology & economic incentives that are not downstream of subsidies / government policy.
Let me contrast this with two different approaches to solving a problem Z (climate change).
Deploy existing competitive technology (fission)
Mitigate the harm directly (geo-engineering)
It seems to me that in general the latter two approaches have a far better track record of counterfactually Actually Solving the Problem. And perhaps more cynically, the hopeful promise of ‘Steering towards Safe Paradigms’ - strategy in combatting climate change is taking away from the above more mundane strategies.
Solar + wind has made a huge dent in energy production, so I feel like this example is confused.
It does seem like this strategy just really worked quite well, and a combination of battery progress and solar would probably by-default replace much of fossil-fuel production in the long run. It already has to a quite substantial degree:
Hydro + Solar + Wind + other renewables has grown to something like 40% of total energy production (edit: in the EU, which feels like the most reasonable reference class for whether this worked).
No, it is not confused. Be careful with reading precisely what I wrote. I said total energy production worldwide, not electricity production in the european union.
As you can see Solar is still a tiny percentage of energy consumption. That is not to say that things will not change—I certainly hope so! I give it significant probability. But if we are to be honest with ourselves than it is currently yet to be seen whether solar energy will prove to be the solution.
Moreover, in the case that solar energy does take over and ′ solve’ climate change that still does not prove the thesis—that solar energy solving climate change being majorly the result of deliberate policy instead of the result of market forces / ceteris paribus technological development.
The strong version of this argument seems false (eg Habryka’s comment), but I think the weak version is true. That is, energy put into “purposely and deliberately develop a technology Y that is fundamentally different than X that does the same role as X without harm Z but slightly less competitively.” is inefficient compared to energy put into strategies (i), (ii), and (iii).
Please read carefully what I wrote—I am talking about energy consumption worldwide not electricity consumption in the EU. Electricity in the EU accounts only for a small percentage of carbon emissions.
See
As you can see, solar energy is still a tiny percentage of total energy sources. I don’t think it is an accident that the electricity split graph in the EU has been cited in this discussion because it is a proxy that is much more rose-colored.
Energy and electricity are often conflated in discussions around climate change, perhaps not coincidentally because the latter seems much more tractable to generate renewably than total energy production.
Ryan Greenblatt on steering the AI paradigm:
I agree and I would add I am generally suspicious of a general pattern here. Let’s call it the ′ Steering Towards Safe Paradigms’-strategy.
It goes something like this:
>>> There is a technology X causing public harm Z. The problem is that X is very competitive—individual actors benefit greatly from using X.
Instead of working to
(i) [develop technologies to] mitigate the harms of technology X,
(ii) put in existing safeguards or develop new safeguards to prevent harm Z,
(iii) or ban/govern technology X
we should instead purposely and deliberately develop a technology Y that is fundamentally different than X that does the same role as X without harm Z but slightly less competitively. >>>
A good example could be the development of renewable energy to replace fossil fuels to prevent climate change.
I am generally skeptical of strategies that look like ′ steer the paradigm’ as I think it overstates in general how much influence ′ we’ (individuals, governments, human society as a whole etc) have and falls prey to a social desirability bias. In so far as it is based on uncalibrated hopes and beliefs about the actual action space it can be net harmful.
Note that this is distinct from humanity automatically developing a technology Y’ that is more competitive than X without harm Z because Y’ is actually just straight up the next step in the tech tree that is more competitive for mundane market reasons.
An example could be the benefit of plastic billard balls instead of ivory billard balls saving the elephant. Although saving the elephant was iirc part of the reason to develop plastic billard balls the reason it worked was more mundane: plastic billard balls are far cheaper than ivory billard balls. In other words, standard market forces were likely the prime cause, not deliberate and purposeful policy to develop a less competitive but safer technology.
Let’s zoom in to the renewable energy example. Are these examples where the ′ Steering towards Safe Paradigms’-strategy was succesful?
The new tech (fusion, fission, solar, wind) is based on fundamental principles than the old tech (oil and gas).
Fusion would be an example but perpetually thirty years away. Fission works but wasnt purposely develloped to fight climate change. Wind is deliberatily and purposefully developed to combat climate change but unfortunately it is not competitive without large subsidies and most likely never will [mostly because of low energy density and off-peak load balance concerns].
Solar is at least somewhat competitive with fossil fuels [except because of load balancing it may not be able to replace fossil fuels completely] and purposely developped out of environmental concerns and would be the best example.
I think my main question marks here is: solar energy is still a promise. It hasnt even begun to make a dent in total energy consumption ( a quick perplexity search reveals only 2 percent of global energy is solar-generated). Despite the hype it is not clear climate change will be solved by solar energy.
Moreover, the real question is to what degree the development of competitive solar energy was the result of a purposeful policy. People like to believe that tech development subsidies have a large counterfactual but imho this needs to be explicitly proved and my prior is that the effect is probably small compared to overall general development of technology & economic incentives that are not downstream of subsidies / government policy.
Let me contrast this with two different approaches to solving a problem Z (climate change).
Deploy existing competitive technology (fission)
Mitigate the harm directly (geo-engineering)
It seems to me that in general the latter two approaches have a far better track record of counterfactually Actually Solving the Problem. And perhaps more cynically, the hopeful promise of ‘Steering towards Safe Paradigms’ - strategy in combatting climate change is taking away from the above more mundane strategies.
Solar + wind has made a huge dent in energy production, so I feel like this example is confused.
It does seem like this strategy just really worked quite well, and a combination of battery progress and solar would probably by-default replace much of fossil-fuel production in the long run. It already has to a quite substantial degree:
Hydro + Solar + Wind + other renewables has grown to something like 40% of total energy production (edit: in the EU, which feels like the most reasonable reference class for whether this worked).
No, it is not confused. Be careful with reading precisely what I wrote. I said total energy production worldwide, not electricity production in the european union.
As you can see Solar is still a tiny percentage of energy consumption. That is not to say that things will not change—I certainly hope so! I give it significant probability. But if we are to be honest with ourselves than it is currently yet to be seen whether solar energy will prove to be the solution.
Moreover, in the case that solar energy does take over and ′ solve’ climate change that still does not prove the thesis—that solar energy solving climate change being majorly the result of deliberate policy instead of the result of market forces / ceteris paribus technological development.
The strong version of this argument seems false (eg Habryka’s comment), but I think the weak version is true. That is, energy put into “purposely and deliberately develop a technology Y that is fundamentally different than X that does the same role as X without harm Z but slightly less competitively.” is inefficient compared to energy put into strategies (i), (ii), and (iii).
Please read carefully what I wrote—I am talking about energy consumption worldwide not electricity consumption in the EU. Electricity in the EU accounts only for a small percentage of carbon emissions.
See
As you can see, solar energy is still a tiny percentage of total energy sources. I don’t think it is an accident that the electricity split graph in the EU has been cited in this discussion because it is a proxy that is much more rose-colored.
Energy and electricity are often conflated in discussions around climate change, perhaps not coincidentally because the latter seems much more tractable to generate renewably than total energy production.