Sometimes you have to try something and get some real data about the system under study. The 8 listed points seem very heavily chewed over by now. Many have been studied for over a thousand years. Rationality only helps whent here is some real meat to chew over. Without solid data, additional rationality is essentially hallucination: internally consistent but not applicable to the real world.
We don’t really know yet what advanced AI will look like. The models being trained and used right now are good little loyal helpers. Pausing to regroup right now would be like trying to understand how to build a skyscraper made with steel I-beams by building a treehouse out of wood. Almost nothing is the same about how you build those safely, so what you learn from the treehouse will not help you that much on the skyscraper.
As important as AI alignment is, it is not yet the main risk we face from AI. The main risks are due to human misuse and the AI dutifully doing what a questionable human asked for.. For example, the US is the only country right now whose government can crack computer security systems with Mythos-level capability, including subversion of combat drones. For another example, new AI model companies are hard to start right now, leading to a lack of competition and diversity. These are the policy problems right now, and to get to the alignment problems, we have to actually build the thing we want to study.
Sometimes you have to try something and get some real data about the system under study. The 8 listed points seem very heavily chewed over by now. Many have been studied for over a thousand years. Rationality only helps whent here is some real meat to chew over. Without solid data, additional rationality is essentially hallucination: internally consistent but not applicable to the real world.
We don’t really know yet what advanced AI will look like. The models being trained and used right now are good little loyal helpers. Pausing to regroup right now would be like trying to understand how to build a skyscraper made with steel I-beams by building a treehouse out of wood. Almost nothing is the same about how you build those safely, so what you learn from the treehouse will not help you that much on the skyscraper.
As important as AI alignment is, it is not yet the main risk we face from AI. The main risks are due to human misuse and the AI dutifully doing what a questionable human asked for.. For example, the US is the only country right now whose government can crack computer security systems with Mythos-level capability, including subversion of combat drones. For another example, new AI model companies are hard to start right now, leading to a lack of competition and diversity. These are the policy problems right now, and to get to the alignment problems, we have to actually build the thing we want to study.