Coordination is hard. “Assigning Molochian elements a lower value” is a kind of coordination. Making rules, and punishing people when they break them is another. Even if attack is stronger than defense, the punishment could be stronger yet (because it is a kind of attack). I agree that it is difficult, not sure if impossible.
It doesn’t require conscious or direct coordination, but it does require a chain of cause and effect which affects many people. If society agrees that chasing after material goods rather than meaningful pursuits is bad taste, then the world will become less molochian. It doesn’t matter why people think this, how the effect is achieved, or if people are aware of this change. Human values exist in us because of evolution, but we may accidentally destroy them with technology, through excessive social competition, or through eugenics/dysgenics.
I don’t think rules make people better. One doesn’t become virtuous because we make it impossible for them to break the law, true virtue is when you have the freedom to do evil but choose not to. This sounds like mere philosophy, but values are in an entirely different category than rules. Value judgements and facts are necessarily unrelated, values cannot be derived, deduced or otherwise calculated, they’re arbitrary and axiomatic. In fact, AGIs cannot have values, they can only act as if they do.
the punishment could be stronger
I probably did not exlain myself well enough. People get away with bad things because there’s loopholes in the law which doesn’t get punished because they’re technically okay. You cannot cover all attack vectors, because you cannot calculate all of them.
If you did manage to find the, say 20 million different attack vectors of a system, then you’d need to defend against 20 million actions. But perhaps 19 million of these actions are already done by perfectly innocent people, for perfectly innocent reasons, and if you start going after those who exploit these vectors, then you will also start harming innocent people who don’t even know of these attack vectors. (example: In some states, collecting rain water is illegal)
Innocent behaviour and malicious behaviour overlaps, with no easy way to discern the two. Then you will either have to punish innocent people, or leave the attack vector open. If you leave open too many attack vectors, exploitation will become the norm and degrade society entirely. If you keep closing the attack vectors, then human freedom will tend towards zero over time, which also means that mutually beneficial actions between individuals will tend towards zero
Coordination is hard. “Assigning Molochian elements a lower value” is a kind of coordination. Making rules, and punishing people when they break them is another. Even if attack is stronger than defense, the punishment could be stronger yet (because it is a kind of attack). I agree that it is difficult, not sure if impossible.
It doesn’t require conscious or direct coordination, but it does require a chain of cause and effect which affects many people. If society agrees that chasing after material goods rather than meaningful pursuits is bad taste, then the world will become less molochian. It doesn’t matter why people think this, how the effect is achieved, or if people are aware of this change. Human values exist in us because of evolution, but we may accidentally destroy them with technology, through excessive social competition, or through eugenics/dysgenics.
I don’t think rules make people better. One doesn’t become virtuous because we make it impossible for them to break the law, true virtue is when you have the freedom to do evil but choose not to. This sounds like mere philosophy, but values are in an entirely different category than rules. Value judgements and facts are necessarily unrelated, values cannot be derived, deduced or otherwise calculated, they’re arbitrary and axiomatic. In fact, AGIs cannot have values, they can only act as if they do.
I probably did not exlain myself well enough. People get away with bad things because there’s loopholes in the law which doesn’t get punished because they’re technically okay. You cannot cover all attack vectors, because you cannot calculate all of them.
If you did manage to find the, say 20 million different attack vectors of a system, then you’d need to defend against 20 million actions. But perhaps 19 million of these actions are already done by perfectly innocent people, for perfectly innocent reasons, and if you start going after those who exploit these vectors, then you will also start harming innocent people who don’t even know of these attack vectors. (example: In some states, collecting rain water is illegal)
Innocent behaviour and malicious behaviour overlaps, with no easy way to discern the two. Then you will either have to punish innocent people, or leave the attack vector open. If you leave open too many attack vectors, exploitation will become the norm and degrade society entirely. If you keep closing the attack vectors, then human freedom will tend towards zero over time, which also means that mutually beneficial actions between individuals will tend towards zero