I agree with this. It’s super undesirable. On the other hand, so are wars and famines and what have you. Tradeoffs exist.
Think of it like the financial system. Some people are going for a high score in the money economy, and that powers both good and bad things. If we built coordination markets, some people would hyperfixate on them in a very unhealthy way, become fabulously wealthy in reputation terms, and then be exposed as child molestors or what have you. Again, tradeoffs exist.
oh, so this is a temporary before-AI-inevitably-either-kills-everyone-or-solves-everything thing, not a plan for making the AI-that-solves-everything-including-X-risk?
It’s an adjunct to the AI that solves everything, maybe? It can coexist with everything else in human society, and I would argue that it will improve those things along all the axes that any of have a right to care about.
And like, the only way you can get people to stop building the AI that’s gonna kill everyone is some sort of massive labor strike against the companies building that stuff. Another enormous coordination problem—it’s not in any one capabilities researcher’s self-interest to stop the train, but if the train doesn’t slow down, then we all die.
I agree with this. It’s super undesirable. On the other hand, so are wars and famines and what have you. Tradeoffs exist.
Think of it like the financial system. Some people are going for a high score in the money economy, and that powers both good and bad things. If we built coordination markets, some people would hyperfixate on them in a very unhealthy way, become fabulously wealthy in reputation terms, and then be exposed as child molestors or what have you. Again, tradeoffs exist.
oh, so this is a temporary before-AI-inevitably-either-kills-everyone-or-solves-everything thing, not a plan for making the AI-that-solves-everything-including-X-risk?
It’s an adjunct to the AI that solves everything, maybe? It can coexist with everything else in human society, and I would argue that it will improve those things along all the axes that any of have a right to care about.
And like, the only way you can get people to stop building the AI that’s gonna kill everyone is some sort of massive labor strike against the companies building that stuff. Another enormous coordination problem—it’s not in any one capabilities researcher’s self-interest to stop the train, but if the train doesn’t slow down, then we all die.