To the extent that the “valley” is just the intersection of the curves of the sanity waterline and the capability curve, I think every major new technology has had this concern. Machine guns were too horrific to contemplate in 1910. Nukes were clearly more power than humans were ready for.
I wouldn’t argue that it’s not true or that the valley doesn’t exist. I’d argue that it isn’t a crux for any decision one can make. The people making the tools are different from the people who are worried about it, and stopping them isn’t possible. All we can do is to try to raise the sanity waterline fast enough that we can handle it when they succeed.
Honestly, I still suspect we’ll destroy ourselves through conventional means before uncontrollable super-powered AI does it. It may well be tool-AI-enhanced human stupidity, but it’ll still be humans at the root of the decision tree.
To the extent that the “valley” is just the intersection of the curves of the sanity waterline and the capability curve, I think every major new technology has had this concern. Machine guns were too horrific to contemplate in 1910. Nukes were clearly more power than humans were ready for.
I wouldn’t argue that it’s not true or that the valley doesn’t exist. I’d argue that it isn’t a crux for any decision one can make. The people making the tools are different from the people who are worried about it, and stopping them isn’t possible. All we can do is to try to raise the sanity waterline fast enough that we can handle it when they succeed.
Honestly, I still suspect we’ll destroy ourselves through conventional means before uncontrollable super-powered AI does it. It may well be tool-AI-enhanced human stupidity, but it’ll still be humans at the root of the decision tree.