Related, just seen today: The curse of smart people. SPOILER: “an ability to convincingly rationalize nearly anything.”
The AI box experiment seems to support this. People who have been persuaded that it would be irrational to let an unfriendly AI out of the box are being persuaded to let it out of the box.
The ability of smarter or more knowledgeable people to convince less intelligent or less educated people of falsehoods (e.g. parents and children) shows that we need to put less weight on arguments and more weight on falsifiability.
I wouldn’t use the Ai box experiment as an example for anything because it is specifically designed to be a black box: It’s exciting precisely because the outcome confuses the heck out of people. I’m having trouble parsing this in Bayesian terms but I think you’re committing a rationalist sin by using an event that your model of reality couldn’t predict in advance as evidence that your model of reality is correct.
I strongly agree that we need to put less weight on arguments but I think falsifiability is impractical in everyday situations.
The AI box experiment seems to support this. People who have been persuaded that it would be irrational to let an unfriendly AI out of the box are being persuaded to let it out of the box.
The ability of smarter or more knowledgeable people to convince less intelligent or less educated people of falsehoods (e.g. parents and children) shows that we need to put less weight on arguments and more weight on falsifiability.
I wouldn’t use the Ai box experiment as an example for anything because it is specifically designed to be a black box: It’s exciting precisely because the outcome confuses the heck out of people. I’m having trouble parsing this in Bayesian terms but I think you’re committing a rationalist sin by using an event that your model of reality couldn’t predict in advance as evidence that your model of reality is correct.
I strongly agree that we need to put less weight on arguments but I think falsifiability is impractical in everyday situations.