(And, trust me, brains have found a whole lot of the bad ones. What do you expect, when you run programs that screwed themselves into existence on computers made of meat?)
Fairly flippant for what’s supposed to be a primer.
The rationality of Rationality: AI to Zombies isn’t about using cold logic to choose what to care about. Reasoning well has little to do with what you’re reasoning towards. If your goal is to enjoy life to the fullest and love without restraint, then better reasoning (while hot or cold, while rushed or relaxed) will help you do so. But if your goal is to annihilate as many puppies as possible, then this-kind-of-rationality will also help you annihilate more puppies.
This seems like bad salesmanship. There’s no need to say things like “rationality is compatible with puppy annihilation” in a paragraph that’s supposed to be reassuring people rationality is not immoral or hostile to human emotions. Instead, I would say you should to the opposite, and tell people that if they really love someone, that means that it is actually a rational thing to want to make sacrifices for them, because if you weigh everything up their happiness matters more to you than their own.
So, whereas the message of this post can be summarized as “rationality is neutral”, I think the post’s message should be closer to “rationality is good—it lets you do things like love EVEN BETTER”.
Thanks! These are good suggestions. I intended for these essays to be geared more towards old-friends-who-are-intrigued-by-this-rationality-business, and didn’t intend it to be preface-for-the-book-type-material. I agree that skewing more towards your message would be the right thing to do in the latter context, though :-)
I actually like that line. There are a lot of people and organizations that are portrayed as rational and evil. Walmart sacrificing all soft values to maximize profit and the robot overlords systematically destroying or enslaving humanity are also views of rationality. They can be used as objections as much as Spock can. This quick joke shows that problems like this are considered, even if they aren’t dealt with in depth here.
Part of it might just be the order. Compare that paragraph to the following alternative:
The rationality of Rationality: AI to Zombies isn’t about using cold logic to choose what to care about. Reasoning well has little to do with what you’re reasoning towards. If your goal is to annihilate as many puppies as possible, then this kind of rationality will help you annihilate more puppies. But if your goal is to enjoy life to the fullest and love without restraint, then better reasoning (while hot or cold, while rushed or relaxed) will also help you do so.
Fairly flippant for what’s supposed to be a primer.
This seems like bad salesmanship. There’s no need to say things like “rationality is compatible with puppy annihilation” in a paragraph that’s supposed to be reassuring people rationality is not immoral or hostile to human emotions. Instead, I would say you should to the opposite, and tell people that if they really love someone, that means that it is actually a rational thing to want to make sacrifices for them, because if you weigh everything up their happiness matters more to you than their own.
So, whereas the message of this post can be summarized as “rationality is neutral”, I think the post’s message should be closer to “rationality is good—it lets you do things like love EVEN BETTER”.
Thanks! These are good suggestions. I intended for these essays to be geared more towards old-friends-who-are-intrigued-by-this-rationality-business, and didn’t intend it to be preface-for-the-book-type-material. I agree that skewing more towards your message would be the right thing to do in the latter context, though :-)
Okay, gotcha.
I would find the latter more useful, if I was going to send other people these links.
These are good essays. Thanks!
I actually like that line. There are a lot of people and organizations that are portrayed as rational and evil. Walmart sacrificing all soft values to maximize profit and the robot overlords systematically destroying or enslaving humanity are also views of rationality. They can be used as objections as much as Spock can. This quick joke shows that problems like this are considered, even if they aren’t dealt with in depth here.
Part of it might just be the order. Compare that paragraph to the following alternative: