I haven’t read Roko’s blog, but from the reflection in Eliezer’s opposition I find I somewhat agree.
To the extent that morality is about what you do, the more you can do the higher the stakes.
If you can drive a car, your driving amplifies your ability to do good. And it amplifies your ability to do bad. If you have a morality that leaves you doing more good than bad, and driving increases the good and the bad you do proportionately, then your driving is a good thing.
True human beings have an insatiable curiousity, and they naturally want to find out about things, and one of the things they like is to find out how to do things. Driving a car is a value in itself until you’ve done it enough that it gets boring.
But if you have a sense of good and bad apart from curiousity, then it will probably seem like a good thing for good smart people to get the power to do lots of things, while stupid or evil people should get only powers that are reasonably safe for them to have.
I haven’t read Roko’s blog, but from the reflection in Eliezer’s opposition I find I somewhat agree.
To the extent that morality is about what you do, the more you can do the higher the stakes.
If you can drive a car, your driving amplifies your ability to do good. And it amplifies your ability to do bad. If you have a morality that leaves you doing more good than bad, and driving increases the good and the bad you do proportionately, then your driving is a good thing.
True human beings have an insatiable curiousity, and they naturally want to find out about things, and one of the things they like is to find out how to do things. Driving a car is a value in itself until you’ve done it enough that it gets boring.
But if you have a sense of good and bad apart from curiousity, then it will probably seem like a good thing for good smart people to get the power to do lots of things, while stupid or evil people should get only powers that are reasonably safe for them to have.