What really struck me with this parable is that it’s so well-written that I felt genuine horror and revulsion at the idea of an AI making heaps of size 8. Because, well… 2!
So, aside from the question of whether an AI would come to moral conclusions such as “heaps of size 8 are okay” or “the way to end human suffering is to end human life”, the question I’m taking away from this parable is, are we any more enlightened than the Pebblesorters? Should we, in fact, be sending philosophers or missionaries to the Pebblesorter planet to explain to them that it’s wrong to murder someone just because they built a heap of size 15?
Why not keep it in the form of a pile of 13 and a pile of 2, until you find the other two pebbles you’re looking for? That would be the ETHICALLY RESPONSIBLE thing to do!
What really struck me with this parable is that it’s so well-written that I felt genuine horror and revulsion at the idea of an AI making heaps of size 8. Because, well… 2!
So, aside from the question of whether an AI would come to moral conclusions such as “heaps of size 8 are okay” or “the way to end human suffering is to end human life”, the question I’m taking away from this parable is, are we any more enlightened than the Pebblesorters? Should we, in fact, be sending philosophers or missionaries to the Pebblesorter planet to explain to them that it’s wrong to murder someone just because they built a heap of size 15?
Maybe they just hadn’t finished it yet…
Why not keep it in the form of a pile of 13 and a pile of 2, until you find the other two pebbles you’re looking for? That would be the ETHICALLY RESPONSIBLE thing to do!