I am assuming that the agents of the ethical system are self-interested, and that requires them to have identities. This assumption isn’t necessarily true, but it covers a large enough space of possible worlds to be worth considering. It’s also a part of the space that is easier for us to understand than its complement.
Not sure where the “killing sentient organisms” comes in. That also introduces a set of assumptions. When, at work, I send 500,000,000,000 BLAST jobs out to the computing grid, to run on 700 computers comprising 2800 CPUs, with 5,000 different outer-loop starting points, how many organisms am I killing when I end the run?
I am not sure you got my point. I’ll try again. To efficiently search a space, you need some variation in the trials that are performed. However, that variation does not necessarily need to be embodied in the genomes of intelligent agents. It could be in the form of variations in lab experiments performed. Progress today does not depend on genetic variation between humans. It depends on memetic variation—and the memes are usually not embodied as agents that are conscious or do much cooperating. As far as I can tell, if you understand this, your original questions seem to fall apart.
I am assuming that the agents of the ethical system are self-interested, and that requires them to have identities. This assumption isn’t necessarily true, but it covers a large enough space of possible worlds to be worth considering. It’s also a part of the space that is easier for us to understand than its complement.
Not sure where the “killing sentient organisms” comes in. That also introduces a set of assumptions. When, at work, I send 500,000,000,000 BLAST jobs out to the computing grid, to run on 700 computers comprising 2800 CPUs, with 5,000 different outer-loop starting points, how many organisms am I killing when I end the run?
I am not sure you got my point. I’ll try again. To efficiently search a space, you need some variation in the trials that are performed. However, that variation does not necessarily need to be embodied in the genomes of intelligent agents. It could be in the form of variations in lab experiments performed. Progress today does not depend on genetic variation between humans. It depends on memetic variation—and the memes are usually not embodied as agents that are conscious or do much cooperating. As far as I can tell, if you understand this, your original questions seem to fall apart.