I feel like the most likely implementation, given human nature, would be a castrated genie. The genie gives negative weight to common destructive problems. Living things moving at high speeds or being exposed to dangerous levels of heat are bad. No living things falling long distances. If such things are unavoidable, then it may just refuse to operate, avoiding complicity even at the cost of seeing someone dead who might have lived. Most wishes fizzle. But wishing is, at least, seen as a not harmful activity. Lowest common denominator values and ‘thin skull’ type standards are not ideal from a utilitarian standpoint. But they facilitate write-once run-anywhere solutions and mass marketing.
I’m guessing that the outcome pump is stateless, which makes calculating expected values a bit harder. But if the machine can fail, that potentially implies some kind of state, unless the failure rate is perfectly consistent from one reset to another in which case the pumps might need to work in groups of two or three to prevent their own failure. Failure of one pump would be a reset condition. Failure of three pumps simultaneously would be unlikely, at least due to any internal issues. External threats which take out all three could be a different story, especially if they were close together. (But do they need to be?)
Would 10 pumps located at various points around the world, each protecting the other with their utility function, invite a planet killer astronomical threat?
I feel like the most likely implementation, given human nature, would be a castrated genie. The genie gives negative weight to common destructive problems. Living things moving at high speeds or being exposed to dangerous levels of heat are bad. No living things falling long distances. If such things are unavoidable, then it may just refuse to operate, avoiding complicity even at the cost of seeing someone dead who might have lived. Most wishes fizzle. But wishing is, at least, seen as a not harmful activity. Lowest common denominator values and ‘thin skull’ type standards are not ideal from a utilitarian standpoint. But they facilitate write-once run-anywhere solutions and mass marketing.
I’m guessing that the outcome pump is stateless, which makes calculating expected values a bit harder. But if the machine can fail, that potentially implies some kind of state, unless the failure rate is perfectly consistent from one reset to another in which case the pumps might need to work in groups of two or three to prevent their own failure. Failure of one pump would be a reset condition. Failure of three pumps simultaneously would be unlikely, at least due to any internal issues. External threats which take out all three could be a different story, especially if they were close together. (But do they need to be?)
Would 10 pumps located at various points around the world, each protecting the other with their utility function, invite a planet killer astronomical threat?