On repugnance, I think I’ve been thinking too much in terms of human minds only. In that case there really doesn’t seem to be a practical problem: certainly if C is now, continuous improvements might get us to a repugnant A—but my point is that that path wouldn’t be anywhere close to optimal. Total-ut prefers A to C, but there’d be a vast range of preferable options every step of the way—so it’d always end up steering towards some other X rather than anything like A.
I think that’s true if we restrict to human minds (the resource costs of running a barely content one being a similar order of magnitude to those of running a happy one).
But of course you’re right as soon as we’re talking about e.g. rats (or AI-designed molecular scale minds...). I can easily conceive of metrics valuing 50 happy rats over 1 happy human. I don’t think rat-world fits most people’s idea of utopia.
I think that’s the style of repugnance that’d be a practical danger: vast amounts of happy-but-simple minds.
Thanks. I’ll check out the infinite idea.
On repugnance, I think I’ve been thinking too much in terms of human minds only. In that case there really doesn’t seem to be a practical problem: certainly if C is now, continuous improvements might get us to a repugnant A—but my point is that that path wouldn’t be anywhere close to optimal. Total-ut prefers A to C, but there’d be a vast range of preferable options every step of the way—so it’d always end up steering towards some other X rather than anything like A.
I think that’s true if we restrict to human minds (the resource costs of running a barely content one being a similar order of magnitude to those of running a happy one).
But of course you’re right as soon as we’re talking about e.g. rats (or AI-designed molecular scale minds...). I can easily conceive of metrics valuing 50 happy rats over 1 happy human. I don’t think rat-world fits most people’s idea of utopia.
I think that’s the style of repugnance that’d be a practical danger: vast amounts of happy-but-simple minds.
Yep, that does seem a risk. I think that’s what the “muzak and potatoes” formulation of repugnance is about.