So it has a current utility of (1-ε)10, and can increase this by reducing ε - hence by building even more paperclips.
I take ε to be the probability that something weird is happening like you’re hallucinating your paperclips. Why would building more paperclips reduce ε? If you are dreaming, you’re just making more dream paperclips.
I’m sure you’d spend your time with trying to find increasingly elaborate ways to probe for bugs in Descartes’ demon’s simulation. It is not clear to me why your increasingly paranoid bug probes would involve making paperclips.
I agree that making more paperclips does not reduce ε, but an unsure AI might build more paperclips nonetheless.
If x is the probability that any one paperclip is hallucinated, the AI will never be certain it has created 10 paperclips (or any for that matter) as long as x > 0. But it can increase the probability it has created 10 paperclips by making 3^^^3 of them.
Bug probes may be a more efficient way to increase the probability, but that isn’t certain.
It is not clear to me why your increasingly paranoid bug probes would involve making paperclips.
It need not. The problem occurs for any measure that burns resources (and probing the universe for bugs in the Descartes demon would be spectacular at burning resources).
I take ε to be the probability that something weird is happening like you’re hallucinating your paperclips. Why would building more paperclips reduce ε? If you are dreaming, you’re just making more dream paperclips.
I’m sure you’d spend your time with trying to find increasingly elaborate ways to probe for bugs in Descartes’ demon’s simulation. It is not clear to me why your increasingly paranoid bug probes would involve making paperclips.
I agree that making more paperclips does not reduce ε, but an unsure AI might build more paperclips nonetheless.
If x is the probability that any one paperclip is hallucinated, the AI will never be certain it has created 10 paperclips (or any for that matter) as long as x > 0. But it can increase the probability it has created 10 paperclips by making 3^^^3 of them.
Bug probes may be a more efficient way to increase the probability, but that isn’t certain.
It need not. The problem occurs for any measure that burns resources (and probing the universe for bugs in the Descartes demon would be spectacular at burning resources).