Well, your grandparent’s comment spoke of a problem with adding randomness, rather than with lack of necessity to add randomness. Maybe i simply misunderstood it.
Note btw that the self description will have to include randomness, to be an accurate description of an imperfect agent.
Let me try to explain. Assume you have an agent that’s very similar to our proof-theoretic ones, but also has a small chance of returning a random action. Moreover, the agent’s self-description includes a mention of the random variable V that makes the agent return a random action.
The first problem is that statements like “A()==a” are no longer logical statements. That seems easy to fix by making the agent look at conditional expected values instead of logical implications.
The second and more serious problem is this: how many independent copies of V does the world contain? Imagine the agent is faced with Newcomb’s Problem. If the world program contains only one copy of V that’s referenced by both instances of the agent, that amounts to abusing the problem formulation to tell the agent “your copies are located here and here”. And if the two copies of the agent use uncorrelated instances of V, then looking at conditional expected values based on V buys you nothing. Figuring out the best action reduces to same logical question that the proof-theoretic algorithms are faced with.
Well, your grandparent’s comment spoke of a problem with adding randomness, rather than with lack of necessity to add randomness. Maybe i simply misunderstood it.
Note btw that the self description will have to include randomness, to be an accurate description of an imperfect agent.
Let me try to explain. Assume you have an agent that’s very similar to our proof-theoretic ones, but also has a small chance of returning a random action. Moreover, the agent’s self-description includes a mention of the random variable V that makes the agent return a random action.
The first problem is that statements like “A()==a” are no longer logical statements. That seems easy to fix by making the agent look at conditional expected values instead of logical implications.
The second and more serious problem is this: how many independent copies of V does the world contain? Imagine the agent is faced with Newcomb’s Problem. If the world program contains only one copy of V that’s referenced by both instances of the agent, that amounts to abusing the problem formulation to tell the agent “your copies are located here and here”. And if the two copies of the agent use uncorrelated instances of V, then looking at conditional expected values based on V buys you nothing. Figuring out the best action reduces to same logical question that the proof-theoretic algorithms are faced with.