It seems strange for an agent to take another agent and a situation and return a choice.
I also think this approach matches our intuition about how counterfactuals work. We imagine ourselves as the same except we’re choosing this particular behavior. Surely, in the formal reasoning, there might also be a distinction between the initial agent and the agent within that counterfactual, considering it’s present in our own imaginations?
The Agent needs access to a self pointer, and it is parameterized so it doesn’t have to be a static pointer, as it was in the original paper—this approach in particular needs it to be dynamic in this way.
There are also use cases where a bit of code receives a pointer not to its exact self—when it is called as a subagent, it will get the parent’s pointer.
I don’t suppose you could clarify:
It seems strange for an agent to take another agent and a situation and return a choice.
Yeah, this is essentially my position as well. My most recent attempt at articulating this is Why 1-boxing doesn’t imply backwards causation.
The Agent needs access to a self pointer, and it is parameterized so it doesn’t have to be a static pointer, as it was in the original paper—this approach in particular needs it to be dynamic in this way.
There are also use cases where a bit of code receives a pointer not to its exact self—when it is called as a subagent, it will get the parent’s pointer.