For example, in Lockean property rights theory, “who is the rightful owner of a house” is a path-dependent question.
Ah, I think this is a fundamentally different kind of abstraction than the “aggregating together distinct things with similar effects” type of abstraction I mentioned. To distinguish, I suggest we use the name “causal abstraction” for the kind I mentioned, and the name “protocol abstraction” (or something else) for this concept. So:
Causal abstraction: aggregating together distinct phenomena that have similar causal relations into a lumpy concept that can be modelled as having the same causal relations as its constituents
Protocol abstraction: extending your ontology with new “epiphenomenal” variables that follow certain made-up rules (primarily for the use in social coordination, so that there is a ground truth even with deception? - but can also be used on an individual level, in values)
It’s analogous to how an idealized version of personal identity might require a continuous stream of gradually changing agents and so on, but in practice all we have to go on is what memories people have about how things used to be.
I feel like personal identity has both elements of causal abstraction and of protocol abstraction. E.g. social relationships like debts seem to be strongly tied to protocol abstraction, but there’s also lots of social behavior that only relies on causal abstraction.
If you then train an AI to understand the ownership relation and it learns the relation that we have actually implemented rather than the idealized version we have in mind, it can think that what we really care about is who is “recorded” as the owner of a house in the current physical state rather than who is “legitimately” the owner of the house, and in the extreme cases that can lead it to take some bizarre actions when you ask it to optimize something that has to do with the concept of property rights.
I agree.
Coming up with a normative theory of agency in the case of protocol abstraction actually sounds like a fairly important task. I have some ideas about how to address causal abstraction, but I haven’t really thought much about protocol abstraction before.
I think your distinction between causal and protocol abstractions makes sense and it’s related to my distinction between causally relevant vs causally irrelevant latent variables. It’s not quite the same, because abstractions which are rendered causally irrelevant in some world model can still be causal in the sense of aggregating together a bunch of things with similar causal properties.
I feel like personal identity has both elements of causal abstraction and of protocol abstraction. E.g. social relationships like debts seem to be strongly tied to protocol abstraction, but there’s also lots of social behavior that only relies on causal abstraction.
I agree.
Coming up with a normative theory of agency in the case of protocol abstraction actually sounds like a fairly important task. I have some ideas about how to address causal abstraction, but I haven’t really thought much about protocol abstraction before.
Can you clarify what you mean by a “normative theory of agency”? I don’t think I’ve ever seen this phrase before.
Can you clarify what you mean by a “normative theory of agency”? I don’t think I’ve ever seen this phrase before.
What I mean is stuff like decision theory/selection theorems/rationality; studies of what kinds of ways agents normatively should act.
Usually such theories do not take abstractions into account. I have some ideas for how to take causal abstractions into account, but I don’t think I’ve seen protocol abstractions investigated much.
In a sense, they could technically be handled by just having utility functions over universe trajectories rather than universe states, but there are some things about this that seem unnatural (e.g. for the purpose of Alex Turner’s power-seeking theorems, utility functions over trajectories may be extraordinarily power-seeking, and so if we could find a narrower class of utility functions, that would be useful).
Ah, I think this is a fundamentally different kind of abstraction than the “aggregating together distinct things with similar effects” type of abstraction I mentioned. To distinguish, I suggest we use the name “causal abstraction” for the kind I mentioned, and the name “protocol abstraction” (or something else) for this concept. So:
Causal abstraction: aggregating together distinct phenomena that have similar causal relations into a lumpy concept that can be modelled as having the same causal relations as its constituents
Protocol abstraction: extending your ontology with new “epiphenomenal” variables that follow certain made-up rules (primarily for the use in social coordination, so that there is a ground truth even with deception? - but can also be used on an individual level, in values)
I feel like personal identity has both elements of causal abstraction and of protocol abstraction. E.g. social relationships like debts seem to be strongly tied to protocol abstraction, but there’s also lots of social behavior that only relies on causal abstraction.
I agree.
Coming up with a normative theory of agency in the case of protocol abstraction actually sounds like a fairly important task. I have some ideas about how to address causal abstraction, but I haven’t really thought much about protocol abstraction before.
I think your distinction between causal and protocol abstractions makes sense and it’s related to my distinction between causally relevant vs causally irrelevant latent variables. It’s not quite the same, because abstractions which are rendered causally irrelevant in some world model can still be causal in the sense of aggregating together a bunch of things with similar causal properties.
I agree.
Can you clarify what you mean by a “normative theory of agency”? I don’t think I’ve ever seen this phrase before.
What I mean is stuff like decision theory/selection theorems/rationality; studies of what kinds of ways agents normatively should act.
Usually such theories do not take abstractions into account. I have some ideas for how to take causal abstractions into account, but I don’t think I’ve seen protocol abstractions investigated much.
In a sense, they could technically be handled by just having utility functions over universe trajectories rather than universe states, but there are some things about this that seem unnatural (e.g. for the purpose of Alex Turner’s power-seeking theorems, utility functions over trajectories may be extraordinarily power-seeking, and so if we could find a narrower class of utility functions, that would be useful).