I’m not following your final point. Regardless of determinism, the “state space” I can explore as an embedded agent is constrained by the properties of the local environment. If I value things like a walkable neighborhood, but I’m stuck in a pile of rubble, that’s going to constrain my available state space and accordingly it’s going to constrain my ability to have any rewarding outcome. McTraffic, by not allotting freedoms to me when executing their transportation redesign impeded on my freedom (which was mostly afforded to me through my and my neighbors property rights).
Freedoms (properly encoded), I believe are the proper framing for creating utility functions/value-systems for critters like our friendly neighborhood traffic agent. Sure, the traffic agent values transportation efficiency, but since it also values other agent’s freedom to property rights, they will limit their execution of their traffic efficiency preferences within a multi-agent shared environment to minimize the restriction to property rights. To me, this seems simpler, and less error prone than any approach that tries to infer my values (or human preferences more generally) and act according to that inference.
Freedoms assume awareness of external (embedded) agency, they are values you afford to other agents. They have a payoff because you are then afforded them back. This helps to ensure agents do not unilaterally bulldoze (literally or figuratively) the “available state space” for other agents to explore and exploit.
Hmm, Looks like I should add an examples section and more background on what I mean related to freedom. What you are describing sounds like a traffic system that values ergodic efficiency of it’s managed network and you are showing a way that a participant can have very non-ergodic results. It sounds like that is more of an engineering problem than what I’m imagining.
Examples off the top of my head of what I mean with respect to loss of freedom resulting from a powerful agent’s value system include things like:
paperclip maximizer terraforming the earth prevents any value-systems other than paperclip maximization from sharing the earth’s environment.
human’s value for cheap foodstuffs results in mono-culture crop fields, which cuts off forest grassland ecosystem’s values, (hiding places, alternating food stuffs which last through the seasons, etc.)
Drug dependent parent changes a child’s environment, preventing freedom for a reliable schedule, security, etc.
Or, riffing off your example: superintelligent traffic controller starts city-planning, bulldozing blocks of car-free neighborhoods because they stood in the way of a 5% city-wide traffic flow improvement
Essentially what I’m trying to describe is that freedoms need to be a value onto themselves that has certain characteristics that are functionally different than the common utility function terminology that revolves around metric maximization (like gradient descent). Freedoms describe boundary conditions within which metric maximization is allowed, but describe steep penalties for surpassing their bounds. Their general mathematical form is a manifold surrounding some state-space, whereas it seems the general form of most utility function talk is finding a minima/maxima of some state space.