However, a combination of explicit modularisation and hierarchical breakdown of “agents”, loss functions and inductive biases that promote sparsity in the DNN, representation engineering and alignment, and autoencoder interpretability may together cook up a “stone soup” story of agent interpretability. So, it doesn’t seem like a notable distinction between “agents” and “agencies”, as you point in the post as well.
Current reality is way more messy, but you can already recognize people intuitively fear some of these outcomes. Extrapolation of calls for treaties, international regulatory bodies, and government involvement is ‘we need security services to protect humans’. A steelman of some of the ‘we need freely distributed AIs to avoid concentration of power’ claims is ‘we fear the dictatorship failure mode’.
I think there is just no way around establishing global civilisational coherence if people want to preserve some freedoms (and not be dead), anyway.
I pondered in the comments to Drexler’s 2022 post about the Open Agency Model what the difference between an “agency” and an “agent” really comes down to. Drexler (as well as later Conjecture in their CoEm proposal) emphasised interpretability.
However, a combination of explicit modularisation and hierarchical breakdown of “agents”, loss functions and inductive biases that promote sparsity in the DNN, representation engineering and alignment, and autoencoder interpretability may together cook up a “stone soup” story of agent interpretability. So, it doesn’t seem like a notable distinction between “agents” and “agencies”, as you point in the post as well.
I think there is just no way around establishing global civilisational coherence if people want to preserve some freedoms (and not be dead), anyway.
This prompted me to write “Open Agency model can solve the AI regulation dilemma”.