I think that the concept of “agency” (although maybe “intelligence” would be a better word?), in the context of AI alignment, implies the ability to learn the environment and exploit this knowledge towards a certain goal. The only way to pursue a goal effectively without learning is having hard-coded knowledge of the environment. But, where would this knowledge come from? For complex environments, it is only likely to come from learning algorithms upstream.
So, a rock is definitely not an agent since there is nothing it learns about its environment (I am not even sure what the input/output channels of a rock are supposed to be). Q-learning is an agent, but the resulting policy is not an agent in itself. Similarly, AlphaGo is a sort of agent when regarded together with the training loop (it can in principle learn to play different games), but not when disconnected from it. Evolution is an agent, even if not a very powerful one. An ant colony is probably a little agentic because it can learn something, although I’m not sure how much.
Observations inspired by your comment: While this shouldn’t necessarily be so, it seems the particular formulations make a lot of difference when it comes to exchanging ideas. If I read your comment without the
(although maybe “intelligence” would be a better word?)
bracket, I immediatelly go “aaa, this is so wrong!”. And if I substitute “intelligent” for “agent”, I totally agree with it. Not sure whether this is just me, or whether it generalizes to other people.
More specifically, I agree that from the different concepts in the vicinity of “agency”, “the ability to learn the environment and exploit this knowledge towards a certain goal” seems to be particularly important to AI alignment. I think the word “agency” is perhaps not well suited for this particular concept, since it comes with so many other connotations. But “intelligence” seems quite right.
I am not even sure what the input/output channels of a rock are supposed to be
I guess you imagine that the input is the physical forces affecting the ball and the output is the forces the ball exerts on the environment. Obviously, this is very much not useful for anything. But it suddenly becomes non-trivial if you consider something like the billiard-ball computer (seems like a theoretical construct, not sure if anybody actually built one...but it seems like a relevant example anyway).
I think that the concept of “agency” (although maybe “intelligence” would be a better word?), in the context of AI alignment, implies the ability to learn the environment and exploit this knowledge towards a certain goal. The only way to pursue a goal effectively without learning is having hard-coded knowledge of the environment. But, where would this knowledge come from? For complex environments, it is only likely to come from learning algorithms upstream.
So, a rock is definitely not an agent since there is nothing it learns about its environment (I am not even sure what the input/output channels of a rock are supposed to be). Q-learning is an agent, but the resulting policy is not an agent in itself. Similarly, AlphaGo is a sort of agent when regarded together with the training loop (it can in principle learn to play different games), but not when disconnected from it. Evolution is an agent, even if not a very powerful one. An ant colony is probably a little agentic because it can learn something, although I’m not sure how much.
Yep, that totally makes sense.
Observations inspired by your comment: While this shouldn’t necessarily be so, it seems the particular formulations make a lot of difference when it comes to exchanging ideas. If I read your comment without the
bracket, I immediatelly go “aaa, this is so wrong!”. And if I substitute “intelligent” for “agent”, I totally agree with it. Not sure whether this is just me, or whether it generalizes to other people.
More specifically, I agree that from the different concepts in the vicinity of “agency”, “the ability to learn the environment and exploit this knowledge towards a certain goal” seems to be particularly important to AI alignment. I think the word “agency” is perhaps not well suited for this particular concept, since it comes with so many other connotations. But “intelligence” seems quite right.
I guess you imagine that the input is the physical forces affecting the ball and the output is the forces the ball exerts on the environment. Obviously, this is very much not useful for anything. But it suddenly becomes non-trivial if you consider something like the billiard-ball computer (seems like a theoretical construct, not sure if anybody actually built one...but it seems like a relevant example anyway).