The main crux I see about this take on behavior is whether it’s even possible or tractable to deconfuse and formalize the range of behaviors of goal-directed systems.
It seems like a goal-directed system could decide to [temporarily] enact any behavior (that it was physically capable of), for example for the purpose of deception. How does this square with your current thoughts?
Here are some responses I can imagine:
Any behavior is possible, but some behaviors are far more likely than others, and probabilistic characterizations of the behavior of goal-directed systems would still be very useful
Computational resource limits put a bound on how much a system can pretend to be another system
If we can structurally detect non-deception, then we can get much stronger constraints on the system’s behavior
Even though it feels intuitively like a sufficiently intelligent system could decide to enact any behavior, it is in fact a specific system that will necessarily have a more limited range of behaviors, and we can somehow figure out exactly which behaviors it can and can’t enact
My first reaction is that for the system to accomplish its goal, it must eventually behave goal-directedly. It’s easy to imagine an AI accomplishing goal X by pursuing goal Y (for example building a robot to do X), but it’s hard to imagine the AI accomplishing goal X by not accomplishing any goal.
As for your proposed answers:
I guess this is a probabilistic take on my argument that it will eventually need to be goal-directed to do things
I would say more “constraints on how well the actual goal must be accomplished”
That’s true, but I’m not sure yet that we can detect deception.
It seems like a goal-directed system could decide to [temporarily] enact any behavior (that it was physically capable of), for example for the purpose of deception. How does this square with your current thoughts?
Here are some responses I can imagine:
Any behavior is possible, but some behaviors are far more likely than others, and probabilistic characterizations of the behavior of goal-directed systems would still be very useful
Computational resource limits put a bound on how much a system can pretend to be another system
If we can structurally detect non-deception, then we can get much stronger constraints on the system’s behavior
Even though it feels intuitively like a sufficiently intelligent system could decide to enact any behavior, it is in fact a specific system that will necessarily have a more limited range of behaviors, and we can somehow figure out exactly which behaviors it can and can’t enact
Interesting points!
My first reaction is that for the system to accomplish its goal, it must eventually behave goal-directedly. It’s easy to imagine an AI accomplishing goal X by pursuing goal Y (for example building a robot to do X), but it’s hard to imagine the AI accomplishing goal X by not accomplishing any goal.
As for your proposed answers:
I guess this is a probabilistic take on my argument that it will eventually need to be goal-directed to do things
I would say more “constraints on how well the actual goal must be accomplished”
That’s true, but I’m not sure yet that we can detect deception.
Basically my intuition.