“Exercise 7. How can you discover someone’s goals? Assume you either cannot ask them, or would not trust their answers.”
I’d guess that the best way is to observe what they actually do and figure out what goal they might be working towards from that.
That has the unfortunate consequence of automatically assuming that they’re effective at reaching their goal, though. So you can’t really use a goal that you’ve figured out in this way to estimate how good an agent is at getting to its goals.
And it has the unfortunate side effect of ascribing ‘goals’ to systems that are way too simple for that to be meaningful. You might as well say that the universe has a “goal” of maximizing its entropy. I’m not sure that it’s meaningful to ascribe a “goal” to a thermostat—while it’s a convenient way of describing what it does (“it wants to keep the temperature constant, that’s all you need to know about it”), in a community of people who talk about AI I think it would require a bit more mental machinery before it could be said to have “goals”.
“Exercise 7. How can you discover someone’s goals? Assume you either cannot ask them, or would not trust their answers.”
I’d guess that the best way is to observe what they actually do and figure out what goal they might be working towards from that.
That has the unfortunate consequence of automatically assuming that they’re effective at reaching their goal, though. So you can’t really use a goal that you’ve figured out in this way to estimate how good an agent is at getting to its goals.
And it has the unfortunate side effect of ascribing ‘goals’ to systems that are way too simple for that to be meaningful. You might as well say that the universe has a “goal” of maximizing its entropy. I’m not sure that it’s meaningful to ascribe a “goal” to a thermostat—while it’s a convenient way of describing what it does (“it wants to keep the temperature constant, that’s all you need to know about it”), in a community of people who talk about AI I think it would require a bit more mental machinery before it could be said to have “goals”.