I like this a lot. I think it’s going to be really important when analyzing a created agent to compare the style/extent of its wanting to human wanting. I expect we will still to create something that has a limited subset of the wanting expressed by humans. I don’t think enough thought has yet gone into analyzing what aspects of wanting are expressed by current RL agents, and how we could measure that objectively.
I like this a lot. I think it’s going to be really important when analyzing a created agent to compare the style/extent of its wanting to human wanting. I expect we will still to create something that has a limited subset of the wanting expressed by humans. I don’t think enough thought has yet gone into analyzing what aspects of wanting are expressed by current RL agents, and how we could measure that objectively.