I have a hobby of doing and reading personality research. One time recently an alignment researcher asked me whether an AI would soon be able to reach my level in personality research. My answer was no, and I still think it will be no for a new scaled-up GATO.
Like as some small-scale measureable thing? No, if we could do that then we could probably also make an AI that was better than me just be optimizing that metric.
I’ve been thinking about this question for a while today, and I’ve had a hard time coming up with a good concrete answer, even though I feel quite confident in my assertion. My best bet for an answer is that I just haven’t seen a technically legible reason that they might be able to do it.
But that’s not a very satisfying argument, so I’ll try to give an explanation. Naive personality research is super duper easy and I bet you could already set up a system so GPT-3 could do it, as long as you are willing to pay it a bunch of money for participants. A lot of personality research can be done just by word-associations, which GPT-3 is good at.
The trouble comes when you want to do anything deeper. If you want to know how personality actually works, you need to think of many levels at once; how people behave, how that behavior translates into your measurement method, etc.. This is not simply an extrapolation of the word-association game, and I have not seen any AI methods that are likely capable of that yet. The situation is further worsened by the fact that there is a lot of text on the internet that describes naive methods, so it would likely jump on that instead.
I have a hobby of doing and reading personality research. One time recently an alignment researcher asked me whether an AI would soon be able to reach my level in personality research. My answer was no, and I still think it will be no for a new scaled-up GATO.
Do you think there’s a metric that could be used whether or not the AI is better or worse than you? If so what would be the metric?
Like as some small-scale measureable thing? No, if we could do that then we could probably also make an AI that was better than me just be optimizing that metric.
Curious if you have some technically legible reason for this.
I’ve been thinking about this question for a while today, and I’ve had a hard time coming up with a good concrete answer, even though I feel quite confident in my assertion. My best bet for an answer is that I just haven’t seen a technically legible reason that they might be able to do it.
But that’s not a very satisfying argument, so I’ll try to give an explanation. Naive personality research is super duper easy and I bet you could already set up a system so GPT-3 could do it, as long as you are willing to pay it a bunch of money for participants. A lot of personality research can be done just by word-associations, which GPT-3 is good at.
The trouble comes when you want to do anything deeper. If you want to know how personality actually works, you need to think of many levels at once; how people behave, how that behavior translates into your measurement method, etc.. This is not simply an extrapolation of the word-association game, and I have not seen any AI methods that are likely capable of that yet. The situation is further worsened by the fact that there is a lot of text on the internet that describes naive methods, so it would likely jump on that instead.