This task seems so far out of distribution for “text on the internet” that I’m not sure what we are supposed to learn from GPT-3′s performance here.
I mean the idea that GPT-3 could understand meta information about it’s own situation and identity like “algorithm aiming to complete or extend this text.” is really far out there.
If we want to assess GPT-3′s symbol grounding maybe we should try tasks that can conceivably be learned from the training data and not something most humans would fail at.
My case for GPT-n bullishness wouldn’t be that GPT-3 is as smart as even a 3-year old child. It’s scaling curves not yet bending and multi-modality (DALL-E definitely does symbol grounding).
This task seems so far out of distribution for “text on the internet” that I’m not sure what we are supposed to learn from GPT-3′s performance here.
I mean the idea that GPT-3 could understand meta information about it’s own situation and identity like “algorithm aiming to complete or extend this text.” is really far out there.
If we want to assess GPT-3′s symbol grounding maybe we should try tasks that can conceivably be learned from the training data and not something most humans would fail at.
My case for GPT-n bullishness wouldn’t be that GPT-3 is as smart as even a 3-year old child. It’s scaling curves not yet bending and multi-modality (DALL-E definitely does symbol grounding).