It feels like a “gotcha” rebuke, but it honestly doesn’t seem like it really addresses the article’s point. Unless you think GPT-3 would perform better if given more time to work on it?
How does it not address the article’s point? What I’m saying is that Armstrong’s example was an unfair “gotcha” of GPT-3; he’s trying to make some sort of claim about its limitations on the basis of behavior that even a human would also exhibit. Unless he’s saying we humans also have this limitation...
Yes, I think GPT-3 would perform better if given more time to work on it (and fine-tuning to get used to having more time). See e.g. PaLM’s stuff about chain-of-thought prompting. How much better? I’m not sure. But I think its failure at this particular task tells us nothing.
It feels like a “gotcha” rebuke, but it honestly doesn’t seem like it really addresses the article’s point. Unless you think GPT-3 would perform better if given more time to work on it?
How does it not address the article’s point? What I’m saying is that Armstrong’s example was an unfair “gotcha” of GPT-3; he’s trying to make some sort of claim about its limitations on the basis of behavior that even a human would also exhibit. Unless he’s saying we humans also have this limitation...
Yes, I think GPT-3 would perform better if given more time to work on it (and fine-tuning to get used to having more time). See e.g. PaLM’s stuff about chain-of-thought prompting. How much better? I’m not sure. But I think its failure at this particular task tells us nothing.