Our sense is that models are relatively bad at tasks like LMCA
FWIW, my take is that they’re not bad compared to the reasonably smart human distribution! And surprisingly bad at free-form / long-form stuff given their LMCA performance.
(Maybe Em meant bad compared to coding)
(edit: She in fact meant compared coding, physics, Math etc., just in general their other capabilities. Which was actually kinda obvious from her comment, on reflection :P But I still wanted to avoid confusion bc I think ppl sometimes get the wrong impression when we say models are bad at LMCA)
FWIW, my take is that they’re not bad compared to the reasonably smart human distribution! And surprisingly bad at free-form / long-form stuff given their LMCA performance.
(Maybe Em meant bad compared to coding)
(edit: She in fact meant compared coding, physics, Math etc., just in general their other capabilities. Which was actually kinda obvious from her comment, on reflection :P But I still wanted to avoid confusion bc I think ppl sometimes get the wrong impression when we say models are bad at LMCA)