[Question] What experiment settles the Gary Marcus vs Geoffrey Hinton debate?

It started with this video of Hinton taking a jab at Marcus: https://​​twitter.com/​​tsarnick/​​status/​​1754439023551213845

And here is Marcu’s answer:


As far as I understand, Gary Marcus argues that LLMs memorize some of their training data, while Hinton argues that no such thing takes place, it’s all just patterns of language.

I found these two papers on LLM memorization:

https://​​arxiv.org/​​abs/​​2202.07646 - Quantifying Memorization Across Neural Language Models

https://​​browse.arxiv.org/​​abs/​​2311.17035 - Scalable Extraction of Training Data from (Production) Language Models

Am I missing something here? Are these two positions compatible or does one need to be wrong for the other one to be correct? What is the crux between them and what experiment could be devised to test it?

No comments.