Just a data point that support hold_my_fish’s argument: Savant Kim Peek did likely memorize gigabytes of information and could access them quite reliably:
Note that while he has amazing recall, to me, it seems that he has much less synthesis than you might expect. I remember that only later in life did he start to draw some connections between data he consumed. He doesn’t seem to form an integrated world model from all of the data.
In that sense, Kim Peek is more like a database with free lookup while GPT-3 is more like a human in that it provides integrated answers but can’t quote verbatim in general (or tell where its knowledge is coming from).
Maybe there is a trade-off between integration and recall.
Just a data point that support hold_my_fish’s argument: Savant Kim Peek did likely memorize gigabytes of information and could access them quite reliably:
https://personal.utdallas.edu/~otoole/CGS_CV/R13_savant.pdf
Ooh interesting! Can you say how you’re figuring that it’s “gigabytes of information?”
9000 books with one megabyte of information each. Note, that he memorized whole telephone books that way and tests have shown that he has >95% recall.
Still mulling this over. I may end up revising the post and/or writing a follow-up. :)
Note that while he has amazing recall, to me, it seems that he has much less synthesis than you might expect. I remember that only later in life did he start to draw some connections between data he consumed. He doesn’t seem to form an integrated world model from all of the data.
In that sense, Kim Peek is more like a database with free lookup while GPT-3 is more like a human in that it provides integrated answers but can’t quote verbatim in general (or tell where its knowledge is coming from).
Maybe there is a trade-off between integration and recall.