I’m sure Eliezer has stated somewhere a level of sophistication he expects our techniques will never reach, and I wish I was grading that prediction instead.
I think this Manifold market is such a prediction, from September 2022:
By the end of 2026, will we have transparency into any useful internal pattern within a Large Language Model whose semantics would have been unfamiliar to AI and cognitive science in 2006?
Yudkowsky bought this down to 25%, and I think predicted NO. It’s a bit tricky because it could also be a prediction that the internal patterns within LLMs are all very simple and not semantically novel, but I don’t think this was Yudkowsky’s thesis.
I think this Manifold market is such a prediction, from September 2022:
https://manifold.markets/EliezerYudkowsky/by-the-end-of-2026-will-we-have-tra
Yudkowsky bought this down to 25%, and I think predicted NO. It’s a bit tricky because it could also be a prediction that the internal patterns within LLMs are all very simple and not semantically novel, but I don’t think this was Yudkowsky’s thesis.