Thanks. For others interested, the relevant quote seems to be:
TechCrunch and Axios report that he will work under team lead Nick Joseph on pre-training, “focused on using Claude to accelerate pre-training research.”
It seems to be a bit vague. You could imagine various uses of Claude in pre-training research, which may or may not be RSI. For instance you could use it to build better safety evals during pre-training, or build faster tokenizers etc. I don’t see where he or Anthropic have said that he’s there “explicitly to do recursive self-improvement”, but maybe Zvi is basing this on non-public information.
Presumably, he is quite enthusiastic about this approach and would like to see how it can be made to work at scale (where one cannot do a full run from scratch for every small modification, so it’s not quite straightforward).
Thanks. For others interested, the relevant quote seems to be:
It seems to be a bit vague. You could imagine various uses of Claude in pre-training research, which may or may not be RSI. For instance you could use it to build better safety evals during pre-training, or build faster tokenizers etc. I don’t see where he or Anthropic have said that he’s there “explicitly to do recursive self-improvement”, but maybe Zvi is basing this on non-public information.
Later in that post they discuss his March “autoresearch” efforts, specifically
https://x.com/karpathy/status/2030371219518931079
https://github.com/karpathy/autoresearch
https://x.com/karpathy/status/2031135152349524125
Presumably, he is quite enthusiastic about this approach and would like to see how it can be made to work at scale (where one cannot do a full run from scratch for every small modification, so it’s not quite straightforward).
Ah, I missed that. In that case, you’re right, autoresearch is close enough to RSI.