Self-improvement for general intelligence had seen minor successes before.
This is the very first sentence in the Iterated distillation and amplification (IDA) collapsible section, and is clearly not being offered as evidence that is meant to justify extrapolating to the very last sentence in that section:
Now, the models have become sufficiently good at verifying more subjective things (e.g. the quality of a work product), allowing the use of IDA to improve the model at many tasks.
The rest of your post has a lot of other objections that seem invalid or confused, like attempting to use the lack of the paper’s peer review as meaningful evidence about whether a technique like that might generalize or not, but I don’t think it’s worth getting into them because the entire argument is premised on a misunderstanding of what evidence is being offered for what purpose.
True, I didn’t include my review of the other article they reference in that last sentence (https://arxiv.org/pdf/2210.11610). That article also contains no evidence to support their claims. I’ll have my full review of AI 2027 finished in a few weeks.
Even granting what you said about the gap between the first and last sentence, the sentence in which the article is referenced is: “Self-improvement for general intelligence had seen minor successes before.”
The report referenced clearly has nothing to do with “general intelligence”: they test the model on 5 narrow algorithmic tasks. And they explicitly and repeatedly say that they have provided no evidence for the model’s applicability in wider tasks.
The AI 2027 authors reference the report apparently as evidence of AI’s successes in self-improvement in “general intelligence”. The report contains no such evidence. So the report is misrepresented by the AI 2027 authors
The link to Supervising strong learners by amplifying weak experts is in the following sentence:
This is the very first sentence in the
Iterated distillation and amplification (IDA)collapsible section, and is clearly not being offered as evidence that is meant to justify extrapolating to the very last sentence in that section:The rest of your post has a lot of other objections that seem invalid or confused, like attempting to use the lack of the paper’s peer review as meaningful evidence about whether a technique like that might generalize or not, but I don’t think it’s worth getting into them because the entire argument is premised on a misunderstanding of what evidence is being offered for what purpose.
True, I didn’t include my review of the other article they reference in that last sentence (https://arxiv.org/pdf/2210.11610). That article also contains no evidence to support their claims. I’ll have my full review of AI 2027 finished in a few weeks.
Even granting what you said about the gap between the first and last sentence, the sentence in which the article is referenced is: “Self-improvement for general intelligence had seen minor successes before.”
The report referenced clearly has nothing to do with “general intelligence”: they test the model on 5 narrow algorithmic tasks. And they explicitly and repeatedly say that they have provided no evidence for the model’s applicability in wider tasks.
The AI 2027 authors reference the report apparently as evidence of AI’s successes in self-improvement in “general intelligence”. The report contains no such evidence. So the report is misrepresented by the AI 2027 authors