Right on. That’s a good point. So really I guess the conclusion is: Compute is the bottleneck; an AI chatbot or whatever could totally learn random facts the very first time it encounters them, if you had things set up to amplify that data into some auxiliary dataset and then train on it. Costs a few orders of magnitude more compute perhaps, but gets the job done. Right? (And this could be automated & “smart” in the sense that the AI could decide what stuff to memorize/internalize, what stuff to forget, and what stuff to add to some software database.)
Right. Of course if the sample efficiency of learning improves, the cost goes down, but that’s not really crucial for anything. The learning part of AGI is already essentially solved, it just needs to be put into a place where it’s getting fed the right data.
Right on. That’s a good point. So really I guess the conclusion is: Compute is the bottleneck; an AI chatbot or whatever could totally learn random facts the very first time it encounters them, if you had things set up to amplify that data into some auxiliary dataset and then train on it. Costs a few orders of magnitude more compute perhaps, but gets the job done. Right? (And this could be automated & “smart” in the sense that the AI could decide what stuff to memorize/internalize, what stuff to forget, and what stuff to add to some software database.)
Right. Of course if the sample efficiency of learning improves, the cost goes down, but that’s not really crucial for anything. The learning part of AGI is already essentially solved, it just needs to be put into a place where it’s getting fed the right data.