If we look at the game of Go, AI managed to be vastly better than humans. An AI that can outcompete humans at any task the way that AlphaGo can outcompete human at Go is a serious problems even if it’s not capable of directly figuring out how to build nanobots.
This is exactly how we train modern large ANNs, and LLMs specifically: by training them on the internet, we are training them on human thoughts and thus (partially) distilling human minds.
While that’s true that currently most of the training data we put into LLMs seems human-created, I don’t think there’s a good reason to assume that will stay true.
AlphaGo for example is not trained with any human data at all. AlphaGo is trained based on a bunch of computer-generated data and uses that principle to achieve superhuman performance.
If you would want to make an LLM better at arithmetic, you would autogenerate a lot of data of correct arithmetic reasoning. If having an LLM that’s good at arithmetic is valuable to you, you can autogenerate text of arithmetic problems that is as large as the current human-generated text corpus from the internet.
There are going to be many tasks for which you can produce training data that will allow a LLM to successfully do the task in a way that humans currently can’t. If you train your model to solve a lot of human tasks that humans currently can’t solve the resulting LLM is likely farther from human minds than GPT4 happens to be.
If we look at the game of Go, AI managed to be vastly better than humans. An AI that can outcompete humans at any task the way that AlphaGo can outcompete human at Go is a serious problems even if it’s not capable of directly figuring out how to build nanobots.
While that’s true that currently most of the training data we put into LLMs seems human-created, I don’t think there’s a good reason to assume that will stay true.
AlphaGo for example is not trained with any human data at all. AlphaGo is trained based on a bunch of computer-generated data and uses that principle to achieve superhuman performance.
If you would want to make an LLM better at arithmetic, you would autogenerate a lot of data of correct arithmetic reasoning. If having an LLM that’s good at arithmetic is valuable to you, you can autogenerate text of arithmetic problems that is as large as the current human-generated text corpus from the internet.
There are going to be many tasks for which you can produce training data that will allow a LLM to successfully do the task in a way that humans currently can’t. If you train your model to solve a lot of human tasks that humans currently can’t solve the resulting LLM is likely farther from human minds than GPT4 happens to be.