Why is it at all plausible that bigger feedforward neural nets trained to predict masses of random data recorded from humans or whatever random other stuff would ever be superintelligent? I think AGI could be near, but I don’t see how this paradigm gets there. I don’t even see why it’s plausible. I don’t even see why it’s plausible that either a feedforward network gets to AGI, or an AI trained on this random data gets to AGI. These of course aren’t hard limits at all. There’s some big feedforward neural net that destroys the world, so there’s some training procedure that makes a world-destroying neural net. But like, gradient descent? On neural nets with pretty simple connectivity? Also there’s some search that finds AIs that are better and better at predicting text, far beyond LLM level, until it’s doing superintelligent general computations. But that’s so extremely… inefficient? IDK.
I think the main concern is that feed forward nets are used as a component in systems that achieve full AGI. For instance, deepmind’s agent systems include a few networks and run a few times before selecting an action. Current networks are more like individual pieces of the human brain, like a visual system and a language system. Putting them together and getting them to choose and pursue goals and subgoals appropriately seems all too plausible.
Now, some people also think that just increasing the size of nets and training data sets will produce AGI, because progress has been so good so far. Those people seem to be less concerned with safety. This is probably because such feedforward nets would be more like tools than agents. I tend to agree with you that this approach seems unlikely to.produce real AGI much less ASI, but it could produce very useful systems that are superhuman in limited areas. It already has in a few areas, such as protein folding.
Why is it at all plausible that bigger feedforward neural nets trained to predict masses of random data recorded from humans or whatever random other stuff would ever be superintelligent? I think AGI could be near, but I don’t see how this paradigm gets there. I don’t even see why it’s plausible. I don’t even see why it’s plausible that either a feedforward network gets to AGI, or an AI trained on this random data gets to AGI. These of course aren’t hard limits at all. There’s some big feedforward neural net that destroys the world, so there’s some training procedure that makes a world-destroying neural net. But like, gradient descent? On neural nets with pretty simple connectivity? Also there’s some search that finds AIs that are better and better at predicting text, far beyond LLM level, until it’s doing superintelligent general computations. But that’s so extremely… inefficient? IDK.
I think the main concern is that feed forward nets are used as a component in systems that achieve full AGI. For instance, deepmind’s agent systems include a few networks and run a few times before selecting an action. Current networks are more like individual pieces of the human brain, like a visual system and a language system. Putting them together and getting them to choose and pursue goals and subgoals appropriately seems all too plausible.
Now, some people also think that just increasing the size of nets and training data sets will produce AGI, because progress has been so good so far. Those people seem to be less concerned with safety. This is probably because such feedforward nets would be more like tools than agents. I tend to agree with you that this approach seems unlikely to.produce real AGI much less ASI, but it could produce very useful systems that are superhuman in limited areas. It already has in a few areas, such as protein folding.