related: i strongly believe that while general intelligence is an initial prewired network structure at first, most of valuable general intelligence including the parts tested by any iq test can be learned as long as your brain works anywhere near human baseline. If you can understand language enough to have a conversation, distillation can transfer others’ iq structures to your brain. It’s just very very very hard to do the training that would produce this, it would look like something even more difficult and complicated than n-back, which I do not think does a good job, it’s merely a demo that something like this is possible.
Relatedly, while the strongest hard ASIs will be able to run faster than humans, it is my view that human brains can encode the skill and knowledge of hard ASI using a substrate of neuron cells. It would be a somewhat inefficient emulation, but as self-programmable FPGAs ourselves, what we need is dense, personalized, knowledge-traced[1] training data.
Of course, that doesn’t much reassure, as to do that requires not being destroyed by a hard ASI first.
But importantly we don’t currently know how to do that, if it’s even possible without involving ASIs, or making use of detailed low-level models of a particular brain, or requiring hundreds of subjective years to achieve substantial results, or even more than one of these at once.
This has the shape of a worry I have about immediate feasibility of LLM AGIs (before RL and friends recycle the atoms). They lack automatic access to skills for agentic autonomous operation, so the first analogy is with stroke victims. What needs to happen for them to turn AGIs is a recovery program, teaching of basic agency skills and their activation at appropriate times. But if LLMs are functionally more like superhumanly erudite low-IQ humans, figuring out how to teach them the use of those skills might be too difficult, and won’t be immediately useful for converting compute to research even if successful.
related: i strongly believe that while general intelligence is an initial prewired network structure at first, most of valuable general intelligence including the parts tested by any iq test can be learned as long as your brain works anywhere near human baseline. If you can understand language enough to have a conversation, distillation can transfer others’ iq structures to your brain. It’s just very very very hard to do the training that would produce this, it would look like something even more difficult and complicated than n-back, which I do not think does a good job, it’s merely a demo that something like this is possible.
Relatedly, while the strongest hard ASIs will be able to run faster than humans, it is my view that human brains can encode the skill and knowledge of hard ASI using a substrate of neuron cells. It would be a somewhat inefficient emulation, but as self-programmable FPGAs ourselves, what we need is dense, personalized, knowledge-traced[1] training data.
Of course, that doesn’t much reassure, as to do that requires not being destroyed by a hard ASI first.
phrase definition from eg https://stanford.edu/~cpiech/bio/papers/deepKnowledgeTracing.pdf, and see also papers citing this
But importantly we don’t currently know how to do that, if it’s even possible without involving ASIs, or making use of detailed low-level models of a particular brain, or requiring hundreds of subjective years to achieve substantial results, or even more than one of these at once.
This has the shape of a worry I have about immediate feasibility of LLM AGIs (before RL and friends recycle the atoms). They lack automatic access to skills for agentic autonomous operation, so the first analogy is with stroke victims. What needs to happen for them to turn AGIs is a recovery program, teaching of basic agency skills and their activation at appropriate times. But if LLMs are functionally more like superhumanly erudite low-IQ humans, figuring out how to teach them the use of those skills might be too difficult, and won’t be immediately useful for converting compute to research even if successful.