I think this is royghly correct for neural network based minds. Integrating new knowledge so that it becomes usable in the way we call understanding must be done by either thinking, or a process so sophisticated in its understanding of that mind that it would require substantial work from a superintelligence. It would have to anticipate many of the trajectories those weights could cause through complex wanting space. The easiest rout would be simulating that thinking; other routes might have similar implications of creating a duplicate of that mind.
To put it another way, understanding is a process happening over time in both the learning and expression, making it much more complex than just adding some weights like a set of training example does.
I think this is royghly correct for neural network based minds. Integrating new knowledge so that it becomes usable in the way we call understanding must be done by either thinking, or a process so sophisticated in its understanding of that mind that it would require substantial work from a superintelligence. It would have to anticipate many of the trajectories those weights could cause through complex wanting space. The easiest rout would be simulating that thinking; other routes might have similar implications of creating a duplicate of that mind.
To put it another way, understanding is a process happening over time in both the learning and expression, making it much more complex than just adding some weights like a set of training example does.