When working something out with someone else, I find it helpful to try to only communicate measurable and concrete thoughts / predictions, and not any intermediate state. I find that, even among people of very similar intelligence, “internal” intermediate abstract states can be wildly different and incompatible. Attempting to reconcile the abstract state can lead to lots of frustration.
It feels a bit like skiing down a forest run. Coming up to a tree, two similarly good skiers could go around it either via the left or via the right. Their position will agree both before and after the tree, but while passing it they’ll be in very different positions (and in fact forcing an averaging of those positions would be disaster).
Probably this is similar to trying to average out an arbitrary intermediate layer of two neural nets trained on the same dataset but with different random initializations & seeds. Actually, that would be a fun experiment to run. Maybe sometimes it works, and sometimes it doesn’t. Maybe thinking is sometimes smooth and other times highly discontinuous.
When working something out with someone else, I find it helpful to try to only communicate measurable and concrete thoughts / predictions, and not any intermediate state. I find that, even among people of very similar intelligence, “internal” intermediate abstract states can be wildly different and incompatible. Attempting to reconcile the abstract state can lead to lots of frustration.
It feels a bit like skiing down a forest run. Coming up to a tree, two similarly good skiers could go around it either via the left or via the right. Their position will agree both before and after the tree, but while passing it they’ll be in very different positions (and in fact forcing an averaging of those positions would be disaster).
Probably this is similar to trying to average out an arbitrary intermediate layer of two neural nets trained on the same dataset but with different random initializations & seeds. Actually, that would be a fun experiment to run. Maybe sometimes it works, and sometimes it doesn’t. Maybe thinking is sometimes smooth and other times highly discontinuous.