I feel like reading this and thinking about it gave me a “new idea”! Fun! I rarely have ideas this subjectively new and wide in scope!
Specifically, I feel like I instantly understand what you mean by this, and yet also I’m fascinated by how fuzzy and magical and yet precise the language here (bold and italic not in original) feels...
What we have learnt in these years is that it is possible to build an intelligence that has a much more fragmented cognitive manifold than humans do.
The phrase “cognitive manifold” has legs!
It showed up in “Learning cognitive manifolds of faces”, in 2017 (a year before BERT!) in a useful way, that integrates closely with T-SNE-style geometric reasoning about the proximity of points (ideas? instances? examples?) within a conceptual space!
Also, in “External Hippocampus: Topological Cognitive Maps for Guiding Large Language Model Reasoning” it motivates a whole metaphor for modeling and reasoning about embedding spaces (at least I think that’s what’s going on here after 30 seconds of skimming) and then to a whole new way of characterizing and stimulating weights in an LLM that is algorithmically effective!
I’m tempted to imagine that there’s an actual mathematical idea related to “something important and real” here! And, maybe this idea can be used to characterize or agument the cognitive capacity of both human minds and digital minds...
...like it might be that each cognitively coherent microtheory (maybe in this sense, or this, or this?) in a human psyche “is a manifold” and that human minds work as fluidly/fluently as they do because maybe we have millions of “cognitive manifolds” (perhaps one for each cortical column?) and then maybe each idea we can think about effectively is embedded in many manifolds, where each manifold implies a way of reasoning… so long as one (or a majority?) of our neurological manifolds can handle an idea effectively, maybe the brain can handle them as a sort of “large, effective, and highly capable meta-manifold”? </wild-speculation-about-humans>
Then LLMs might literally only have one such manifold which is an attempt to approximate our metamanifold… which works!?
Or whatever. I’m sort of spitballing here...
I’m enjoying the possibility that the word “cognitive manifold” is actually very coherently and scientifically and numerically meaningful as a lens for characterizing all possible minds in terms of the, number, scope, smoothness, of their “cognitive manifolds” in some deeply real and useful way.
It would be fascinating if we could put brain connectomes and LLM models into the same framework and characterize each kind of mind in some moderately objective way, such as to establish a framework for characterizing intelligence in some way OTHER than functional performance tests (such as those that let us try to determine the “effective iq” of a human brain or a digital model in a task completion context).
If it worked, we might be able to talk quite literally about the scope, diversity, smoothness, etc, of manifolds, and add such characterizations up into a literal number for how literally smart any given mind was.
Then we could (perhaps) dispense with words like “genius” and “normie” and “developmentally disabled” as well as “bot” and “AGI” and “weak ASI” and “strong ASI” and so on? Instead we could let these qualitative labels be subsumed and obsoleted by an actually effective theory of the breadth and depth of minds in general?? Lol!
I doubt it, if course. But it would be fun if it was true!
Shameless self-writing promotion as your comments caught my attention (first this, now this comment on cognitive manifolds): I wrote about how we might model superintelligence as “meta metacognition” (possible parallel to your “manifold approximating our metamanifold”) — see third order cognition.
I need to create a distilled write-up as the post isn’t too easily readable… it’s long so please just skim if you are interested. The main takeaway though is that if we do model digital intelligence this way, we can try to precisely talk about how it relates to human intelligence and explore those factors within alignment research/misalignment scenarios.
I determine these factors to describe the relationship: 1) second-order identity coupling, 2) lower-order irreconcilability, 3) bidirectional integration with lower-order cognition, 4) agency permeability, 5) normative closure, 6) persistence conditions, 7) boundary conditions, 8) homeostatic unity.
I feel like reading this and thinking about it gave me a “new idea”! Fun! I rarely have ideas this subjectively new and wide in scope!
Specifically, I feel like I instantly understand what you mean by this, and yet also I’m fascinated by how fuzzy and magical and yet precise the language here (bold and italic not in original) feels...
The phrase “cognitive manifold” has legs!
It showed up in “Learning cognitive manifolds of faces”, in 2017 (a year before BERT!) in a useful way, that integrates closely with T-SNE-style geometric reasoning about the proximity of points (ideas? instances? examples?) within a conceptual space!
Also, in “External Hippocampus: Topological Cognitive Maps for Guiding Large Language Model Reasoning” it motivates a whole metaphor for modeling and reasoning about embedding spaces (at least I think that’s what’s going on here after 30 seconds of skimming) and then to a whole new way of characterizing and stimulating weights in an LLM that is algorithmically effective!
I’m tempted to imagine that there’s an actual mathematical idea related to “something important and real” here! And, maybe this idea can be used to characterize or agument the cognitive capacity of both human minds and digital minds...
...like it might be that each cognitively coherent microtheory (maybe in this sense, or this, or this?) in a human psyche “is a manifold” and that human minds work as fluidly/fluently as they do because maybe we have millions of “cognitive manifolds” (perhaps one for each cortical column?) and then maybe each idea we can think about effectively is embedded in many manifolds, where each manifold implies a way of reasoning… so long as one (or a majority?) of our neurological manifolds can handle an idea effectively, maybe the brain can handle them as a sort of “large, effective, and highly capable meta-manifold”? </wild-speculation-about-humans>
Then LLMs might literally only have one such manifold which is an attempt to approximate our metamanifold… which works!?
Or whatever. I’m sort of spitballing here...
I’m enjoying the possibility that the word “cognitive manifold” is actually very coherently and scientifically and numerically meaningful as a lens for characterizing all possible minds in terms of the, number, scope, smoothness, of their “cognitive manifolds” in some deeply real and useful way.
It would be fascinating if we could put brain connectomes and LLM models into the same framework and characterize each kind of mind in some moderately objective way, such as to establish a framework for characterizing intelligence in some way OTHER than functional performance tests (such as those that let us try to determine the “effective iq” of a human brain or a digital model in a task completion context).
If it worked, we might be able to talk quite literally about the scope, diversity, smoothness, etc, of manifolds, and add such characterizations up into a literal number for how literally smart any given mind was.
Then we could (perhaps) dispense with words like “genius” and “normie” and “developmentally disabled” as well as “bot” and “AGI” and “weak ASI” and “strong ASI” and so on? Instead we could let these qualitative labels be subsumed and obsoleted by an actually effective theory of the breadth and depth of minds in general?? Lol!
I doubt it, if course. But it would be fun if it was true!
Shameless self-writing promotion as your comments caught my attention (first this, now this comment on cognitive manifolds): I wrote about how we might model superintelligence as “meta metacognition” (possible parallel to your “manifold approximating our metamanifold”) — see third order cognition.
I need to create a distilled write-up as the post isn’t too easily readable… it’s long so please just skim if you are interested. The main takeaway though is that if we do model digital intelligence this way, we can try to precisely talk about how it relates to human intelligence and explore those factors within alignment research/misalignment scenarios.
I determine these factors to describe the relationship: 1) second-order identity coupling, 2) lower-order irreconcilability, 3) bidirectional integration with lower-order cognition, 4) agency permeability, 5) normative closure, 6) persistence conditions, 7) boundary conditions, 8) homeostatic unity.