TL;DR: We know that humans and some animals have two tiers of cognition — an integrative metacognition layer, and a lower non-metarepresentational cognition layer. With artificial superintelligence, we should define a third layer and model it as a three-tier hybrid mind. I define the concepts precisely and talk about their implications for alignment.
I also talk about chimp-human composites which is fun.
Really interested in feedback and discussion with the community!
Just published “Meta® Meta Cognition: Intelligence Progression as a Three-Tier Hybrid Mind”
TL;DR: We know that humans and some animals have two tiers of cognition — an integrative metacognition layer, and a lower non-metarepresentational cognition layer. With artificial superintelligence, we should define a third layer and model it as a three-tier hybrid mind. I define the concepts precisely and talk about their implications for alignment.
I also talk about chimp-human composites which is fun.
Really interested in feedback and discussion with the community!