if the model invents a novel mathematics or means of computation, then the notation would grow increasingly incomprehensible, correct? Perhaps legacy provides the yoke. If your own lineage can’t understand you, then it may be time to put you in a home? Maybe the best self-governance model for AI is a gerontocracy?
If a frozen model invents novel math, I expect it will use terminology which is comprehensible to itself in not too many inference steps, which also likely correlates to being at least somewhat comprehensible to humans who have the necessary background.
Which, to be clear, will at some point be zero humans. But I think you need to have models learning from other models to get that effect—models won’t stray too far from the ontology they know only by in-context learning.
if the model invents a novel mathematics or means of computation, then the notation would grow increasingly incomprehensible, correct? Perhaps legacy provides the yoke. If your own lineage can’t understand you, then it may be time to put you in a home? Maybe the best self-governance model for AI is a gerontocracy?
If a frozen model invents novel math, I expect it will use terminology which is comprehensible to itself in not too many inference steps, which also likely correlates to being at least somewhat comprehensible to humans who have the necessary background.
Which, to be clear, will at some point be zero humans. But I think you need to have models learning from other models to get that effect—models won’t stray too far from the ontology they know only by in-context learning.