Conditioning on the arisal of highly-capable long-term coherent planning (i.e., ‘superintelligence’), I think the only difference between these perspectives is where one expects the effective ‘locus of desires’ to be—in the weights or in the activations.
In human-alignment, we might contrast the brain-qua-architecture with the actual brain of a particular person: the brain-qua-architecture has lots of goals (eating, etc.), and so too people have particular goals as instantiations of that architecture (money, power, etc.). The former architecture-level misalignment has solutions which look like Confucius, Hobbes, etc., while the latter ‘persona’-level alignment looks like making said person read Cicero until they know what True Virtue means.
I think a putative LLM-based superintelligence could have its locus of desires either baked-in or instantiated as a persona, but the causes and remediations to either of these as engineering problems must be quite different.
Perhaps a deconfusion would be to dissolve the notion of personae into clouds of correlated beliefs and action—there is a face beneath the mask only to the extent that behaviour deviates from that desired by the ‘mask’ while serving that of the ‘face’. But I don’t think this is a complete description—incoherence or coexistence of different loci of desires seems important. Very Janusian.
Conditioning on the arisal of highly-capable long-term coherent planning (i.e., ‘superintelligence’), I think the only difference between these perspectives is where one expects the effective ‘locus of desires’ to be—in the weights or in the activations.
In human-alignment, we might contrast the brain-qua-architecture with the actual brain of a particular person: the brain-qua-architecture has lots of goals (eating, etc.), and so too people have particular goals as instantiations of that architecture (money, power, etc.). The former architecture-level misalignment has solutions which look like Confucius, Hobbes, etc., while the latter ‘persona’-level alignment looks like making said person read Cicero until they know what True Virtue means.
I think a putative LLM-based superintelligence could have its locus of desires either baked-in or instantiated as a persona, but the causes and remediations to either of these as engineering problems must be quite different.
Perhaps a deconfusion would be to dissolve the notion of personae into clouds of correlated beliefs and action—there is a face beneath the mask only to the extent that behaviour deviates from that desired by the ‘mask’ while serving that of the ‘face’. But I don’t think this is a complete description—incoherence or coexistence of different loci of desires seems important. Very Janusian.