Fascinating essay. It put into words a lot of inferences I’ve been independently making, while also suggesting a fun angle (the distributed social computations) to look at it from.
I suspect the framework is less fake in certain aspects than might seem at first glance, too! As in, it actually corresponds to some mechanical realities of how human minds are implemented. In terms of the post I’ve linked, you could view “stopping” as momentarily ignoring the self-model you’ve derived and looking at the actual shards that implement your values (and then re-compiling your identity to more closely resemble your “true desires”). Or maybe even going further, trying to screen off the influence of the shards as well, and attempting to look at the world and yourself through the lens of a personality-less general-purpose optimization process (which is mainly useful when there’s something gravely important you want to achieve, and you don’t want to be biased by the flaws your role assigns you).
Fascinating essay. It put into words a lot of inferences I’ve been independently making, while also suggesting a fun angle (the distributed social computations) to look at it from.
I suspect the framework is less fake in certain aspects than might seem at first glance, too! As in, it actually corresponds to some mechanical realities of how human minds are implemented. In terms of the post I’ve linked, you could view “stopping” as momentarily ignoring the self-model you’ve derived and looking at the actual shards that implement your values (and then re-compiling your identity to more closely resemble your “true desires”). Or maybe even going further, trying to screen off the influence of the shards as well, and attempting to look at the world and yourself through the lens of a personality-less general-purpose optimization process (which is mainly useful when there’s something gravely important you want to achieve, and you don’t want to be biased by the flaws your role assigns you).