I’ve switched from considering uploading to be obviously possible at sufficient technological advancement to considering it probably intractable. More specifically, I expect the mind to be importantly shaped by a lot of rarely-activating mechanisms, which are intractable to map out. You could probably eventually make a sort of “zombie upload” that ignores those mechanisms, but it would be unable to update to new extreme conditions.
Idk, the shift happened a while ago. Maybe mostly just reflecting on how evolution acts on a holistic scale, making it easy to incorporate “gradients” from events that occur only one or a few times in one’s lifetime, if these events have enough effect on survival/reproduction. Part of a bigger change in priors towards the relevance of long tails associated with my LDSL sequence.
Yeah, until recently I thought the same thing, based on my belief that distilling a teacher model which has been trained by RL into a student model preserved not just distributions over outputs but also mostly preserved the mechanisms behind those outputs. Which as far as I can tell was an incorrect belief.
Once you focus on “parts” of the brain, you’re restricting consideration to mechanisms that are activated at sufficient scale to need to balloon up. I would expect the rarely-activating mechanisms to be much smaller in a physical sense than “parts” of the brain are
I’ve switched from considering uploading to be obviously possible at sufficient technological advancement to considering it probably intractable. More specifically, I expect the mind to be importantly shaped by a lot of rarely-activating mechanisms, which are intractable to map out. You could probably eventually make a sort of “zombie upload” that ignores those mechanisms, but it would be unable to update to new extreme conditions.
What changed your mind? Any rabbit holes in particular I can go down?
Idk, the shift happened a while ago. Maybe mostly just reflecting on how evolution acts on a holistic scale, making it easy to incorporate “gradients” from events that occur only one or a few times in one’s lifetime, if these events have enough effect on survival/reproduction. Part of a bigger change in priors towards the relevance of long tails associated with my LDSL sequence.
Yeah, until recently I thought the same thing, based on my belief that distilling a teacher model which has been trained by RL into a student model preserved not just distributions over outputs but also mostly preserved the mechanisms behind those outputs. Which as far as I can tell was an incorrect belief.
Counterpoint: brain is extremely energy-demanding and every part of it that is not working often gets brutally selected out.
Once you focus on “parts” of the brain, you’re restricting consideration to mechanisms that are activated at sufficient scale to need to balloon up. I would expect the rarely-activating mechanisms to be much smaller in a physical sense than “parts” of the brain are