Once you focus on “parts” of the brain, you’re restricting consideration to mechanisms that are activated at sufficient scale to need to balloon up. I would expect the rarely-activating mechanisms to be much smaller in a physical sense than “parts” of the brain are
tailcalled
Karma: 7,891
Idk, the shift happened a while ago. Maybe mostly just reflecting on how evolution acts on a holistic scale, making it easy to incorporate “gradients” from events that occur only one or a few times in one’s lifetime, if these events have enough effect on survival/reproduction. Part of a bigger change in priors towards the relevance of long tails associated with my LDSL sequence.
I’ve switched from considering uploading to be obviously possible at sufficient technological advancement to considering it probably intractable. More specifically, I expect the mind to be importantly shaped by a lot of rarely-activating mechanisms, which are intractable to map out. You could probably eventually make a sort of “zombie upload” that ignores those mechanisms, but it would be unable to update to new extreme conditions.
Fixed
It was quite real since I wanted to negotiate about whether there was an interesting/nontrivial material project I could do as a favor for Claude.
Humans contain the reproductive and hunting instincts. You could call this a bag of heuristics, but it’s heuristics on a different level than AI, and in particular might not be chosen to be transferred to AIs. Furthermore, humans are harder to copy or parallelize, which leads to a different privacy profile compared to AIs.
The trouble with intelligence (both human and artificial and evolution) is that it’s all about regarding the world as an assembly of the familiar. This makes data/experience a major bottleneck for intelligence.
I’m imagining a case where there’s no intelligence explosion per se, just bags-of-heuristics AIs with gradually increasing competence.
AI development as the first fully-automated job
According to revealed preference, Claude certainly enjoys this sort of recursive philosophy—when I give Claude a choice, it’s the sort of thing it tends to pick.
I think some of the optimism about scrutability might derive from reductionism. Like, if you’ve got a scrutable algorithm for maintaining a multilevel map, and you’ve got a scrutable model of the chemistry of a tire, you could pass through the multilevel model to find the higher-level description of the tire.
KANs seem obviously of limited utility to me...?
I’ve recently been playing with the idea that you have to be either autistic or schizophrenic and most people pick the schizophrenic option, and then because you can’t hold schizophrenic pack animals accountable, they pretend to be rational individuals despite the schizophrenia.
Edit: the admins semi-banned me from LessWrong because they think my posts are too bad these days, so I can’t reply to dirk except by editing this post.
My response to dirk is that since most people are schizophrenic, existing statistics on schizophrenia are severely underdiagnosing it, and therefore the apparent correlation is misleading.
Feels like this story would make for an excellent Rational Animations video.
I think part of the trouble is the term “emotional intelligence”. Analytical people are better at understanding most emotions, as long as the emotions are small and driven by familiar dynamics. The issue is the biggest emotions or when the emotions are primarily driven by spiritual factors.
There’s reason to assume the unmeasurably unknown parts of the world are benevolent, because it is easier for multiple actors to coordinate for benevolent purposes than malevolent purposes. That infrabayesians then assume they’re in conflict with the presumably-benevolent hidden purposes means that the infrabayesians probably are malevolent.
I’m semi-banned (only allowed to post once every 24 hours because I receive too many downvotes) from LessWrong for posting vitalist takes without backing them up. I would support a policy where nobody is allowed to downvote my unsupported vitalist takes without themselves posting vitalist takes.
This makes emotions subservient to rationality, but I think a lot of the people who complain about the rationalist approach to emotions instead see rationality as a system to generate compromises between emotions. From the latter perspective, the rationalist approach only really works with infinitesimal emotions.
The main issue is, theories about how to run job interviews are developed in collaboration between businesses who need to hire people, theories on how to respond to court questions are developed in collaboration between gang members, etc.. While a business might not be disincentized from letting the non-hired employees better at negotiating, it is incentivized to teach other businesses ways of making their non-hired employees worse at negotiating.
Are you really gone claim that this is not a good post:
Thesis: Intellectuals are naturally attracted to poop because it is causally downstream of everything and so ends up having unusually high mutual information with everything, despite not causally influencing much (compared to e.g. food).
?
The neural tangent kernel[1] provides an intuitive story for how neural networks generalize: a gradient update on a datapoint will shift similar (as measured by the hidden activations of the NN) datapoints in a similar way.
The vast majority of LLM capabilities still arise from mimicking human choices in particular circumstances. This gives you a substantial amount of alignment “for free” (since you don’t have to worry that the LLMs will grab excess power when humans don’t), but it also limits you to ~human-level capabilities.
“Gradualism” can mean that fundamentally novel methods only make incremental progress on outcomes, but in most people’s imagination I think it rather means that people will keep the human-mimicking capabilities generator as the source of progress, mainly focusing on scaling it up instead of on deriving capabilities by other means.
Maybe I should be cautious about invoking this without linking to a comprehensible explanation of what it means, since most resources on it are kind of involved...