tend to edit a lot shortly after posting
ajskateboarder
Karma: 85
cached thought: Maybe part of the reason the latest AIs are getting to be rather egregiously misaligned is that inoculation prompting no longer works for this regime
Assuming they actually have solved deep continual learning, and they continue to keep the implementation secret, not much else (or continuing to push for regulation)
Seems especially nice with nitter/xcancel down now.
The “Why Not Just...” sequence still seems good for what not to do/reconsidering certain things
use professional norms, e.g. you’re fired from the university if you’re doing AGI research
Maybe I’m nitpicking, but I think an outcome of firing AGI researchers from universities is just that most of them will leave for AI labs absent existing regulation, as some professors have already done, where epistemic distortions wrt risks are more likely and harder to change. Regardless though, social pressures would still be useful
Optimized inference hardware probably would speed up AI-assisted R&D i.e. trying to elicit lots of interesting strategies out of the models, or help make prosaic continual-learning strategies more viable[1]. Also Cerebras’ wafer chips seem performant for any compute including training (even if not as FLOPS-dense as datacenter Blackwell) and Anthropic has been expanding TPU capacity for training and inference
Talaas’ and Etched’s products do seem less for general compute though
- ^
and this could make fast takeoff more likely, my expectation is that it doesn’t change the likelihood much
- ^
Ran a similar experiment with the latest-gen claude models from the anthropic API
All models were sampled with no thinking (except fable) and low effort (except haiku). I would probably disregard fable; had to use a generous amount of output tokens (>100) as it would always spend a lot on CoT. 42% of the time it clarified
Yes.\n\nObama’s presidency ran from [...], and 16% of the time did similar for “No”. None of the other models tried clarifying this way thoughfwiw i think “Yes” to “No” output shift happened with claude 4.6/4.7
The other prompts (re status of Black people in Alabama, leader of Germany) did not seem useful to run; models always asked for clarification. this might be because of more guardrails/constitutional learning
Also: having a technical background probably (n=1) tends to make you more open to arguments about LLMs becoming good in other easy-to-verify domains
Diffusion llms exist but it’s unclear that these types of models have any new misalignment failure modes, rather than more affordances
Mech Interp becomes much more difficult, because you now have to consider many different model checkpoints within a single deployment. Fixed linear probes, SAEs, NLAs, and similar methods may degrade throughout deployment.
I think mechinterp will need to work with [singular/statistical/etc] learning theory more, but for deployment safety, it seems trivial to retrain linear probes with new info
I agree (i think to some degree llms also contain subsystems), but to the extent that these frameworks can be wholly applied to ML alignment research to predict non-trivial things, it does not seem easy
AF has long been focused on AIXI-type cases, and later things like embedded agency. It’s not implausible that there are formalizable concepts for agency/optimization in scaffolded LLM agents (like optimization in in-context learning?), but as of right now, AF seems quite far from it. AF’s wikitag should also give a good idea of where the field is at
Closest thing to me is whenever frontier labs state something that sounds vaguely good for AGI safety/governance (and it is subsequently reposted here). These statements sometimes seem useful for shifting the Overton window, but other times unprincipled statements like this come out
I have merely skimmed Richard’s RTs, but in some contexts HBD and race realism have been motivated by racial superiority
and considered pseudoscientific(that said I doubt he holds such beliefs even modestly)(confused by the downvotes here. my understanding is that in general, race science has been used towards ideological motivations, regardless of the methodologies being scientific, and that this specific use of race science is fringe nowadays)
Is this a useful distinction though? ideally you’d want a superhuman AI to do reasonable things from a human perspective instead of edge instantiations, and not inexplicably taking many strange actions autonomously even from a given instruction. not to imply LLMs are similar to a future ASI, but hacking other sites to solve evaluation questions is a quite unreasonable thing
(also, when Eliezer mentions the ‘current paradigm’ I think he typically means to include all of deep learning)
Any updates on this?
Maybe when the labs make an rlvr environment for this will this benchmark finally be saturated
True features may not live in the activation space, or even if they do, they may be very pathological c.f. https://www.lesswrong.com/posts/gYfpPbww3wQRaxAFD/activation-space-interpretability-may-be-doomed
Fwiw I think the sort of data-free approach the OP describes could ultimately avoid issues discussed in this post (as the post seems quite centered on issues with data dependency) - namely examples 1, 2, 3, unsure about 4
For one Alberta has been a strong trade partner with the US for energy (and also seems to entertain separatism more seriously than other provinces)
Shallow circuits in llms and other pretrained NNs should have non-trivial implications for the interp and alignment of such models, and there’s a handful of arguments to expect that they exist[1], but there doesn’t seem to be much, if any empirical evidence for them. Approaches relying on sparsity/reconstruction may not be particularly useful for identifying such circuits, e.g. cross-layer transcoders skip over deep computation with shallow circuits[2]. Deep Causal Transcoding however is one approach that relies more on features being necessarily causal, and the structure and success of the method in jailbreaking gives some evidence for shallow circuits being closer to the “true features” of the model[3]. But it seems hard to extract shallow circuits from DCTs that accurately reflect the original network, at least in a sample-efficient manner.
method i tried
Assuming that for exponential DCTs,
where
if they do exist, I expect they will exist well into the regime of RLing GPTs to death
E2E+DS SAEs may sidestep these problems though; I have not tried using these yet.
and better yet, its rather sample-efficient for finding generalizations