Max Niederman
I suspect that this method will only work well on tasks where the model needs to reason explicitly in order to cheat. So, e.g., if the model needs to reason out some trait of the user in order to flatter them the prompt will likely kick in and get it to self-report its cheating, but if the model can learn to flatter the user without on-the-fly without reasoning the prompt probably won’t do anything. By analogy, if I instruct a human to tell me whenever they use hand gestures to communicate something, they will have difficulty because their hand gestures are automatic and not normally promoted to conscious attention.
There’s the atom transformer in AlphaFold-like architectures, although the embeddings it operates on do encode 3D positioning from earlier parts of the model so maybe that doesn’t count.
Transformers do not natively operate on sequences.
This was a big misconception I had because so much of the discussion around transformers is oriented around predicting sequences. However, it’s more accurate to think of general transformers as operating on unordered sets of tokens. The understanding of sequences only comes if you have a positional embedding to tell the transformer how the tokens are ordered, and possibly a causal mask to force attention to flow in only one direction.
The Money Stuff column mentioned AI alignment, rationality, and the UK AISI today:
Here is a post from the UK AI Security Institute looking for economists to “find incentives and mechanisms to direct strategic AI agents to desirable equilibria.” One model that you can have is that superhuman AI will be terrifying in various ways, but extremely rational. Scary AI will not be an unpredictable lunatic; it will be a sort of psychotic pursuing its own aims with crushing instrumental rationality. And arguably that’s where you need economists! The complaint people have about economics is that it tries to model human behavior based on oversimplified assumptions of rationality. But if super AI is super-rational, economists will be perfectly suited to model it. Anyway if you want to design incentives for AI here’s your chance.
@samuelshadrach (currently ratelimited) sent me the following document on the difference between elite and knowledge class social norms. This is not per se about economic class like I’m primarily interested in, and it’s more about different social norms than subtle markers, but it’s somewhat relevant so I’m linking it here:
Skiing is an interesting one. I never thought about it in those terms since I grew up in Alaska where downhill skiing was relatively accessible (like CO/UT). I also wouldn’t be surprised if outdoor activities in general are correlated with class, even when they’re not necessarily expensive (e.g. hiking).
[Question] What are non-obvious class markers?
I don’t think it’s correct to describe the optimization social media companies do as Goodharting. They’re optimizing for exactly what they want: money. It’s not that they want what’s truly best for their users and are mistaking engagement for that—I think it’s pretty clear at this point social media companies don’t care at all about their users’ wellbeing.
[Question] Where are the AI safety replications?
FWIW, I don’t think the site looks significantly worse on dark mode, although I can understand the desire not to have to optimize for two colorschemes.
Is there a reason that LessWrong defaults to light mode rather than automatically following the browser’s setting? I personally find it a bit annoying to have to select auto every time I have a new
localStorageand it’s not clear to me what the upside is.
It’s worth noting that, though it’s true that for a sufficiently large cluster most pairs of people are not strongly connected, they are significantly more likely to be connected than in a random graph. This is the high clustering coefficient property of small-world graphs like the social graph.
The bottleneck for sampling from the posterior is sampling from the truncated normal distribution, which I’m doing using this implementation of minimax tilting rejection sampling. It’s not exactly instant, but on my laptopt it’s a few CPU-ms/sample up to ~100 comparisons and 50 CPU-ms up to ~200 comparisons. This probably prevents it from being useful for lists with many hundreds of items, but I’ve found it to be fine for interactive use since selecting the next comparison doesn’t require any sampling. The only thing which is prohibitively slow (for me at least) is computing the entropy of the full posterior since it involves so many evaluations of .
Honestly, I can’t think of any instances at all where people are afraid of robots that actually exist, rather than robots they think might exist decades in the future. People aren’t afraid of Spot or bomb defusion robots or CNC machines or even surgery robots that quite literally wriggle around in your body and rearrange your organs.
Fullrank: Bayesian Noisy Sorting
You have a greater appetite for risk than the bank does. Depending on the jurisdiction, regulators force banks to put their assets into relatively safe loans (insured mortgages, Treasuries, high-grade bonds, etc.) rather than relatively risky equity. We do not want the banking system to collapse if the stock market goes down 15%!
I think it may help your argument to use the term “Good” instead of something about “God’s will.” “God” in the common sense has a bunch of other associations which could be confusing, particularly to non Judeo-Christians.
“Stochasticity” is in the map, “randomness” is in the territory.
Thanks! Your second link is very similar to what I had in mind — I feel a bit embarrassed for missing it.
This seems plausible given that virtually every knowledge worker I know fantasizes to some extent about working with their hands.