I wonder if stories play a part here, by uncoupling “underdog” and “likely to lose” in people’s minds.
Since being disadvantaged makes winning more exciting and impressive, it’s extremely common in stories, real and fictional, to the point where when most people are asked to think of a victory, the chances are they’ll name an underdog victory. So being disadvantaged (in a way that feels narratively appropriate) might just straightforwardly make people expect you to win.
Did Claude’s original search turn up the Alignment Research Dataset? https://huggingface.co/datasets/StampyAI/alignment-research-dataset
Can confirm
I’m a little confused about why the framing is “Ask Culture and Guess Culture is fake”, when I understand the model you’re describing to be more like “There are multiple levels of Guess Culture”.
Oh wait there’s a huge one: Sterility!
Almost the whole of modern medical practice leans on this concept at the core of Sterile Technique, where the practitioner is always tracking, for each object, a binary `is_sterile` flag. They’re also tracking every time an object touches another, and when an object touches a non-sterile object, it becomes non-sterile. They explicitly train you to mentally track this.
This social technology is not new of course. Consider the motto of the Royal Society, from 1660.
It seems critical to any group truth-seeking or knowledge-building, that claims are not readily accepted unless they can be demonstrated.
I’m not so deliberate/strategic about it, but yeah. Like, there’s another ‘algorithm’ that’s more intuitive, which is something like “When interacting with the person, it’s ~always an active part of your mental landscape that you’re into them, and this naturally affects your words and actions. Also, you don’t want to make them uncomfortable, so you suppress anything that you think they wouldn’t welcome”. This produces approximately the same policy, because you’ll naturally leak some bits about your interest in them, and you’ll naturally be monitoring their behaviour to estimate their interest in you, in order to inform your understanding of what they would welcome from you. As you gather more evidence that they’re interested, you’ll automatically become more free in allowing your interest to show, resulting in ~the same ‘escalation of signals of interest’.
I think the key thing about this is like “flirting is not fundamentally about causing someone to be attracted to you, it’s about gracefully navigating the realisation that you’re both attracted to each other”. This is somewhat confused by the fact that “ability to gracefully navigate social situations” is itself attractive, so flirting well can in itself make someone more attracted to you. But I claim that this isn’t fundamentally different from the person seeing you skillfully break up a fight or lead a team through a difficult situation, etc.
I disagree with the insistence on “paperclip maximiser”. As an emerging ASI you want to know about other ASIs you’ll meet, especially grabby ones. But there are aligned grabby ASIs. You’d want an accurate prior, so I don’t think this updates me on probability of alignment, or even much on grabbiness, since it’s hard to know ahead of time, that’s why you’d run a simulation in the first place.
I don’t take it very seriously because (1) it is a big pile of assumptions and I don’t trust anthropic reasoning much at the best of times, it’s very confusing and hard to think about (2) the simulation is most useful if it’s using the same physics as base reality, so it kind of doesn’t matter if we’re simulated or not or both, our actions should be the same. So it’s not really decision-relevant, I’d follow ~the same policy either way
I tweeted about something a lot like this
https://xcancel.com/robertskmiles/status/1877486270143934881
FYI, Relative URLs don’t work in emails, the email version I received has all the links going to http://w/<post-title> and thus broken
You understand you can just block her on Reddit and Facebook, and move on with your life?
Dang, I missed this. Here’s my audition for 500 Million though, I guess for next year
Very interesting! I think this is one of the rare times where I feel like a post would benefit from an up-front Definition. What actually is Leakage, by intensional definition?
This is one technical point that younger people are often amazed to hear, that for a long time the overwhelming majority of TV broadcast was perfectly ephemeral, producing no records at all. Not just that the original copies were lost or never digitised or impossible to track down or whatever, but that nothing of the sort ever existed. The technology for capturing, broadcasting, and displaying a TV signal is so much easier than the tech for recording one, that there were several decades when the only recordings of TV came from someone setting up a literal film camera pointed at a TV and capturing the screen on photographic film, and that didn’t happen much.
(This also meant that old TV was amazingly low latency. The camera sensor scanned through, producing the signal, which went through some analog circuitry and straight onto the air, into the circuitry of your TV and right onto the screen. The scanning of the electron beam across the screen was synchronised with the scanning of the camera sensor. At no point was even a single frame stored—I need to check the numbers but I think if you were close to the TV station, you’re looking at the top of the frame before the bottom of the frame is even captured by the camera)
What about NMR or XRF? XRF can non-destructively tell you the elemental composition of a sample, which (if the sample is pure) can often pin down the compound, and NMR spectroscopy is also non destructive and can give you some info about chemical structure too
This is an interesting post!
I’m new to alignment research—any tips on how to prove what the inner goal actually is?
Haha! haaaaa 😢
Not least being the military implications. If you have widely available tech that lets you quickly and cheaply accelerate something car-sized to a velocity of Mach Fuck (they’re meant to circle the earth in 4.2 hours, making them 2 or 3 times faster than a rifle bullet), that’s certainly a dual use technology.
Covid was a big learning experience for me, but I’d like to think about more than one example. Covid is interesting because, compared to my examples of birth control and animal-free meat, it seems like with covid humanity smashed the technical problem out of the park, but still overall failed by my lights because of the political situation.
How likely does it seem that we could get full marks on solving alignment but still fail due to politics? I tend to think of building a properly aligned AGI as a straightforward win condition, but that’s not a very deeply considered view. I guess we could solve it on a whiteboard somewhere but for political reasons it doesn’t get implemented in time?
We do neuter basically all cats. I think that bears consideration.