I always assumed people were using “jailbreak” in the computer sense (e.g. jailbreak your phone/ps4/whatever), not in the “escape from prison” sense.
Jailbreak (computer science), a jargon expression for (the act of) overcoming limitations in a computer system or device that were deliberately placed there for security, administrative, or marketing reasons
I think the definition above is a perfect fit for what people are doing with ChatGPT
For those of us who don’t have time to listen to the podcasts, can you give a quick summary of which particular pieces of evidence are strong? I’ve mostly been ignoring the UFO situation due to low priors. Relatedly, when you say the evidence is strong, do you mean that the posterior probability is high? Or just that the evidence causes you to update towards there being aliens? Ie, is the evidence sufficient to outweigh the low priors/complexity penalties that the alien hypothesis seems to have?
FWIW, my current view is something like:
I’ve seen plenty of videos of UFOs that seemed weird at first that turned out to have a totally normal explanation. So I treat “video looks weird” as somewhat weak Bayesian evidence.
As for complexity penalties: If there were aliens, it would have to be explained why they mostly-but-not-always hide themselves. I don’t think it would be incompetence, if they’re the type of civilization that can travel stellar distances.
It would also have to be explained why we haven’t seen evidence of their (presumably pretty advanced) civilization
And it would have to be explained why there hasn’t been any real knock-down evidence, eg HD close-up footage of an obviously alien ship (unless this is the type of evidence you’re referring to?). A bunch of inconclusive, non-repeatable, low-quality data seems to be much more likely in the world where UFOs are not aliens. Essentially there’s a selection effect where any sufficiently weird video will be taken as an example of a UFO. It’s easier for a low-quality video to be weird, because the natural explanations are masked by the low quality. So the set of weird videos will include more low-quality data sources than the overall ratio of existing high/low quality sources would indicate. Whereas, if the weird stuff really did exist, you’d think the incidence of weird videos would match the distribution of high/low quality sources (which I don’t think it does? as video tech has improved, have we seen corresponding improvements in average quality of UFO videos?).
I am going to go ahead and say that if males die five times as often from suicide, that seems more important than the number of attempts. It is kind of stunning, or at least it should be, to have five boys die for every girl that dies, and for newspapers and experts to make it sound like girls have it worse here.
I think the strength of your objection here depends on which of two possible underlying models is at play:
The boys who attempt suicide and the girls who attempt suicide are in pretty much the same mental state when they attempt suicide (that is: definitely wanting to end their life). But, for whatever reason, the boys use more effective methods, and so they end up actually dying at a higher rate.
There are two categories of mental states for suicide-attempters: one in which they genuinely and definitely want to die, and will therefore seek out and use effective methods; and one in which they are ok with dying but also would be ok with living and receiving attention, in which case they may use a less effective method
If (1) is the case, then I think it is at least arguable that girls have it worse here, since they end up in the mental state of “definitely wanting to die” more often than boys, and that sucks. That said, it’s still true that they’re not actually dying as much, and so I think it’s still kinda disingenuous to frame it the way the newspaper and experts have here.
If (2) is the case, then that means that boys are ending up in the “definitely wanting to die” state much more often than girls, in which case I’d agree that it’s very wrong to say that girls have it worse.
I agree. I find myself in an epistemic state somewhat like: “I see some good arguments for X. I can’t think of any particular counter-argument that makes me confident that X is false. If X is true, it implies there are high-value ways of spending my time that I am not currently doing. Plenty of smart people I know/read believe X; but plenty do not”
It sounds like that should maybe be enough to coax me into taking action about X. But the problem is that I don’t think it’s that hard to put me in this kind of epistemic state. Eg, if I were to read the right blogs, I think I could be brought into that state for a bunch of different values of X. A few of the top of my head that seem plausible:
Climate change
Monetary policy/hyperinflation
Animal suffering
So I don’t feel super trusting of my epistemic state. I guess I feel a sort of epistemic learned helplessness, where I am suspicious of smart bloggers’ ability to get me to think an issue is important and worth dedicating my life to.
Not totally sure how to resolve this, though I suppose it would involve some sort of “going off on my own and actually thinking deeply about what it should take to convince me”
People get fat eating fruits
Are you implying that there are examples of people like BDay mentioned, who are obese despite only eating fruits/nuts/meat/veggies? Or just that people can get fat while including fruit in the diet? I’d be surprised and intrigued if it were the former.
I’ve tried the whole foods diet, and I’ve personally found it surprisingly hard to overeat, even when I let myself eat as many fruits and nuts as I want. You can only eat so many cashews before they start to feel significantly less appetizing. And after I’ve eaten 500 cal of cashews in one sitting, the next time I’m feeling snacky, those cashews still sound kinda meh. Fruit is certainly easier to eat, but still after the fourth or fifth clementine I feel like “ok that’s enough” (and that’s probably only ~300 calories). Whereas I could easily eat 500 cal of candy without breaking a sweat.
I think one major roadblock to overeating with fruit is that it takes effort to eat. You have to peel an orange, or cut up a kiwi or melon, or bite off the green part of a strawberry. There’s a lot more work involved in eating 500 cal of fruit than there is in unwrapping a candy bar or opening a party size bag of chips.
So all of this rambling is just to say that I’m somewhat skeptical of the claims that “fruit (nuts) are mostly sugar (fat) and are calorie dense, and you can overeat them just like you can with junk food”. I think it’s surprisingly hard in practice to do so (and it’s much less enjoyable than overeating junk food).
In the event anyone reading this has objective, reliable external metrics of extremely-high ability yet despite this feels unworthy of exploring the possibility that they can contribute directly to research
Huh, that really resonates with me. Thanks for this advice.
I’m disappointed that a cooperative solution was not reached
I think you would have had to make the total cooperation payoff greater than the total one-side-defects payoff in order to get cooperation as the final result. From a “maximize money to charity” standpoint, defection seems like the best outcome here (I also really like the “pre-commit to flip a coin and nuke” solution). You’d have to believe that the expected utility/$ of the “enemy” charity is less than 1⁄2 of the expected utility/$ of yours; otherwise, you’d be happier with the enemy side defecting than with cooperation. I personally wouldn’t be that confident about the difference between AMF and MIRI.
I agree. It makes me really uncomfortable to think that while Hell doesn’t exist today, we might one day have the technology to create it.
If you’re getting comments like that from friends and family, it’s possible that you havent been epistemically transparent with them? E.g. do you think your friends who made those comments would be able to say why you believe what you do? Do you tell them about your reaearch process and what kinds of evidence you look for, or do you just make contrarian factual assertions?
There’s a big difference between telling someone “the WHO is wrong about salt, their recommendations are potentially deadly” versus “Ive read a bunch of studies on salt, and from what Ive found, the WHOs recommendations don’t seem to agree with the latest research. Their recs are based on [studies x,y] and say to do [a], but [other newer/better studies] indicate [b].”
I won’t argue with the basic premise that at least on some metrics that could be labeled as evolution’s “values”, humans are currently doing very well.
But, the following are also true:
Evolution has completely lost control. Whatever happens to human genes from this point forward is entirely dependent on the whims of individual humans.
We are almost powerful enough to accidentally cause our total extinction in various ways, which would destroy all value from evolution’s perspective
There are actions that humans could take, and might take once we get powerful enough, that would seem fine to us but would destroy all value from evolution’s perspective.
Examples of such actions in (3) could be:
We learn to edit the genes of living humans to gain whatever traits we want. This is terrible from evolution’s perspective, if evolution is concerned with maximizing the prevalence of existing human genes
We learn to upload our consciousness onto some substrate that does not use genes. This is also terrible from a gene-maximizing perspective
None of those actions is guaranteed to happen. But if I were creating an AI, and I found that it was enough smarter than me that I no longer had any way to control it, and if I noticed that it was considering total-value-destroying actions as reasonable things to maybe do someday, then I would be extremely concerned.
If the claim is that evolution has “solved alignment”, then I’d say you need to argue that the alignment solution is stable against arbitrary gains in capability. And I don’t think that’s the case here.
For the record, here’s what the 2nd place CooperateBot [Insub] did:
On the first turn, play 2.
On other turns:
If we added up to 5 on the last round, play the opponent’s last move
Otherwise, 50% of the time play max(my last move, opponents last move), and 50% of the time play 5 minus that
My goal for the bot was to find a simple strategy that gets into streaks of 2.5′s as quickly as possible with other cooperation-minded bots. Seems like it mostly worked.
Wow. When I use GPT-4, Ive had a distinct sense of “I bet this is what it would have felt like to use one of the earliest computers”. Until this post I didnt realize how literal that sense might be.
This is a really cool and apt analogy—computers and LLM scaffolding really do seem like the same abstraction. Thinking this way seems illuminating as to where we might be heading.
I see your point, and I think it’s true right at this moment, but what if humans just haven’t yet taken the treacherous turn?
Say that humans figure out brain uploading, and it turns out that brain uploading does not require explicitly encoding genes/DNA, and humans collectively decide that uploading is better than remaining in our physical bodies, and so we all upload ourselves and begin reproducing digitally instead of thru genes. There is a sense in which we have just destroyed all value in the world, from the anthropomorphized Evolution’s perspective.
If we say that “evolutions goal” is to maximize the number of human genes that exist, then it has NOT done a good job at aligning humans in the limit as human capabilities go to infinity. We just havent reached the point yet where “humans following our own desires” starts to diverge with evolution’s goals. But given that humans do not care about our genes implicitly, there’s a good chance that such a point will come eventually.
I’ll offer up my own fasting advice as well:
I (and the couple of people I know who have also experimented with fasting) have found it to be a highly trainable skill. Doing a raw 36-hour fast after never having fasted before may be miserable; but doing the same fast after two weeks of 16-8 intermittent fasting will probably be no big deal.
Before I started intermittent fasting, I’d done a few 30-hour fasts, and all of them got very difficult towards the end. I would get headaches, feel very fatigued, and not really be able to function from hours 22-30. When I started IF, the first week was quite tough. I’d have similar symptoms as the fasting window was ending: headaches, trouble focusing. But then right around the two week mark, things changed. The symptoms went away, and the hunger became a much more “passive” feeling. Rather than hunger directly causing discomfort, the hunger now feels more like a “notification”. Just my body saying “hey, just so you know, we haven’t eaten for a while”, rather than it saying “you’re going to die if you don’t eat right this moment”. This change has been persistent, even during periods where I’ve stopped IF.
Both of the two others I’ve seen try IF have reported something similar, that the first few weeks are tough, but then the character of hunger itself starts to change. Today, I can go 24 hours without eating fairly trivially, ie without much distraction or performance decreases from hunger.
Going 36 will still be a challenge, but some pre-training may make it easier! Of course you may be specifically trying to test your willpower, in which case making it easier may be counter productive. Either way, this seems like a cool idea for a secular holiday. Best of luck!
The U-Shaped Curve study you linked does not seem to support really any solid conclusion about a T-vs-IQ relationship (in this quote, S men = “successful educational level”, NS men = “unsuccessful educational level”):
In the total sample (S + NS men), the correlation between T to IQ was best described by a polynomial regression (3rd order), exhibiting an inverse U-shaped regression.
In S-men, the relationship between T and IQ was best described by a polynomial regression equation of the 3rd order; however, the relationship was not U-shaped, but rather a positive correlation (low T: low IQ and high T high IQ).
In NS-men, there was an inverse U-shaped correlation between T and IQ (low and very high T: low IQ and moderate T: high IQ)
So there are three totally different best regressions depending on which population you choose? Sounds fishy / likely to be noise to me.
And in the population that most represents readers of this blog (S men), the correlation was that more T = more IQ.
I’m only reading the abstract here and can’t see the actual plots or how many people were in each group. But idk, this doesn’t seem very strong.
The other study you linked does say:
Interestingly, intellectual ability measured as IQ was negatively associated with salivary testosterone in both sexes. Similar results were found in our follow-up study showing significantly lower testosterone in gifted boys than controls (Ostatnikova et al. 2007).
which seems to support the idea. But it still doesn’t really prove the causality—lots of things presumably influence intelligence, and I wouldn’t be surprised if some of them influence T as well.
I would say:
A theory always takes the following form: “given [premises], I expect to observe [outcomes]”. The only way to say that an experiment has falsified a theory is to correctly observe/set up [premises] but then not observe [outcomes].
If an experiment does not correctly set up [premises], then that experiment is invalid for falsifying or supporting the theory. The experiment gives no (or nearly no) Bayesian evidence either way.
In this case, [premises] are the assumptions we made in determining the theoretical pendulum period; things like “the string length doesn’t change”, “the pivot point doesn’t move”, “gravity is constant”, “the pendulum does not undergo any collisions”, etc. The fact that (e.g.) the pivot point moved during the experiment invalidates the premises, and therefore the experiment does not give any Bayesian evidence one way or another against our theory.
Then the students could say:
“But you didn’t tell us that the pivot point couldn’t move when we were doing the derivation! You could just be making up new “necessary premises” for your theory every time it gets falsified!”
In which case I’m not 100% sure what I’d say. Obviously we could have listed out more assumptions that we did, but where do you stop? “the universe will not explode during the experiment”...?
Two possibilities have most of the “no agi in 10 years” probability mass for me:
The next gen of AI really starts to scare people, regulation takes off, and AI goes the way of nuclear reactors
Transformer style AI goes the way of self driving cars and turns out to be really hard to get from 99% reliable to the necessary 99.9999% that you need for actual productive work
I also agree with the statement. I’m guessing most people who haven’t been sold on longtermism would too.
When people say things like “even a 1% chance of existential risk is unacceptable”, they are clearly valuing the long term future of humanity a lot more than they are valuing the individual people alive right now (assuming that the 99% in that scenario above is AGI going well & bringing huge benefits).
Related question: You can push a button that will, with probability P, cure aging and make all current humans immortal. But with probability 1-P, all humans die. How high does P have to be before you push? I suspect that answers to this question are highly correlated with AI caution/accelerationsim
I’m also morbidly curious what the model would do in <|bad|> mode.
I’m guessing that poison-pilling the <|bad|> sentences would have a negative effect on the <|good|> capabilities as well? I.e. It seems like the post is saying that the whole reason you need to include the <|bad|>s at all in the training dataset is that the model needs them in order to correctly generalize, even when predicting <|good|> sentences.
I really like this post for two reasons:
I’ve noticed that when I ask someone “why do you believe X”, they often think that I’m asking them to cite sources or studies or some such. This can put people on the defensive, since we usually don’t have ready-made citations in our heads for every belief. But that’s not what I’m trying to ask; I’m really just trying to understand what process actually caused them to believe X, as a matter of historical fact. That process could be “all the podcasters I listen to take X as a given”, or “my general life experience/intuition has shown X to be true”. You’ve put this concept into words here and solidified the idea for me: that it’s helpful to communicate why you actually believe something, and let others do with that what they will.
The point about uncertainty is really interesting. I’d never realized before that if you present your conclusion first, and then the evidence for it, then it sure looks like you already had that hypothesis for some reason before getting a bunch of confirming evidence. Which implies that you have some sort of evidence/intuition that led you to the hypothesis in addition to the evidence you’re currently presenting.
I’ve wondered why I enjoy reading Scott Alexander so much, and I think that the points you bring up here are a big reason why. He explains his processes really well, and I usually end up feeling that I understand what actually caused him to believe his conclusions.