15 miles north of San Francisco. hruvulum@gmail.com
My probability that AI research will end all human life is .93. Last updated 13 Aug 2023.
Currently I am willing to meet with almost anyone on the subject of AI extinction risk.
RHollerith
Karma: 2,425
What I want to be notified about is signs (e.g., satellite photos) that Russia or China is evacuating its cities.
Also, any threat by Russia or China to attack the US with nukes if Washington does not take specific action X before date Y.
Russia’s using a nuke in Ukraine wouldn’t increase my P(nuclear attack on the US) enough to cause me to relocate to a rural area, but those 2 other things would.
This channel seems pretty good:
Before the internet became a “mass medium” (i.e., before 1993) it was drastically less agreeable (and drastically less extroverted) than it is today. The difference between then and now is absolutely huge.
If I needed to find disagreeable people today (that I didn’t already know), I’d hang around in my county’s law library or maybe the Clerk of the Court’s office.
We have yet to touch on the topic of timing: comedians who perform in front of an audience often say that timing is important.
There can be a delay of a few seconds between punchline and the start of the laughter, and once the laughter begins, it usually gets loud very quickly. This behavior suggests that there is a (social?) cost to laughing at something most of the audience choose not to laugh at and also probably a cost to not laughing when most are laughing.
Destroying the fabric of the universe sounds hard even for a superintelligence. “hard”: probably impossible even if the superintelligence makes it its only priority.
In one of his appearances on video this year, Eliezer said IIRC that all of the intent-alignment techniques he knows of stop working once the AI’s capabilities improve enough, mentioning RLHF. Other than that I am not knowledgeable enough to answer you.
The people who coined the term “AI alignment” were worried about AI research’s causing the end of humanity. The source of your confusion seems to be the fact that the term has taken on other meanings. So let us take a step back and look at the original meaning of “the race between capabilities research and alignment research”.
None of the AIs so far deployed are a danger to the survival of the human species because they’re not capable enough. For example, although it is much more capable than a human or a team of humans at Go and Chess, Alpha Zero does not know that, e.g., it is running on a computer and does not know that there are people who could at any moment decide to shut that computer off, which would severely curtail its ability to win the chess game is it currently playing. It’s lack of knowledge of reality means Alpha Zero is not a danger to our survival. GPT-4 has vast knowledge of reality, and if you ask it to make a plan, it will make a plan, but there’s a good chance that the plan has a bad flaw in it. This unreliability in its planning capability means that GPT-4 is not a danger to our survival.
Soon the AI labs will create an AI that is much more capable at every important endeavor than our most capable institutions (e.g., Cambridge University or the FBI) as well of course as our most capable individuals. If that AI does not care about human survival or human preferences, then we are done for, but no one knows (and no one even has a good plan for finding out) how to make an AI care about us even a little tiny bit. MIRI has been trying to figure it out for about 20 years, and they’re telling us that maybe they can do it given another 3 or 4 decades, but we probably don’t have 3 or 4 decades—unless by some miracle all the AI labs get shut down.
Repeating myself: the only thing preventing AI from accidentally or intentionally wiping us out is the fact that so far no AI has the necessary capabilities (e.g., in making plans that can survive determined human opposition, e.g., in knowledge of reality), but the AI labs are bent on giving AI the relevant capabilities despite the fact that they don’t have any good plan (nor even a halfway-decent plan) for ensuring the first AI with the capabilities to wipe us out cares enough about us to refrain from doing so. (Once an AI becomes capable enough to kill us all, the only way it can be made safe is to make it care about us right from when the AI is first turned on, i.e., in its initial design.)
AI alignment is often presented as conceptually distinct from capabilities. However, the distinction seems somewhat fuzzy
From my point of view, the distinction is not fuzzy at all, and I hope this comment has enabled you to see my point of view: namely, our civilization is getting pretty good a capabilities research, but progress in alignment research has proved much more difficult. Here I am using “alignment” in its original, “strict” sense to refer only to methods that continue to work even after the AI become much more capable than people and human organizations.
I’ve had these kinds of thoughts [referring to the OP] many times. I don’t think they were very healthy for me
I suspect that they are unhealthy for a lot of people.
Jordan Peterson asserts that many mass murders are motivated by desire for revenge against the Universe and offers as evidence the observation that many mass murders choose the most innocent victims they can. (More precisely, since Peterson is religious, he has “revenge against God”, but the nearest translation for non-religious people would be “revenge against the Universe”.) And of course for a person to accumulate a large pool of vengeful feelings, the person must believe or at least strongly suspect that the target of the feelings is responsible for the bad things that happened to the person.
I cannot recall whether Peterson ever explicitly said so, but I am left with the impression that Peterson considers vengeful feelings directed at the Universe (and possibly also other large collective entities, e.g., the Cambodian nation) to be more likely to accumulate to the extent that they blot out the ability to think rationally in the pursuit of one’s own self-interest than vengeful feeling directed at a mere person or clique of persons. (Harboring vengeful feelings directed at the Cambodian nation would be particularly harmful to you if you were Cambodian or lived in Cambodia.)
The statements strike me as more credible and more interesting than they would be delivering in the speaking style of the people who usually talk on the topic, but then it is no surprise that winners of presidential elections have compelling vocal skills.
I sometimes imagine that making it so that anyone who works for or invests in an AI lab is unwelcome at the best Bay Area parties would be a worthwhile state of affairs to work towards, which is sort of along the same lines as you write.
There is a questionable trend to equate ML skills with the ability to do alignment work.
Yes!
I was going to upvote this (for reporting what LeCun said on Twitter) till I got to “I asked ChatGPT 3.5 to criticize this”.
According to my notes, what got me to resolve to avoid water fasting is the first .66 of this next interview with longevity researcher Valter Longo:
https://thedoctorskitchen.com/podcasts/62-fasting-and-medicine-with-prof-valter-longo
You can avoid the 90-second ads on that page by using yt-dlp to download the interview audio.
The next paragraph in my notes is the next URL, which describes what I replaced water fasts with, namely the fasting-mimicking diet.
https://kahn642.medium.com/265fc68f8e19
But the usual purpose of the fasting-mimicking diet is not fat loss. (It is autophagy.) It’s low on protein. So maybe that last URL is irrelevant to you.
Please let me know if there’s anything else I can do for you. You seem to have the potential to contribute to the desperate fight to save the world from AI, so I want you as healthy as possible.
It definitely is easier to stop eating completely! A water fast trades convenience against a significant risk of permanent damage (e.g., never regaining all the muscle you lost) or death.
Weight lifting during a water fast will not help
I agree! I was using the obvious unsuitability of strength training during a water fast as an argument against the water fast relative to the other ways to burn fat. (Weight lifting plus eating enough protein often enough is better at preserving muscle mass during an attempt to lose fat than eating enough protein often enough without the weight lifting.)
Yes. I basically cook everything I eat from scratch. I don’t eat any seeds or any fats or oils except for coconut, avocado, olive oil and fat from cow’s milk and lamb’s meat.
Hmm. I was going to write, “cows and lambs which I know not to have been fed seed oils,” but on second thought I do not know that to be the case. In particular, I use Kerrygold butter, which promises to be from cows fed on at least 95% grass, but as far as I know, the remaining 5% could include a large dose of seed oils. Kerrygold melts or more precisely gets soft at a much lower temperature than another brand of butter that claims to be 100% grass-fed, which means that the fatty-acid composition is much different than the other butter. The addition of seed oil to the cows diet could explain the difference.
My BMI is under 25. My motivation in entering this conversation is to try to talk you out of the water fast, not to learn how I might lose fat.
Water fasting strikes me as an inefficient way to do what you want to do. Sure, obviously, if you keep on getting no calories, eventually the body is going to burn fat, but paradoxically it prefers to burn protein from the muscles first (!) and the only way to stop that is with sufficiently intense exercise of the muscles, i.e., weight lifting, while trying to lose fat.
This next Andrew Huberman lecture describes how to burn fat via “using cold to create shiver”, exercise, “non-exercise movements such as fidgeting”, supplements and prescription drugs. I don’t recall any mention of fasting in this lecture though it has been a few months since I listened to the lecture. To be precise, because I would not have been surprised to hear Huberman warn against water fasting, I probably would not have remembered that, but I would’ve been quite surprised to hear him recommend it, and almost certainly would’ve remembered that. (So, he almost certainly does not recommend it in the lecture.)
https://www.youtube.com/watch?v=GqPGXG5TlZw
(Sadly, to burn fat using exercise, you have to exercise continuously for about 90 minutes IIRC.)
I did 6.5 days of water fasting once out of ignorance of the risks and disadvantages, and I’ll never voluntarily go that long again without protein and without calories.
There’s a natural human tendency to believe that if one is basically healthy, one’s health interventions should be mild whereas if one is severely ill, drastic health interventions should be chosen. In reality, for most cases severe chronic illness, it takes great expertise or a heroic efforts of rationality sustained usually over years to identify or imagine any intervention that has more than a negligible chance of positively affecting the illness, and people dealing with severe illness should spend a significant fraction of their thinking time and mental energy on avoiding making the situation worse.
Happily for you, there are probably people with very deep expertise on fat loss although I don’t know enough about the subject to tell you who those people are. (And the fact that the drug companies hope to make a lot of money on fat loss makes it much harder to identify the deep experts.)
I personally have had 2 friends who’ve killed themselves by being overly aggressive about trying to rid themselves of chronic illness: one was being prescribed an anti-coagulant and either took more than he should have because of a strong desire to return to his previous healthy lifestyle; the other was enamored of smart drugs, and one of the many combinations of drugs he tried gave him Parkinsonism.
I’ve stayed completely the hell away from seed oils for decades: it’s just your plan for a long water fast that alarms me.
We cannot follow that link into Gmail unless you give us your Gmail username and password.
Yeah, but even if the advice VCs give to people in general is worthless, it remains the case that (like Viliam said) once the VC has invested, its interests are aligned with the interests of any founder whose utility function grows linearly with money. And VCs usually advise the startups they’ve invested in to try for a huge exit (typically an IPO).
I’m curious why you think that.
My probability that there will be any human alive 100 years from now is .07. If MIRI were magically given effective control over all AI research starting now or if all AI research were magically stopped somehow, my probability would change to .98.
Is what I just wrote also badly misleading? Surely your objection is not to ambiguity in the definition of “alive”, but I’m failing to imagine what your objection might be unless we think very differently about how probability works.
(I’m interested in answers from anyone.)