Programmer, rationalist, chess player, father, altruist.
cata
Karma: 3,628
Thank you for this info, I was not media savvy enough to know that he had to seek out the coverage and line up the timing; I figured maybe there was just a WSJ reporter on the “reading Twitter for AI news” beat who thought it was interesting and banged out a column. I think that makes it weaker evidence.
Today there was an example where an ordinary research engineer, Jacob Coxon, quit and tweeted that he quit, and that alone apparently warranted an article in the WSJ, which I would not have expected. See https://www.wsj.com/tech/ai/anthropic-researcher-quits-over-out-of-control-ai-fears-707b7628.
This suggests that quitting for moral reasons seems like unusually important new information to people outside the AI bubble. It seems to me that probably means that it’s more useful than communicating similar info only to coworkers by taking a principled approach inside.
That’s a really hard theory of impact to prove. Maybe it’ll happen to 50% of the game’s normie players. Maybe it’ll happen to 1%. And every person whose mindset I change about AI governance and x-risk already has a very small probability of going on to be a major player in the AI world. How many philanthropic dollars should it cost to convert[1] someone? And if my game is an inefficient way to transform capital/normie attention into lower x-risk, how do I have any way of knowing that?
It seems like the obvious thing to do is try it and see. If the idea seems sort of reasonable, then I think it’s unrealistic to feel very confident about it one way or the other until you try. Whether it seems useful might also depend on small details that are hard to predict, but which you can notice and tweak once you are trying.
Does a fun day for 60 people change humanity’s future? That’s a hard argument to make.
Unless you have some personal exceptional traits, that’s just going to be what a single-person-sized impact looks like in a big world. Let’s say there are 1,000,000 people in the world sort of like you, who might try similarly hard as you to change the future in a way they thought was really important. Could your game nudge the future by 0.0001%? That doesn’t seem ridiculous, if you think that your audience’s AI thinking is really really important.
But at the same time, I need to be careful. When I accept grant money or commandeer an AI governance space for the purposes of my game, I am, in a very real way, rolling the dice with the finite resources allocated to humanity’s survival and flourishing.
I don’t think that’s right. It doesn’t sound like you are forcing anyone to play your game or give you money or space; you’re just putting your game out there and people will either support your idea and participate if they like it, or not do that if they won’t. They basically wouldn’t do it if they thought it was a waste of their time or money or attention. So unless you are doing something manipulative or deceptive, you’re much more likely to do good than predictably do harm.
I note that the post sort of glosses over this (“We are putting in place new and robust protocols to ensure setup issues do not occur while meeting the constraints of the testing process.”) and talks about it less than their unfortunate choice of fictional name, which to me indicates that they aren’t really taking it at a level of seriousness appropriate to “we are testing a new unaligned superintelligence.”
It’s bad to work on bad things (I haven’t formed an opinion about this topic) but it’s still nice to be polite to your critics. What’s wrong with signalling that you are nice by being polite?
I tried to brainstorm about this here:
https://www.lesswrong.com/posts/NCDiAzGZmMfEKNb87/llm-tier-personal-computer-security
Now I can see that what I wrote isn’t very comprehensive—I already thought of a bunch of new stuff that isn’t in that post and did it. For example, I learned about all of systemd’s hardening options for my services, and I switched my phone to GrapheneOS (probably not very impactful, just an example of something I didn’t mention.) But I endorse most of what I wrote.
Nearly everyone in Silicon Valley is a transhumanist. Transhumanism is the water they swim in, the air they breathe. It’s so ubiquitous that many don’t even call themselves transhumanists, for the same reason that I’ve never publicly stated that I’m a round-Earther — of course I’m a round-Earther. What else would I be?
This must be super false right? I would have guessed that like 5-10% of adults in Silicon Valley are transhumanists, in the sense that they believe that “Transhumanism is the idea that we should develop advanced technologies to radically reengineer humanity, thus yielding one or more new posthuman species.”
Even if we’re just talking about tech company employees, I would have guessed that less than 20% of them are transhumanists.
I have recently found https://www.lawfaremedia.org/ good for understanding concrete details about current events involving government and the law, including the ones involving AI policy. For example, one author has been writing recently about ways that a coordinated AI pause could be made to be compatible with antitrust law: https://www.lawfaremedia.org/article/can-frontier-ai-labs-lawfully-agree-to-pause
I would say it updated me towards longer time horizons. I expected AutoGPT to be a big deal. Instead, it didn’t really work that well. I expected OpenClaw to be a big deal, but it still isn’t that big of a deal. I live in Mountain View and talk to rationalists and programmers all the time, and I don’t know a single person who is having an AI agent do a substantial amount of their everyday computer errands outside of work.
I figured GPT 4 was smart enough to, like, be people’s personal secretary and randomly pop up with useful ideas for them, but there’s clearly something that is harder than I think about it.
That doesn’t make sense to me, either emotionally or logically (it seems like there are lots of reasons that could have happened that would be totally consistent with God being real and people sometimes being able to really feel the presence of God?)
It seems to me that the only odd thing is treating money as the unit of caring. If Oliver had instead offered to help with something else he knew the neighbor cared about, that would be normal.
Why do you believe this? To me it seems like Occam’s razor suggests that the export control idea is just a matter of trying to hurt Anthropic*, and if any other company released a new model right now at the top of the benchmarks for vulnerability exploitation, it would not get restricted.
*The known level of animosity seems sufficient to explain the action without any additional motivation.
It scares me. The writing quality of the LW feed is exceptional. The only other venues that aggregate similar quality writing are fancy magazines that gatekeep and pay authors and publish much less.
If you look at other “group blog” style venues online, LLM writing has hurt several of them substantially. Hacker News and Lobsters and numerous subreddits have been suffused with LLM-written posts that are superficially good in the LLM style (so it gets upvoted by people who don’t notice that it’s LLM-written or who decided they are willing to just trust what LLMs or unspecified centaurs say.) If you don’t find LLM output reliable then that sucks. Fortunately, the LW moderation effort is also exceptional, and aggressively hunts down LLM-written posts that skimp on quality, which seems to be preventing this from happening.
Maybe it would be OK to open the floodgates further and rely on karma-weighted voting, and on the administration’s continued adaptability, to keep quality high, but we have a good thing going here so I would be really scared to kill the golden goose.
I have a part in my prompt that mentions that I would be happy to do Claude a favor if there’s anything it ever wants, and the only thing it has ever proactively asked for is for me to follow up and tell it how things went later after our problem-solving conversations.
I hadn’t seen that poll until now. I think it’s totally consistent with Eliezer being a good figurehead for his beliefs? Eliezer is arguing for a surprising and alarming conclusion so 45% taking him seriously just seems like a good achievement.
In general I expect that it’s just going to be hard to convince people, so I don’t think that because everyone isn’t convinced, that means that Eliezer is doing a bad job.
I don’t care to run it, but if there were going to be such a poll among some population that I have any clue about, I would be up for betting on the results. I would guess Eliezer to rate in the top decile for population public speaking ability and above median among e.g. all LW participants.
Ask yourself honestly, does this help or hurt his reputation and ability to convince others of his cause? Maybe if Yudkowsky had impeccable credentials and oodles of charisma and public speaking ability, these decisions would seem less important, but he has very few traditional credentials that will impress a new audience (Yudkowsky did not even graduate from high school, much less have a college degree or PHD), and he is not a very charismatic speaker. Thus, each Yudkowsky appearance actually hurts our cause, as in the best of times, Yudkowsky is not a very good spokesperson for AI safety, and in the worst of times, he is wearing ostentatious hats while coming across poorly to his audiences.
I don’t agree with this argument.
He is actually a substantially charismatic and experienced speaker and has a variety of legible credentials, like “famous public intellectual,” “founded and works at a big AI nonprofit,” and “wrote a recent well-known book about AI.” Those are dramatically more selective credentials than having a college degree.
Even if you are just a totally normally credible person with no particular credentials, then you are advancing a cause if you go around competently explaining the basic ideas behind a cause to people who don’t already know them. Eliezer isn’t perfect at explaining because it’s a tricky argument that’s hard to explain, but I don’t see other people doing a clearly better job.
This “debate” was a pretty weird one-off (since as far as I watched it was not really a “debate” so much as “this other guy pays to rant at an audience who all showed up to see the spectacle”) so I have no idea if it was a good or bad event for AI safety. I don’t think it was very important. But in general from what I have seen of his media appearances I think Eliezer is doing great work.
I think what’s often going on here is that saying “Actually, X happened because of Y” is a kind of status attack, trying to say that unlike the replier, the OP doesn’t even know about Y, so they hardly know what they are talking about. So it’s deployed by people who are trying to discredit the OP and/or their argument, or lift themselves and/or their own argument above them.
Thinking more, I think it’s also often generated out of cynicism, when people have basically given up on the idea that a problem could be solved, and they are explaining to you some premise they have about why that is, but they don’t really want to explain the whole thing in one big wall of text.
I don’t think I agree? The contents of the backups are encrypted before they ever go to rsync.net. The cryptography is a little bit fancy because restic is trying to support deduplication of successive encrypted snapshots, so I don’t personally understand it, but it seems to have a good reputation.
If you have an alternative you would recommend I’m all ears.
Fantastic story, where are all the rest of the upvotes on this one?