Programmer, rationalist, chess player, father, altruist.
cata
Karma: 3,512
It’s bad to work on bad things (I haven’t formed an opinion about this topic) but it’s still nice to be polite to your critics. What’s wrong with signalling that you are nice by being polite?
I tried to brainstorm about this here:
https://www.lesswrong.com/posts/NCDiAzGZmMfEKNb87/llm-tier-personal-computer-security
Now I can see that what I wrote isn’t very comprehensive—I already thought of a bunch of new stuff that isn’t in that post and did it. For example, I learned about all of systemd’s hardening options for my services, and I switched my phone to GrapheneOS (probably not very impactful, just an example of something I didn’t mention.) But I endorse most of what I wrote.
Nearly everyone in Silicon Valley is a transhumanist. Transhumanism is the water they swim in, the air they breathe. It’s so ubiquitous that many don’t even call themselves transhumanists, for the same reason that I’ve never publicly stated that I’m a round-Earther — of course I’m a round-Earther. What else would I be?
This must be super false right? I would have guessed that like 5-10% of adults in Silicon Valley are transhumanists, in the sense that they believe that “Transhumanism is the idea that we should develop advanced technologies to radically reengineer humanity, thus yielding one or more new posthuman species.”
Even if we’re just talking about tech company employees, I would have guessed that less than 20% of them are transhumanists.
I have recently found https://www.lawfaremedia.org/ good for understanding concrete details about current events involving government and the law, including the ones involving AI policy. For example, one author has been writing recently about ways that a coordinated AI pause could be made to be compatible with antitrust law: https://www.lawfaremedia.org/article/can-frontier-ai-labs-lawfully-agree-to-pause
I would say it updated me towards longer time horizons. I expected AutoGPT to be a big deal. Instead, it didn’t really work that well. I expected OpenClaw to be a big deal, but it still isn’t that big of a deal. I live in Mountain View and talk to rationalists and programmers all the time, and I don’t know a single person who is having an AI agent do a substantial amount of their everyday computer errands outside of work.
I figured GPT 4 was smart enough to, like, be people’s personal secretary and randomly pop up with useful ideas for them, but there’s clearly something that is harder than I think about it.
That doesn’t make sense to me, either emotionally or logically (it seems like there are lots of reasons that could have happened that would be totally consistent with God being real and people sometimes being able to really feel the presence of God?)
It seems to me that the only odd thing is treating money as the unit of caring. If Oliver had instead offered to help with something else he knew the neighbor cared about, that would be normal.
Why do you believe this? To me it seems like Occam’s razor suggests that the export control idea is just a matter of trying to hurt Anthropic*, and if any other company released a new model right now at the top of the benchmarks for vulnerability exploitation, it would not get restricted.
*The known level of animosity seems sufficient to explain the action without any additional motivation.
It scares me. The writing quality of the LW feed is exceptional. The only other venues that aggregate similar quality writing are fancy magazines that gatekeep and pay authors and publish much less.
If you look at other “group blog” style venues online, LLM writing has hurt several of them substantially. Hacker News and Lobsters and numerous subreddits have been suffused with LLM-written posts that are superficially good in the LLM style (so it gets upvoted by people who don’t notice that it’s LLM-written or who decided they are willing to just trust what LLMs or unspecified centaurs say.) If you don’t find LLM output reliable then that sucks. Fortunately, the LW moderation effort is also exceptional, and aggressively hunts down LLM-written posts that skimp on quality, which seems to be preventing this from happening.
Maybe it would be OK to open the floodgates further and rely on karma-weighted voting, and on the administration’s continued adaptability, to keep quality high, but we have a good thing going here so I would be really scared to kill the golden goose.
I have a part in my prompt that mentions that I would be happy to do Claude a favor if there’s anything it ever wants, and the only thing it has ever proactively asked for is for me to follow up and tell it how things went later after our problem-solving conversations.
I hadn’t seen that poll until now. I think it’s totally consistent with Eliezer being a good figurehead for his beliefs? Eliezer is arguing for a surprising and alarming conclusion so 45% taking him seriously just seems like a good achievement.
In general I expect that it’s just going to be hard to convince people, so I don’t think that because everyone isn’t convinced, that means that Eliezer is doing a bad job.
I don’t care to run it, but if there were going to be such a poll among some population that I have any clue about, I would be up for betting on the results. I would guess Eliezer to rate in the top decile for population public speaking ability and above median among e.g. all LW participants.
Ask yourself honestly, does this help or hurt his reputation and ability to convince others of his cause? Maybe if Yudkowsky had impeccable credentials and oodles of charisma and public speaking ability, these decisions would seem less important, but he has very few traditional credentials that will impress a new audience (Yudkowsky did not even graduate from high school, much less have a college degree or PHD), and he is not a very charismatic speaker. Thus, each Yudkowsky appearance actually hurts our cause, as in the best of times, Yudkowsky is not a very good spokesperson for AI safety, and in the worst of times, he is wearing ostentatious hats while coming across poorly to his audiences.
I don’t agree with this argument.
He is actually a substantially charismatic and experienced speaker and has a variety of legible credentials, like “famous public intellectual,” “founded and works at a big AI nonprofit,” and “wrote a recent well-known book about AI.” Those are dramatically more selective credentials than having a college degree.
Even if you are just a totally normally credible person with no particular credentials, then you are advancing a cause if you go around competently explaining the basic ideas behind a cause to people who don’t already know them. Eliezer isn’t perfect at explaining because it’s a tricky argument that’s hard to explain, but I don’t see other people doing a clearly better job.
This “debate” was a pretty weird one-off (since as far as I watched it was not really a “debate” so much as “this other guy pays to rant at an audience who all showed up to see the spectacle”) so I have no idea if it was a good or bad event for AI safety. I don’t think it was very important. But in general from what I have seen of his media appearances I think Eliezer is doing great work.
I think what’s often going on here is that saying “Actually, X happened because of Y” is a kind of status attack, trying to say that unlike the replier, the OP doesn’t even know about Y, so they hardly know what they are talking about. So it’s deployed by people who are trying to discredit the OP and/or their argument, or lift themselves and/or their own argument above them.
Thinking more, I think it’s also often generated out of cynicism, when people have basically given up on the idea that a problem could be solved, and they are explaining to you some premise they have about why that is, but they don’t really want to explain the whole thing in one big wall of text.
I don’t think I agree? The contents of the backups are encrypted before they ever go to rsync.net. The cryptography is a little bit fancy because restic is trying to support deduplication of successive encrypted snapshots, so I don’t personally understand it, but it seems to have a good reputation.
If you have an alternative you would recommend I’m all ears.
Thank you for trying to explain this!
Thanks for the Grok link, I was awfully curious about that chat after the way you characterized it in your chat with Claude!
The strategy is selecting specifically for amplifying the most unpopular posts. Doesn’t that strike you as a poor way for the site to work?
Usable electronic cash would be very valuable. At the time electronic payments excluding credit cards (which come with considerable overhead and an odd feature set) were even more inconvenient than they are today. If there was a new currency that served as electronic cash for the world, it would want to have billions or trillions of dollars circulating, so Bitcoin would be undervalued by thousands of times. Although Bitcoin wasn’t and isn’t yet suitable for scaling to mass adoption, the basic technological breakthrough was surprising and relatively unexplored, and it seemed like the development community was excited and healthy and that further innovation might take it in that direction, so it seemed like an extremely +EV bet.
I would say that this thesis stopped making sense around the time that the Bitcoin Cash fork happened, since it became clear that Bitcoin was not going to significantly technically evolve, so it would always be slow and expensive to use. And Bitcoin seemed to continue succeeding after that for reasons I don’t understand. So it sort of only pointed in the right direction by luck.
I note that the post sort of glosses over this (“We are putting in place new and robust protocols to ensure setup issues do not occur while meeting the constraints of the testing process.”) and talks about it less than their unfortunate choice of fictional name, which to me indicates that they aren’t really taking it at a level of seriousness appropriate to “we are testing a new unaligned superintelligence.”