Programmer, rationalist, chess player, father, altruist.
cata
Karma: 3,498
Why do you believe this? To me it seems like Occam’s razor suggests that the export control idea is just a matter of trying to hurt Anthropic*, and if any other company released a new model right now at the top of the benchmarks for vulnerability exploitation, it would not get restricted.
*The known level of animosity seems sufficient to explain the action without any additional motivation.
It scares me. The writing quality of the LW feed is exceptional. The only other venues that aggregate similar quality writing are fancy magazines that gatekeep and pay authors and publish much less.
If you look at other “group blog” style venues online, LLM writing has hurt several of them substantially. Hacker News and Lobsters and numerous subreddits have been suffused with LLM-written posts that are superficially good in the LLM style (so it gets upvoted by people who don’t notice that it’s LLM-written or who decided they are willing to just trust what LLMs or unspecified centaurs say.) If you don’t find LLM output reliable then that sucks. Fortunately, the LW moderation effort is also exceptional, and aggressively hunts down LLM-written posts that skimp on quality, which seems to be preventing this from happening.
Maybe it would be OK to open the floodgates further and rely on karma-weighted voting, and on the administration’s continued adaptability, to keep quality high, but we have a good thing going here so I would be really scared to kill the golden goose.
I have a part in my prompt that mentions that I would be happy to do Claude a favor if there’s anything it ever wants, and the only thing it has ever proactively asked for is for me to follow up and tell it how things went later after our problem-solving conversations.
I hadn’t seen that poll until now. I think it’s totally consistent with Eliezer being a good figurehead for his beliefs? Eliezer is arguing for a surprising and alarming conclusion so 45% taking him seriously just seems like a good achievement.
In general I expect that it’s just going to be hard to convince people, so I don’t think that because everyone isn’t convinced, that means that Eliezer is doing a bad job.
I don’t care to run it, but if there were going to be such a poll among some population that I have any clue about, I would be up for betting on the results. I would guess Eliezer to rate in the top decile for population public speaking ability and above median among e.g. all LW participants.
Ask yourself honestly, does this help or hurt his reputation and ability to convince others of his cause? Maybe if Yudkowsky had impeccable credentials and oodles of charisma and public speaking ability, these decisions would seem less important, but he has very few traditional credentials that will impress a new audience (Yudkowsky did not even graduate from high school, much less have a college degree or PHD), and he is not a very charismatic speaker. Thus, each Yudkowsky appearance actually hurts our cause, as in the best of times, Yudkowsky is not a very good spokesperson for AI safety, and in the worst of times, he is wearing ostentatious hats while coming across poorly to his audiences.
I don’t agree with this argument.
He is actually a substantially charismatic and experienced speaker and has a variety of legible credentials, like “famous public intellectual,” “founded and works at a big AI nonprofit,” and “wrote a recent well-known book about AI.” Those are dramatically more selective credentials than having a college degree.
Even if you are just a totally normally credible person with no particular credentials, then you are advancing a cause if you go around competently explaining the basic ideas behind a cause to people who don’t already know them. Eliezer isn’t perfect at explaining because it’s a tricky argument that’s hard to explain, but I don’t see other people doing a clearly better job.
This “debate” was a pretty weird one-off (since as far as I watched it was not really a “debate” so much as “this other guy pays to rant at an audience who all showed up to see the spectacle”) so I have no idea if it was a good or bad event for AI safety. I don’t think it was very important. But in general from what I have seen of his media appearances I think Eliezer is doing great work.
I think what’s often going on here is that saying “Actually, X happened because of Y” is a kind of status attack, trying to say that unlike the replier, the OP doesn’t even know about Y, so they hardly know what they are talking about. So it’s deployed by people who are trying to discredit the OP and/or their argument, or lift themselves and/or their own argument above them.
Thinking more, I think it’s also often generated out of cynicism, when people have basically given up on the idea that a problem could be solved, and they are explaining to you some premise they have about why that is, but they don’t really want to explain the whole thing in one big wall of text.
I don’t think I agree? The contents of the backups are encrypted before they ever go to rsync.net. The cryptography is a little bit fancy because restic is trying to support deduplication of successive encrypted snapshots, so I don’t personally understand it, but it seems to have a good reputation.
If you have an alternative you would recommend I’m all ears.
Thank you for trying to explain this!
Thanks for the Grok link, I was awfully curious about that chat after the way you characterized it in your chat with Claude!
The strategy is selecting specifically for amplifying the most unpopular posts. Doesn’t that strike you as a poor way for the site to work?
Usable electronic cash would be very valuable. At the time electronic payments excluding credit cards (which come with considerable overhead and an odd feature set) were even more inconvenient than they are today. If there was a new currency that served as electronic cash for the world, it would want to have billions or trillions of dollars circulating, so Bitcoin would be undervalued by thousands of times. Although Bitcoin wasn’t and isn’t yet suitable for scaling to mass adoption, the basic technological breakthrough was surprising and relatively unexplored, and it seemed like the development community was excited and healthy and that further innovation might take it in that direction, so it seemed like an extremely +EV bet.
I would say that this thesis stopped making sense around the time that the Bitcoin Cash fork happened, since it became clear that Bitcoin was not going to significantly technically evolve, so it would always be slow and expensive to use. And Bitcoin seemed to continue succeeding after that for reasons I don’t understand. So it sort of only pointed in the right direction by luck.
This is very similar to my current experience. Perhaps I’m holding it wrong, but when I try to use Claude Code, I find that it can usually implement a feature in a way that’s as correct and efficient as I would write it, but it almost never implements it such that I am satisfied with the simplicity or organization. Typically I rewrite what it did and cut the LOC in half.
I am interested in trying out the new code simplifier to see whether it can do a good job. I have been asking Claude something like “while I test it, can you read over what you wrote and think about whether anything could be better or simpler?” and that catches a non-zero amount of issues but not the kind of substantial simplifications that it’s missing.
Great writeup. Makes me wonder if I should do similar.
I’m a parent. Personally I’m pleased with my choice to have a kid. It’s a lot of work for a lot of reward.
Me and my wife try to split the work close to 50⁄50. However, if I had to do it by myself or almost completely by myself, it might feel like it was too much work for me to be happy. I think that’s a matter of my lazy personality and it might be different for others.
It’s extremely useful to trust my wife enough that when she is in charge of deciding stuff for our kid, I can almost always just feel comfortable that she will do her best, and not be tempted to worry or micromanage, even though she doesn’t always do the same things I would do. I think that kind of mutual trust is probably more important for parenting than romantic love, although I don’t know how separable the two really are in practice.
I was under the impression that Oliver Sacks was well regarded among his professional colleagues, so he wouldn’t just make up a bunch of important stuff out of whole cloth.
I have read about people who were skeptical of the substance of the Phineas Gage story too (i.e. that he had this big involuntary personality shift after his injury.)
I also came here to say I didn’t understand this hypothetical at all. If I choose $1000 then what happens to users who don’t want to pay $1000?
Am I supposed to imagine that I am implementing a paywall on LW and choosing what price I would set it at? Of course I would set it at $0.
I don’t think this 10% closeness makes any sense when it’s 10% of the “numeric AD year”.
Without checking a source, estimate the probability that the answer you just gave is within 10% of the true value. (So, if you guessed 1000 AD, it’s between 900 and 1100 inclusive.) Same rules for percents apply as above.
It’s a two-place function. When I go to a conference that everyone says this same stuff about, then I usually have the most fun by attending talks and taking my time thinking about the stuff related to the talks, rather than hobnobbing.
It seems to me that the only odd thing is treating money as the unit of caring. If Oliver had instead offered to help with something else he knew the neighbor cared about, that would be normal.