(Also, my opposition to mainstream morality diverges from normal five-year-old level ethics in basically the opposite direction from gigabrain EA/AI safety takes. Compared to the five-year-old, I lean even more heavily toward saying true important things, and even less toward leading Molochian parades in exchange for nominal authority.)
johnswentworth
Karma: 64,472
My knee-jerk reaction here is “have you tried asking a five year old?”. Spelling it out in more words: for most of these questions there is an obvious Right Thing To Do. Say true things, spread true important things, don’t lie or strategically hide your views, don’t accelerate capabilities, don’t found capabilities companies, don’t bullshit. Just normal five-year-old level ethics.
And look, I am explicitly on the record saying Value Judgements Are Usually Bullshit and Human Values ≠ Goodness and just generally opposing mainstream morality. But even I can look at the track record of EA and AI Safety and say “look guys, you have not been outperforming the Obvious Right Thing To Do with respect to AI, you should stop with the gigabrain takes and just do the Obvious Right Things”.
Y’know, I did eventually get a smartphone. But even today, I use my phone almost exclusively for maps and private messaging/calls. I have zero social media apps on it, and view zero shortform video content on it. And I think that’s basically the right attitude.
Drawing the analogy to LLMs: there are some narrow use-cases for which LLMs are great. Even two years ago, they were already great as a search engine alternative, including generating simple StackOverflow-style code snippets. In the last few months, they’ve also become viable for generating nontrivial numerical code and (sometimes, with a fair bit of scaffolding and Lean) nontrivial proofs. These use-cases are to LLMs what maps and direct messaging are to smartphones.
… but then there’s also a whole slew of other LLM use-cases which are more analogous to social media or shortform video on smartphones. And those use-cases I still avoid like the plague. Much like both social media and shortform video, they seem addictive and actively mind-killing. As a rough heuristic, if it feels like the LLM is thinking with you or for you, then that’s usually a use-case to avoid scrupulously.
(You might wonder if using LLMs for proofs feels like they’re thinking with/for me, but mostly it feels to me like they’re throwing prodigious amounts of shit at the wall and seeing what sticks, with pretty little thinking-of-the-sort-I-do at all. The power of Lean verification is that it makes that sort of brute-force-y search a viable method for finding proofs.)
LLMs Are Starting To Noticeably Accelerate Our Work
@Vladimir_Nesov I’d be interested in your take on the question of how soon there will be fully open Mythos-level models.
Had to think on it a bit, but I buy it. This comment helped some things click. Thank you!
In every single other domain in which people lead and follow, a follower pretending to fall asleep would absolutely not be a signal encouraging the leader to lead more. It would be a strong signal that the follower does not like what the leader is doing and is opting out of the lead/follow relationship.
Generalizable point: too much of what I see in practice does not actually look like lead/follow; that model fits some of the data, but does not explain all or even most of the stuff which the deep nonconsent model explains.
People being weak, non-agentic, etc, predicts that social equilibria will be non-ideal somehow, but it doesn’t predict which of the many different non-ideal social equilibria we see. Why are mating/dating norms weird in the particular ways we see, rather than some other ways? In particular, there are certain patterns of mating/dating norms which both recur across many societies and seem nonideal (like e.g. people relying on subtle signals), so there must be some systematic underlying causes of those particular norms. Why those particular norms?
I don’t think this answers the puzzle which the post attempted to answer. (Which is not to say the post answered the puzzle correctly, but it did give any answer to the puzzle at all.)
In approximately every other domain of human interaction, people just say what they want. The puzzle is, roughly: why is courtship/seduction specifically different? Why do people not just say what they want, in this domain, while they do in basically every other domain? Why the subtle dance? What is the underlying generator of that difference?
Even prior to Fable, Claude Code had reached the point of accelerating our coding work dramatically. That was a pretty recent development; prior to the last few months, LLM coding contributions were relatively marginal for the sort of coding problems we work on. Even now, Claude Code has mediocre taste, and auto mode would be pretty bad for our research coding work. (Though it is great for more standard cookie-cutter coding projects.) But it can bang out simple empirical tests well, and test basically what we asked for most of the time.
Other than that, it is still mostly Google Search Plus Plus.
Visioning: Concretely Imagining What You Want
I dunno, man, one galaxy seems kinda small ;)
If you were sexually attracted to children, would you be able to admit it to yourself? If you would not be able to admit it to yourself, then you don’t actually know whether or not you’re sexually attracted to children; maybe you are, maybe you aren’t.
This is a special case of a general principle: the only way to know that you don’t secretly want X, desire X, or value X, is if you would be able to admit to yourself that you do want/desire/value X, in worlds where you really do. If the possibility feels too painful to face, then you can’t rule the possibility out.
In order to know your own wants, desires, or values, you need to be ready to face the possibility that they’re not the wants, desires, or values you thought you had, wished you had, or were told you were supposed to have.
I haven’t looked into this very deeply, but I suspect that some of this information is historical propaganda. In particular, when I shallowly looked into the famous slave ship diagram some years ago, it didn’t really add up, and I came away thinking it was likely propaganda from the abolitionists’ side. (… but I investigated only briefly and don’t remember the details, so take this with a lot of salt.) Some of the other things also have a propaganda vibe—e.g. the thing about slavery being bad for the economy, or about the sailors, or Clarkson giving an awfully heroic account of himself.
To be clear, at the time I wasn’t really thinking about other people escalating the playfulness. That didn’t even occur to me as a thing which might happen, until I was thinking about it after the fact.
I didn’t have a particular community in mind. Wish I knew one. Well, other than an elementary school or some other childrens’ community—that would definitely qualify as supercritically playful, but has a bunch of other downsides.
My impression is that kids at play generally do not have trouble coming up with ideas. So I think most people at least had that skill at one point, though it might have been forgotten and/or trained out of them.
Playfulness, like many social stances, sometimes undergoes a critical transition.
People hang out, someone does or says something a little playful, someone else responds playfully. Sometimes the response is a little *less* playful, and if that goes back-and-forth, then the playfulness dies out (left to its own devices). Other times the response is a little *more* playful, and if that goes back-and-forth, then silliness escalates until things get big and interesting.
When I think about recent times when I’ve brought some playfulness into my social environment, e.g. at parties, I notice that almost everyone responds in subcritical ways. So if I want to have a fun playful time, I can drive that by pumping in lots of playfulness myself (e.g. sitting on the roof at LessOnline tossing a big ball to passersby), but it feels like I’m pushing against a current. The silliness doesn’t take on a life of its own.
Me at LessOnline
There are exceptions to this. For instance, a few months ago Garrett and I had some supercritical back-and-forth over a fire pit at Lighthaven. We ended up cooking a lasagna noodle and convincing a slightly-drunk Jesse Hoogland to put the noodle in his pocket for five minutes to test whether it turned into a sticky mess. But that sort of thing is the exception to the rule in the Berkeley rationalist/EA community, as far as I can tell.
I think this is one of the major things I want in a community, that I don’t get from this one.
In John’s ideal community, if someone is on a roof tossing a ball to passersby, other people naturally show up and are like “ooh fun” and decide to join in for a while, and then a little crowd grows in the courtyard/on the roof/on the opposite balcony, all hitting this ball around. And then eventually someone might e.g. run off and find a net, hang that up, and like ten or twenty people end up playing two-story volleyball with the blue ball in the courtyard. Or maybe someone shows up with squirt guns and those get added to the game. Or people start trying to keep the ball aloft by throwing things at it. Or [...] That would be supercritical playfulness.
Alright. Sorry for being grumpy at you, and thank you for the model.
That is useful info, thanks.
That all matches my experience. My day to day interp needs are a long series of relatively isolated experiments with visual outputs; I’m not building a big complex unified edifice.