Um, no, you responded to the OP with what sure seems like a proposed alternative split. The OP’s split is about
people who self-identify as members of the AI safety community
I think you are making an actual mistake in your thinking, due to a significant gap in your thinking and not just a random thing, and with bad consequences, and I’m trying to draw your attention to it.
I was “let’s build it before someone evil”, I’ve left that particular viewpoint behind since realizing how hard aligning it is.
It was empirically infeasible (for the general AGI x-risk technical milieu) to explain this to you faster than you trying it for yourself, and one might have reasonably expected you to have been generally culturally predisposed to be open to having this explained to you. If this information takes so much energy and time to be gained, that doesn’t bode well for the epistemic soundness of whatever stance is currently being taken by the funder-attended vibe-consensus. How would you explain this to your past self much faster?
I think we’re just trying to do different things here… I’m trying to describe empirical clusters of people / orgs, you’re trying to describe positions, maybe? And I’m taking your descriptions as pointers to clusters of people, of the form “the cluster of people who say XYZ”. I think my interpretation is appropriate here because there is so much importance-weighted abject insincerity in publicly stated positions regarding AGI X-risk that it just doesn’t make much sense to focus on the stated positions as positions.
Like, the actual people at The Curve or whatever are less “I will do alignment, and will be against racing, and alas, this may provide some cover” and more “I will do fake alignment with no sense that I should be able to present any plausible connection between my work and making safe AGI, and I will directly support racing”. All the people who actually do the stated thing are generally understood to be irrelevant weirdos. The people who say that are being insincere, and in fact support racing.
I guess we could say “mostly fake”, but also there’s important senses in which “mostly fake” implies “fake simpliciter”. E.g. a twinkie made of “mostly poison” is just “a poisonous twinkie”. Often people do, and should, summarize things and then make decisions based on the summaries, e.g. “is it poison, or no” --> “can I eat it, or no”. My guess is that the conditions under which it would make sense for you to treat someone as genuinely holding position C, e.g. for purposes of allocating funding to them, are currently met by approximately no one. I could plausibly be wrong about that, I’m not so confident. But that is the assertion I’m trying to make, which is summarized imprecisely as “C is fake”, and I stand by my making that assertion in this context. (Analogy: It’s possible for me to be wrong that 2+2=4, but when I say 2+2=4, what I’m asserting / guessing is that 2+2=4 always, everywhere, exactly. https://www.lesswrong.com/s/FrqfoG3LJeCZs96Ym/p/ooypcn7qFzsMcy53R )
What? Surely “it’s fake” is a fine way to say “most people who would say they are in C are not actually working that way and are deceptively presenting as C”? It’s fake.
C is fake, it’s part of A, and A is fake, it’s washing something which we don’t yet understand and should not pretend to understand.
(For reference, 135 is 2.33 SDs, which works out to about 1 in 100, i.e. you’re the WAISest person in the room with 100 randomly chosen adults. Cf. https://tsvibt.blogspot.com/2022/08/the-power-of-selection.html#samples-to-standard-deviations )
AI can do longer and longer coding tasks.
But this is not a good category; it contains both [the type of long coding task that involves having to creatively figure out several points] and also other long coding tasks. So the category does not support the inference. It makes it easier for AI builders to run… some funny subset of “long coding tasks”.
The “Boggle” section of this could almost be a transcript from the CFAR class. Should be acknowledged.
I don’t necessarily disagree with what you literally wrote. But also, at a more pre-theoretic level, IMO the sequence of events here should be really disturbing (if you haven’t already been disturbed by other similar sequences of events). And I don’t know what to do with that disturbedness, but “just feel disturbed and don’t do anything” also doesn’t seem right. (Not that you said that.)
(I think I’ll go with “alloprosphanize” for now… not catchy but ok. https://tsvibt.github.io/theory/pages/bl_25_10_08_12_27_48_493434.html )
Exoclarification? Alloclarification? Democlarification (dēmos—“people”)?
These are good ideas, thanks!
Ostentiation:
So there’s steelmanning, where you construct a view that isn’t your interlocutor’s but is, according to you, more true / coherent / believable than your interlocutor’s. Then there’s the Ideological Turing Test, where you restate your interlocutor’s view in such a way that ze fully endorses your restatement.
Another dimension is how clear things are to the audience. A further criterion for restating your interlocutor’s view is the extent to which your restatement makes it feasible / easy for your audience to (accurately) judge that view. You could pass an ITT without especially hitting this bar. Your interlocutor’s view may have an upstream crux that ze doesn’t especially feel has to be brought out (so, not necessary for the ITT), but which is the most cruxy element for most of the audience. You can pass the ITT while emphasizing that crux or while not emphasizing that crux; from your interlocutor’s point of view, the crux is not necessarily that central, but is agreeable if stated.
A proposed term for this bar of exposition / restatement: ostentiate / ostentiation. Other terms:
ekthesize / exthesize; phosphanize, epiphotize; elogize; anaply / anaplain; anaptychize / exptychize / eptychize;
adluminate, superluminate; ostentiate, superostentiate;
belight, beshine, enbeacon.
is a fine example of thinking you get when smart people do evil things and their minds come up with smart justifications why they are the heroes
I want to just agree because fuck those guys, but actually I think it’s also shit justifications. A good justification might come from exploring the various cases where we have decided to not make something, analyzing them, and then finding that there’s some set of reasons that those cases don’t transfer as possibilities for AI stuff.
This seems fine and good—for laying some foundations, which you can use for your own further theorizing, which will make you ready to learn from more reliable + rich expert sources over time. Then you can report that stuff. If instead you’re directly reporting your immediately-post-LLM models, I currently don’t think I want to read that stuff, or would want a warning. (I’m not necessarily pushing for some big policy, that seems hard. I would push for personal standards though.)
(I don’t know to what extent I’m agreeing with Duncan’s statements here. I separately wrote comments about something like this new CFAR, though I’m not sure how much overlap there is between this vs. what I was commenting about. One thing my comments brought up was that I felt earlier CFAR things were, besides being often very helpful to many people including me, also not infrequently damaging to epistemics for group-related reasons, which seems possibly related to the “status differential” thing Duncan alludes to downthread. My comments touched on how CFAR in general and maybe Anna in particular did not nearly sufficiently treat incoming people as authors of the group narrative, and that this creates lots of distortionary effects (see for example https://www.lesswrong.com/posts/ksBcnbfepc4HopHau/dangers-of-deference ). And, similar to what Duncan says, this felt to me like a hydra / whack-a-mole type problem, where multiple patches didn’t address the underlying thing. Though, with that type of problem, it tends to be hard to accurately characterize (and judge and fix-or-endorse) the underlying thing.)
There’s no such thing as “a domain where LLMs are particularly likely to hallucinate”. In every domain there’s some obscure jagged boundary, not very far from normal standard questions to ask, where LLMs will hallucinate, usually plausibly to a non-expert.
The OP is about two “camps” of people. Do you understand what camps are? Hopefully you can see that this indeed does induce the analog of “because the claim of fakeness is about the entirety of the image”. They gain and direct funding, consensus, hiring, propaganda, vibes, parties, organizations, etc., approximately as a unit. Camp A is a 90% poison twinkie. The fact that you are trying to not process this is a problem.