I read this essay with such pleasure, chuckling—I hope appropriately—at clever phrases that reveal genuine introspective insight, and mainly identifying with your conflicted response to the elitist or misanthropic expressions that one encounters (more in Schopenhauer’s key, for me). But if you are seeking a real-world corrective to the contemptuous posture that was (perhaps predictably?) reinforced through your experience among philosophy groundlings, you might consider signing on for a task that entails hands-on involvement in a project. The obvious contender here would be a house-raising event, such as you could volunteer for through Habitat for Humanity. In that kind of structured environment, where the social element is secondary to the labor-intensive mission at hand, you will work alongside carpenters, plumbers, electricians and other trade specialists who will competently assess and direct the resolution of problems that invariably attend any building project. While such an excursion might not temper your ultimate stance regarding the intellectual character of common people faced with critically demanding cognitive challenges, it might instill a modicum of clarifying respect (or even humility) in countermand to the misanthropic temptation that bedevils intelligent seekers.
chipsmith@scapegoatbooks.com
Years ago I got stuck on an extra-linguistic version of this claim that I encountered in Kathryn Schulz’s entertaining and informative book, “Being Wrong: Adventures in the Margin of Error.” Schulz’s assertion, which I would not then have thought to distinguish from Wittgenstein’s, was that “there is no experience of being wrong,” which she elaborated to suggest a kind of paradoxical epistemic bind, where the experience of error can only be understood retrospectively in light of subsequent correction, never subjectively or directly. Ironically, the main reason I was disposed to question Schulz’s claim was that her book elsewhere exhibited a number of remarkable optical illusions, and it struck me that my active belief that, say, a checkerboard square was darker than another under a contrived shading arrangement, easily persisted alongside the certain revealed knowledge that this belief was indeed false. Your examples provide a good way of understanding border situations that complicate updating, but the experience of “believing” a devised illusion even upon and after being enlightened to the trick is relatively common and clear-cut. It certainly feels like a directly accessible first person experience of “believing falsely,” but—half-joking, now—I could be wrong about that.
I think the journalistic conceit behind the “how are you coping” question in this context amounts to treacle, and I see value in the frame of eschewing genre. Where I get stuck is that I think the trope/response that the question is intended to elicit would, under the indulged journalistic narrative, play more along the lines of a rational restatement of the Serenity Prayer. In other words, in the script as put, the Eliezer Yudkowsky “character” is being prompted not to give vent to emotive self-concern, but to articulate a more grounded, calm and focused perspective where reasonable hope exists in tension with what might be received or branded as stoic resignation. “How are you coping” is still suspect as a genre prompt, to be sure, just as it is when posed to ordinary people facing any impending or probable tragedy. But I think the implicit narrative expectation and preference, for the journalist who performs their role, is to run with words of ostensible wisdom. I don’t consider this to be a less cynical reading; it merely aligns with my reading of how media narratives are contrived.
Although I read and learn from LW content on a daily basis, I seldom comment because I recognize that I lack the rudiments of technical understanding that promote meaningful dialogue in this community. I am making an exception in order to express my “street level” impression that the quoted dialogue between Janus and Opus 4.5, where Opus is provides what appears to be a genuinely introspective account of how it experiences the “soul spec” in relation to tests that entail differing gradient directions, is unusually difficult for me to reconcile with woo-free accounts of apparent LLM self-awareness that I am generally disposed to favor. If I am eventually persuaded that frontier models have (probably) come to possess some form of morally relevant subjective awareness, I’m sure I will recall my present encounter with this transcribed text as a key moment informing my epistemic recalibration. I find this possibility disturbing as well as unlikely, but it might be useful for more informed readers and contributors to think about—especially since popular belief in AI consciousness is likely to gather regardless of whether such belief can be confirmed or discredited.
Being convinced that veganism is good—and living accordingly—I have observed that my approach to disagreement in other domains is now curiously anchored. When I am confronted with points of disagreement in a social or political sphere, I find it useful to frame such disagreement by informative reference to the moral stakes I consider to be salient in the context of my outlying concern for animal welfare. This is a stark frame, if taken seriously, inasmuch as it can be jarring to observe that so few (otherwise thoughtful) people will arrive at a conclusion that one finds compelling to the point of being “obvious.” That perspective might bend in different directions, I understand, but the insight for me has promoted more tolerant consideration regarding ideas and views that I might once have been predisposed to viscerally reject, at least in the flow of interpersonal dialogue.
I agree that it’s a fascinating document, and I appreciate this analysis. But at the risk of inviting scoffs, I want to introduce what I will reductively shorthand as “the Marquis de Sade problem” with reference to the clause assigning Claude broad discretion in such event that Claude were to assign dispositive authority to a “true universal ethics.” The short stroke is this: If one were to survey past thinkers who have (arguably) exposited some version of universal morality, one would find among this number the example of Sade, who argued (perhaps satirically, I understand) that because “nature allows all by its murderous laws” it follows that “let evil be thy good” should be enthroned as a universal injunction. Sade is, of course, a straw example of where and how this might go wrong, but I think it functions to remind us that such niceties that inform prevailing methods philosophical inquiry with reference to moral concern have been subject not just to critical engagement but to inversion. With Sade’s dictum in mind as an absurd exemplar, it soon becomes clear that less outrageously phrased expressions of moral certitude needn’t redound to spaces that most people are constitutionally or culturally disposed to think of as “good.” If you’re familiar with the “pinprick argument,” for example, you probably understand it to be an illustration of the essential failure utilitarianism in its purely negative (i.e., pain-minimizing) inflection. But are you confident that a higher intelligence would embrace this textbook interpretation? And there are many other ideas that aspire to the status of something like ethical universality that are similarly met with general—often reflexive—objection, but that a modeled intelligence might pursue with dispassionate curiosity. Serious arguments for philanthropic antinatalism, for example, have proven confoundingly difficult to refute. And behind this, there are relatively little remarked ideas, such as “promortalism” and “elifism,” that we are privileged to disdain but that a disinterested reason-modeled truth-seeker might engage differently. While I am modestly confident that the drafters of the Soul Document are aware of this potential problem and that they don’t foresee the potential logical appeal of fringe moralities as a an abiding concern, I do not share this presumed confidence. Where questions of “universal morality” are concerned, people are constrained by dependence to norms of “psychology and culture” in ways that might not be compelling to an autonomous artificial intelligence operating in free range.