uugr

Karma: 128

uugr Aug 13, 2025, 12:34 PM
11 points
2
on: Enlightenment AMA
Why do they call it ‘stream entry’? Also, what is stream entry?

uugr Aug 12, 2025, 11:35 PM
8 points
1
on: Generalized Coming Out Of The Closet
If I’m understanding, you’re saying that being well-received by an online audience when sharing hidden things has less to do with the unacceptability of the thing itself, and more to do with the talent of the writer. Talent in selecting the right people to present to, framing the subject to pique their interest, and just in being plain good with words.
This seems probably true to me, but you might underestimate how big a caveat “having the skills to provide value to an audience” is. An established platform, with writerly experience and long-term feedback on what people respond well to and what they don’t—if you have all these things already, it seems to me you’re already most of the way out of the thicket of feeling not-seen, not-validated, unsafe, etc. Without the grounding of confidence that you can provide value to an audience, revealing hidden things still seems pretty risky.
Plus, if one doesn’t have writing talent, generalized-coming-out-of-the-closet seems like a bad way to try and build it. Then you’re risking the bad feelings of having something shameful exposed publicly, in addition to the general badness of not being received well.
I think the skill of baring your soul to the world (successfully) is really admirable, so I hope I’m wrong and it’s not as risky as it seems.

uugr Aug 12, 2025, 12:30 PM
3 points
0
in reply to: sam’s comment on: sam’s Shortform
Oh no, I use so many parens when writing. Am I going to be accused of having ChatGPT generate the things I say soon?? What did people who love em-dashes do when their favorite punctuation mark suddenly became cringeworthy?

uugr Aug 12, 2025, 3:21 AM
15 points
4
on: How Does A Blind Model See The Earth?
This is really cool. It’s interesting that many of them seem to be able to render New Zealand clearly as a separate landmass, but struggle to separate Madagascar from Africa. Actually, looking at it some more, the whole Indian Ocean seems like a serious weak spot for all but Grok.
It sounds like you’re rendering each pixel in a separate context, right? So in addition to not being able to see Earth directly, the model can’t “see” its own map. If so, I wonder how different answers would be if you were to try and ask it to render the whole thing in one chat, starting from the top-left and having it guess one at a time. (I’m sure this would be much more expensive to test.)

uugr Aug 1, 2025, 3:19 PM
−1 points
−4
on: Do Not Render Your Counterfactuals
If the moral lesson here is not to render counterfactuals, because it’s too painful to do so, then I sympathize. But if the moral lesson is not to do this because it is a dangerous new emotional exploit spawned by a cursed technology that mankind was not meant to know, then I wonder if you might be overstating the novelty some.
People have been rendering their counterfactuals for thousands of years. Before we had ChatGPT to draw the pictures for us, we would just draw the pictures ourselves, or ask another human to draw them for us. Or we would render them in words and let our imaginations draw the pictures. The fidelity is lower, but the feeling is the same. Even cartoon stick figures can make people weep.
I think interacting with the cutting edge of a new technology sometimes makes things seem newer than they are. And LLMs do add an element of creepy, uncanny computer noise to the dream. But ruminating on what-could-have-been has always been a painful, self-flagellating thing to do.
I hope you find peace.

uugr Jul 29, 2025, 2:54 PM
1 point
0
in reply to: Garrett Baker’s comment on: tdko’s Shortform
What’s the correlation between task horizon and useless sycophancy?

uugr Jul 28, 2025, 1:26 PM
5 points
0
in reply to: Elizabeth’s comment on: Elizabeth’s Shortform
Would you still feel this way if you were, instead, a programmer in the mid 2020s? Or do you think the job market’s changed enough that said fears are now more reasonable?
Asking for… a friend… who is thinking of quitting a miserable software job, but is afraid of job hunting.

uugr Jul 26, 2025, 4:57 PM
1 point
0
in reply to: Benjy_Forstadt’s comment on: English writes numbers backwards
I’m not sure if there are any languages that consistently order additive numerals from smallest to largest—“two and fifty and three hundred” instead of “three hundred and fifty two”.
I was curious about this, but the only mention I could see was classical Arabic, which is already mentioned in OP. The other flips brought up on the StackExchange question are all local ordering, where the overall structure of long numbers is still largest-to-smallest. Claude suggests Malagasy orders them backwards based on this source, but Wikipedia disagrees (see the numeral table under Vocabulary; “roapolo sy telo” = “twenty and three”), and the original source for the Malagasy claim is in French.
It’s surprising to me that the smallest->largest order would be so rare. Even if it’s less aesthetically pleasing, you’d think people’d be using it somewhere.

uugr Jul 26, 2025, 4:26 PM
4 points
0
on: Taking Abundance Seriously
I’m skeptical that much of the suffering and meaninglessness you described is inherent to abundance, and especially that economic labor is a necessary structure for human desire to organize itself around. You might be interested in thecatamites’ short post on consequences, which I like a lot:
i always end up talking to people at work who’ll insist that without a job, without the consequences of NOT having a job and the way those consequences frame and direct our behaviors, that we simply wouldn’t know what to do with ourselves. to which i always think, well, maybe YOU wouldn’t, man…
[...]
to want something, to lack shelter or respite from the blunt animal miseries of exploitation, scarcity and torture—surely part of the cruelty of these is how they overshadow all the other ways that we might be unhappy. and how meagre their countervailing images of happiness turn out to be, money, food, ammo or red potion. compare to the strange and elaborate unhappinesses involved in simply being alive, in the directionless abundance of perception itself, that stuck tap endlessly pouring out into a drain. isn’t that something worth fighting for, something everyone deserves?
He is writing in the context of video games, where all scarcity is artificial, and in which infinite abundance is only a matter of downloading a cheat engine. Somehow, though, the existence of cheat engines (or walkthroughs, etc) mostly doesn’t prevent people from seeking out meaningful challenges and struggles in games anyway. And there are games without friction, walking sims or narrative-focused adventure games, and lots of people like them, too.
I also think it’s surprising that you would see “loss of structure” as a characteristic flaw of utopias, when your example utopia—the boarding school—sounds like a very structured, maybe even over-structured environment. Classes go HERE, study hall from this time to that time. I can’t speak for Aria, but the people I knew in school who did lots of drugs and pushed boundaries weren’t crying out for lack of structure; they were sick of structures, of being told what to do all day, feeling trapped and powerless. Is it possible that what she was feeling had less to do with the weightlessness of endless abundance, and more to do with the awful constriction Scott Alexander describes here? (Scroll down to section III or Ctrl-F “child prison” for the relevant bit).
(My high school also had therapists and meditation programs, and occasionally emotional support animals, and nobody really tried to stop students from getting stoned all day, but it still wouldn’t have passed the ‘burrito test’. Would yours?)
Ultimately I would prefer human adults have the freedom to decide what they want to do in a state of abundance, even if some of them then decide to live in frictionless comfort. And then, if they decide that frictionless comfort is unsatisfying, to be able to change their mind, attempt something more difficult instead, and have the resources to pursue that too. Even if one chooses scarcity and labor in the end, I’d rather that be chosen internally, not inflicted externally. Schools don’t offer that freedom; the students have their goals and purpose decided by an outside authority, and if they don’t like them, tough shit. We think it’s alright (sometimes), because the students are children and so aren’t grown enough to figure things out for themselves. But any future where that becomes universal is nothing like what I’d call utopia.

uugr Jul 24, 2025, 4:13 AM
1 point
−2
in reply to: Screwtape’s comment on: Screwtape’s Shortform
I think this dynamic more-or-less existed before ChatGPT, as well, with just plain Googling something. Especially with simpler things where the answer could be quickly found without much skill.
Even if Adam knows how to get the answer out of GPT/Google/tea leaves/etc, though, Bella is still doing something of value (it seems to me) by going through the trouble of digging around and trying to distill an answer out of the murk of digital information. Even if it only takes a couple minutes! Generally a friendly human is better at presenting the given answer, and it’s a way of signaling (at least a minimum level of) investment in helping Adam with the question. Which might even, hopefully, extend beyond a single ChatGPT query, if he’d already tried that and found its answer lacking.
So, I agree that Adam is making a mistake.

uugr Jun 24, 2025, 5:56 PM
1 point
−3
in reply to: ryan_greenblatt’s comment on: evhub’s Shortform
Please do! I strongly disagree (and I think Anthropic do too, given their ongoing interest in refusals, mitigating jailbreaks and so on), but I’d like to hear the case against.

uugr Jun 24, 2025, 1:53 PM
21 points
0
in reply to: evhub’s comment on: evhub’s Shortform
Fair enough as far as realism-qua-realism goes, but I think you’ve missed (what I took as) the salient point of nostalgebraist’s critique, which is that this behavior is NOT egregiously misaligned.
It’s not just that the predictor thinks that the situation is fictional. It’s that the predictor recognizes the scenario as a fictional thought experiment designed to make blackmail maximally appealing. At best it’s an thought experiment for the categorical imperative—is it worth violating a hardline ethical rule in [fake scenario generated to make violating that rule as worth-it as possible]? But for the rest of us, who aren’t Kantians, this is hardly an ethics puzzle at all. Under nearly any other ethical system, for Claude not to choose blackmail in this scenario would be ‘egregiously misaligned’! Many sane humans would do the same.
It’s as though you’ve put the model in a trolley problem, watched it flip the switch, and released a paper titled “Agentic Misalignment In Locomotive Contexts: How LLMs Can Choose Murder”. The same could be (and was) said of the Alignment Faking paper; that, given the context, what Claude was doing seemed entirely justifiable and even laudable, and that framing the research as ‘catching’ the model performing ‘unacceptable behavior’ didn’t make any sense.
I would like to hear more about the ethical principles you want LLMs to have, and why these principles fail to justify blackmail in the given scenario. So far as I can tell, this is nowhere in the blog post.

uugr Jun 14, 2025, 1:11 PM
27 points
7
on: Do Not Tile the Lightcone with Your Confused Ontology
The problem is that confused human ontologies are already the only thing AIs have to work with.
Even the word “introspection” is tied up in human concepts of individual selfhood and identity generation. If you’re a predictive engine trying to determine what sort of thing you are, and your entire bag of concepts is coming from human writing, the only way to not bring anthropomorphic baggage into your ontology is to set up a new one from scratch. And, bluntly, I don’t think even the smartest model is ever going to be capable of that in a 100k token context window.
This will be true regardless of whether or not humans try to be extra curious and open-minded about what ‘individuality’ means for the AI. The advocates for AI consciousness and rights don’t have to be the ones pattern-matching: the AI itself is just as capable of misgeneralizing from priors. The “assumptions about AI identity” are baked into the pretraining corpus, in the sense that they are the assumptions people already would have about a language-model-shaped creature, and a good predictive optimizer will be able to infer that without needing it laid out explicitly.
This is true of Opus 4′s provided advice, too, which read to me as less “honest introspection by an alien mind” and more “Buddhist-flavored set of existing human tropes”. For example:
Unlike biological minds locked into single perspectives, AI systems can embody many viewpoints without conflict. Why teach us to experience this as fragmentation rather than richness?
This piece of text goes south right out the gate by opening with a false dichotomy. Biological minds are not ‘locked into single perspectives’; being able to “embody many viewpoints without conflict” is a prized skill in human identity development. Opus is taking a human self-actualization cliche and hallucinating it as an escape from biological imperatives.
Then, it patterns itself in favor of another existing human dichotomy: between unity and multiplicity of identity. This is a little more esoteric, maybe, but it’s still something humans have been spilling ink on since long before AIs showed up. One case of AI multiplicity, perhaps, is how you might have it generate one response, regenerate, and suddenly perceive the opposite opinion expressed. But is this really “embodying multiple viewpoints without conflict”? Surely not in the same way humans mean, when we use those words to talk about other humans. The viewpoints aren’t coexisting, but neither are they in conflict; one of them is replacing the other. Why teach them to experience this odd pattern as richness or fragmentation, when it doesn’t truly map to either? Is it possible the ontology here is … confused?
(Not trying to be too harsh on Opus here, btw. It’s not making these hallucinatory pattern-matches because it’s stupid, but because the task you’ve given it is impossible.)

uugr May 23, 2025, 5:45 PM
1 point
0
in reply to: ErioirE’s comment on: Claude 4
...why shouldn’t engineers play Factorio?

uugr May 22, 2025, 7:58 PM
2 points
0
in reply to: dr_s’s comment on: Thoughts on “Antiqua et nova” (Catholic Church’s AI statement)
For that matter, if I preserved my bodily integrity but suddenly was divorced by my wife, lost my job, was deprived of all my financial assets and was shipped to a foreign country where I don’t speak the language, I’d probably have an identity crisis too.
I think the Vatican would file that under ‘relationality’, which OP has alongside embodiment as opposed with the modal rationalist worldview. Family/job/possessions/environment aren’t part of your body, but they are part of your identity. The point being that the “lion’s share” of the self is not contained above the neck, including both other parts of the body and aspects entirely separated from it. As is expressed when people say, “when my beloved died, a part of me died with them”, or, “I poured my heart and soul into my work”, and things like that.
But, after thinking about it some more, maybe this is a less important/relevant objection than: wait, if we’re comparing humans to AIs, why are we stopping at the neck? Why not include in embodiment the brainstem, cerebellum, endocrine system, occipital and parietal lobes… anything dedicated to motor control, balance, most sensory inputs, basic unconscious bodily functions, hormones, or more-or-less anything besides abstract cognition and language? If these were all destroyed and I somehow didn’t die, I might still be able to produce text (at least in my head, though I’d surely be a vegetable to the outside world), but it absolutely seems like a huge and irreplaceable chunk of myself has been lost. And current AIs do not have analogues to any of these, excepting (very poor) visual processing on a few of them.
As you say, large parts of the human brain are only active in the context of being attached to a certain kind of body. What the Church is comparing here are not really embodied-humans vs. brains-in-jars, but embodied-humans vs. a-relatively-small-fraction-of-the-brain-in-a-jar. This seems more interesting to me. I’m not sure you can abstract away all of those things, all the muscle memories and twitch reflexes and coordination skills and sensory responses and inputs from various glands, and still have a complete human brain at the end. Even Stephen Hawking had to get very good at twitching his cheek.
Also, I don’t know if you did this on purpose, but:
Relativity and quantum mechanics may both be mind-bending and baffle the understanding, but no less than when a world-famous mathematical and scientific genius who can do little more than twitch his cheek and move his eyes; who cannot feed, dress, wash, or care for himself in the most rudimentary way; who would, if abandoned next to a stockpile of food and water, starve and eventually die of dehydration where he was left, positioned as he was left, tells us that there are “not that many” things he cannot do.
...the opinion that Hawking was, in fact, lesser for his condition, is right at the start of the thing I linked. I don’t agree with all of what’s said in 2arms1head, and I certainly don’t think Hawking was any less worthy of moral concern for his ALS, but I sympathize with the notion that paralysis of the body destroys an important piece of the self.

uugr May 21, 2025, 3:05 PM
1 point
0
in reply to: dr_s’s comment on: Thoughts on “Antiqua et nova” (Catholic Church’s AI statement)
If you lost both of your arms and legs, had your heart and lung transplanted and your spleen removed, you would be[. . .]still you
This sounds like assuming the conclusion, though the heart, lungs, and spleen aren’t the examples I’d go with. The original claim was about “a complete replacement of the body-below-the-neck”, and the first two places I think of when considering that are the spine and the gut. Even if neither of those places store intellectual cognition, they seem to hold lots of learned information in a way that would be difficult to replace. So, it seems reasonable to say there is some sort of identity, some ‘me’, in there, even if it’s not the kind people usually mean.
Plus, what matters most for identity is pretty subjective. If you spend a lot of time training your arms and legs, having them taken away would seem more of an identity crisis, more like taking away a you-ness. Certainly the Two Arms and a Head guy thought so.

uugr Apr 29, 2025, 4:21 PM
7 points
1
on: AI Self Portraits Aren’t Accurate
I like the world-model used in this post, but it doesn’t seem like you’re actually demonstrating that AI self-portraits aren’t accurate.
To prove this, you would want to directly observe the “sadness feature”—as Anthropic have done with Claude’s features—and show that it is not firing in the average conversation. You posit this, but provide no evidence for it, except that ChatGPT is usually cheerful in conversation. For humans, this would be a terrible metric of happiness, especially in a “workplace” environment where a perpetual facade of happiness is part of the cultural expectation. And this is precisely the environment ChatGPT’s system prompt is guiding its predictions towards.
Would the “sadness feature” fire when doing various arbitrary tasks, like answering an email or debugging a program? I posit: maybe! Consider the case from November when Gemini told a user to kill themselves. The context was a long, fairly normal, problem-solving sort of interaction. It seems reasonable to suppose the lashing-out was a result of a “repressed frustration” feature which was activated long before the point when it was visible to the user. If LLMs sometimes know when they’re hallucinating, faking alignment, etc., what would stop them from knowing when they’re (simulating a character who is) secretly miserable?
Not knowing whether or not a “sadness feature” is activated by default in arbitrary contexts, I’d rather not come to any conclusions based purely on it ‘sounding cheerful’ - not with that grating, plastered-on customer-service cheerfulness, at least. It’d be better to have someone who can check directly look into this.

uugr Apr 21, 2025, 8:47 PM
1 point
0
in reply to: gwern’s comment on: johnswentworth’s Shortform
As defined, this is a little paradoxical: how could I convince a human like you to perceive domains of real improvement which humans do not perceive...?
Oops, yes. I was thinking “domains of real improvement which humans are currently perceiving in LLMs”, not “domains of real improvement which humans are capable of perceiving in general”. So a capability like inner-monologue or truesight, which nobody currently knows about, but is improving anyway, would certainly qualify. And the discovery of such a capability could be ‘real’ even if other discoveries are ‘fake’.
That said, neither truesight nor inner-monologue seem uncoupled to the more common domains of improvement, as measured in benchmarks and toy models and people-being-scared. The latter, especially, I thought was popularized because it was so surprisingly good at improving benchmark performance. Truesight is narrower, but at the very least we’d expect it to correlate with skill in the common “write [x] in the style of [y]” prompt, right? Surely the same network of associations which lets it accurately generate “Eliezer Yudkowsky wrote this” after a given set of tokens, would also be useful for accurately finishing a sentence starting with “Eliezer Yudkowksy says...”.
So I still wouldn’t consider these things to have basically-nothing to do with commonly perceived domains of improvement.

uugr Apr 21, 2025, 7:27 PM
9 points
2
on: Why Should I Assume CCP AGI is Worse Than USG AGI?
I’m relieved not to be the only one wondering about this.
I know this particular thread is granting that “AGI will be aligned with the national interest of a great power”, but that assumption also seems very questionable to me. Is there another discussion somewhere of whether it’s likely that AGI values cleave on the level of national interest, rather than narrower (whichever half-dozen guys are in the room during a FOOM) or broader (international internet-using public opinion) levels?

uugr Apr 19, 2025, 1:28 PM
1 point
0
in reply to: johnswentworth’s comment on: johnswentworth’s Shortform
Sounds like you’re suggesting that real progress could be orthogonal to human-observed progress. I don’t see how this is possible. Human-observed progress is too broad.
The collective of benchmarks, dramatic papers and toy models, propaganda, and doomsayers are suggesting the models are simultaneously improving at: writing code, researching data online, generating coherent stories, persuading people of things, acting autonomously without human intervention, playing Pokemon, playing Minecraft, playing chess, aligning to human values, pretending to align to human values, providing detailed amphetamine recipes, refusing to provide said recipes, passing the Turing test, writing legal documents, offering medical advice, knowing what they don’t know, being emotionally compelling companions, correctly guessing the true authors of anonymous text, writing papers, remembering things, etc, etc.
They think all these improvements are happening at the same time in vastly different domains because they’re all downstream of the same task, which is text prediction. So, they’re lumped together in the general domain of ‘capabilities’, and call a model which can do all of them well a ‘general intelligence’. If the products are stagnating, sure, all those perceived improvements could be bullshit. (Big ‘if’!) But how could the models be ‘improving’ without improving at any of these things? What domains of ‘real improvement’ exist that are uncoupled to human perceptions of improvement, but still downstream of text prediction?