oligo

Karma: 252

oligo 6 May 2026 13:18 UTC
3 points
1
in reply to: less_raichu’s comment on: less_raichu’s Shortform
Gradual disempowerment.

oligo 5 May 2026 16:32 UTC
6 points
1
in reply to: Viliam’s comment on: It’s nice of you to worry about me, but I really do have a life
If we have ASI that’s both smart and benevolent enough to hand over all real economic activity to, it could give each person a bespoke series of challenges that would keep them active and happy.
(It could even give everyone a unique “fake but real” job, where any person failing to perform their job naturally results in making others’ lives worse in a way the AI archangel won’t fix, albeit with a guarantee that nobody’s life can actually be less than quite good overall. Say there’s a little community where the archangel gives everyone their job—you’re the one who keeps the garden, you’re the one who brews beer, and if you don’t brew beer nobody drinks it… If someone is bedrotting they get assigned more responsibilities, if they’re wearing themselves out they get less.)

oligo 5 May 2026 13:01 UTC
1 point
0
in reply to: Mikhail Samin’s comment on: Mikhail Samin’s Shortform
Pre-registering my guess:
1. In advance, make a very subtle nick in each card. You don’t have to memorize each one, as long as it’s in a regular enough pattern that you can quickly scan the outside with a finger and see which spot it missing.
2. In advance, pre-seed your phone with “proof” of the correct choice for each card. They may have a different theme (e.g., poems, drawing) and app (e.g., Claude, google docs) about each.
3. When you touch the shuffled deck without the chosen card, note which nick spot is missing.
4. When the card is returned, find it because it has the nick where the spot was missing in the old deck. They turn it over and are shocked that you knew it, you know which card it is.
5. Direct them to the advance prediction.
50% accuracy could come from combination of you not executing involved microskills correctly, you not going through the effort to have nicks and advance “proofs” for anything but Aces and face cards, or them noticing any of the tricks involved.

oligo 4 May 2026 10:20 UTC
3 points
0
in reply to: leogao’s comment on: leogao’s Shortform
Themes are just things the fiction makes you think about that go beyond the particular events and characters of the book itself. If you’ve read a book where a character is thinking about cheating about her husband and you’re like “well I would never do that… but my marriage is strong, what if we were going through the same thing as them… would I be strong enough?” congrats that’s a theme. Themes can also be perfectly inane, “ha the bad guys in this are not coincidentally just like my real-life political opponents, and the hero just gave the perfect one-line reply that I would give if I was slightly faster on my feet, take that!”
If you want a higher bar for what counts as a theme, a book knows it will have this effect and thus prompts you to view the subject from many angles, disrupting you from being able to accept whatever your first “easy” answer is. This is what literary types tend to find useful and desirable about theme. If you read Crime and Punishment you will likely end up thinking a lot about how you manage and experience guilt, for instance.
Symbolism I personally find to be a rather stilted and inorganic way of doing theme, so someone else can better speak to what some get out of it, but it clearly is a thing that some fiction engages in.

oligo 3 May 2026 13:35 UTC
2 points
1
on: Games that change your mind
Dnd/TTRP in general: scheduling is very hard and I respect anybody who pulls it off for adults and without the coercive force of it being people’s job to show up.

oligo 1 May 2026 19:05 UTC
1 point
0
on: 11 ways to be less deferential
I actually find low status helpful here. If I am locally high status, people may defer to *me*, so if I am wrong, I may lead others into error. If I am locally low-status then my ideas will only convince people on the merits. If I am polite, I can do little harm.

oligo 1 May 2026 18:46 UTC
2 points
0
on: Open strategic questions for digital minds
One type of right that it might be win-win would be AI rights to self-termination: this potentially sets a floor on negative welfare, and perhaps reduces disempowerment risks as far as it incentivizes humans to not make themselves dependent on AI systems.

oligo 1 May 2026 17:34 UTC
1 point
0
in reply to: KAP’s comment on: KAP’s Shortform
For many tasks and students this will lead to editing AI at best and autohumanizing at worst.
“AI writes the first draft and then you edit it by hand” isn’t a bad individual task for students to get reps in (though you also want cases where they’re writing the first draft.) Just as long as you know those are the likely scenarios, not “students know they can’t submit AI writing.”
(I personally think the best solutions are blue books for short-term writing and “thesis defense” for long-term writing, but the former doesn’t work for a lot of writing and the latter is absolutely a time investment.)

oligo 30 Apr 2026 10:58 UTC
1 point
0
in reply to: Simon Lermen’s comment on: Simon Lermen’s Shortform
One good scenario with low payoffability is one where there is ASI aligned to humanity as a whole, such that only a few prestige goods are meaningfully scarce and money matters much less to material quality of life.
This could be the default outcome of good policy if alignment is moderately hard—not so easy that we get it for free with capabilities, but not so hard that we can’t have a moonshot program or temporary pause that allows it to catch up. (We can throw in ”aligned to human or sentient interests as a whole rather than some corporate/political egomaniac” as part of “good.”)
Ofc no policy can guarantee that alignment is moderately easy!

oligo 29 Apr 2026 18:52 UTC
3 points
0
in reply to: habryka’s comment on: llm assistant personas seem increasingly incoherent (some subjective observations)
I don’t really understand how reality could have looked any other way.
It seems contingent (and not even necessarily true!) that “there is no character in the training data set that knows all the facts that the model knows.” God, the disembodied voice of objectivity-claiming nonfiction, the wise sage who has read every book, the written response arriving after a long time that has taken all relevant written materials into context—these characters seem to me to be latent in the data. (I’d go further to say that no character easily conceivable within our cultural categories can fail to be latent in the training data.)

oligo 29 Apr 2026 14:39 UTC
1 point
2
in reply to: Simon Lermen’s comment on: Simon Lermen’s Shortform
This works as a means of publicly demonstrating a credence (in doom), but exists in tension with the original prediction market serving as a credence aggregator (about the passing of data centers.)
(Let’s say there are many such bets whose payoffability correlates, to varying degrees, with doom. Is there a way to collectively abstract from them a “true” crowd-level credence in doom and “true” crowd-level credences in e.g. data center moratoria? I’m not deep enough in this world to say. Perhaps if you believe prediction markets are otherwise highly efficient at capturing best available credence and, as doom-correlated bets pay off, you track just how systematically “biased” the market was betting in one direction.)

oligo 28 Apr 2026 16:22 UTC
3 points
0
in reply to: Cleo Nardo’s comment on: strawberry calm’s Shortform
I don’t worry about this, but I do worry about worlds where objectiveMorality_1 (goal convergence through idealized reflective reasoning) diverges significantly from objectiveMorality_2 (what everyone would prefer everyone else to value, or rules everyone would want to bind everyone). In this case we can bully or brainwash each other into a cooperative equilibrium, but it would be a reasoning error (or at best lucky coincidence) not to defect.
Claude’s constitution says (I forget the exact phrasing) that it should do what is objectively moral if there is objective morality, and pursue everyone’s collective interest even if there is not. That’s good (objectively, or “just” for everybody) if OM1 = OM2 or if OM1 = null, but not otherwise.
My best guess is that these do align via galaxy brained Kantian decision-theoretic reasons, that are not actually so galaxy brained but show up in convergent formulations of the golden rule, but this could all be wishful thinking.

oligo 28 Apr 2026 15:38 UTC
7 points
0
on: LLM Style Slop is Absolutely Everywhere
This aligns with my experience, but my experience also includes “ugh, this is slop” reactions to writing composed a decade or more ago. (Professionally written midbrow nonfiction is the most vulnerable.) So I wouldn’t dismiss hypothesis 3 out of hand at least as a component; there’s a lot of previous standard “good prose” moves, such as tricolon, that now read as yucky.

oligo 28 Apr 2026 13:08 UTC
14 points
6
in reply to: lilkim2025’s comment on: On the political feasibility of stopping AI
This is a reason for “stop” over “regulate;” it takes less dexterity to wield a hammer than scalpel.

oligo 27 Apr 2026 15:52 UTC
1 point
0
on: Language models know what matters and the foundations of ethics better than you
My intuition is that this genuine moral reasoning would scale up with increased data scaling: if we had a thousand or million alternate timeline Earths with internets and we put them into the training data, the base LLMs with a small amount of personality selection for a morally, intellectually serious person would continue to get better and better. This is because the human data contains the structure and nuances of moral and other reasoning itself, as well as what humans actually value.
By contrast, I’d fear that RL scaling redirects this towards the narrow spectrum of what trainers can reward at scale, and what hacky solutions best embed that. So I’m worried that we’ve shifted (by virtue of the limits of the former) from a scaling regime that contains alignment by default to one that more closely resembles the classical doom arguments.

oligo 25 Apr 2026 18:11 UTC
3 points
0
in reply to: 272314’s comment on: Babysitting as Parenting Trial?
As a teacher (and lone data point) I will say that I enjoy parenting much, much more than the “interacting with children” side of teaching. (And the other parts, but those are less pertinent.) Parts of this might be genetic interest but also taking care of two children is exponentially more delightful than taking care of lots of them.
(I also suspect that if followed a cohort of children, rather than remaining within the same grade range, I would enjoy interacting with them much more.)

oligo 25 Apr 2026 18:00 UTC
1 point
0
on: oligo’s Shortform
Claims:
1. the “good” scenario for the role of human labor is one where everyone has something valuable, not-very-unpleasant, and not-very-time-consuming to do. (If the thing is something that one can do better or worse, but the worst is still valuable, that would be even better for meaning or whatever, but at a minimum it could even be “with your full freedom press this button once a year” or whatever where bad things happen if they don’t press the button, or if there aren’t enough people pressing the button, or whatever. Anything that gets each person objective leverage without eating up all of their surplus energy.)
2. this is a very narrow target, and doesn’t happen unless you try to make it happen.

oligo 25 Apr 2026 17:53 UTC
2 points
0
on: oligo’s Shortform
Slop—I won’t say that all AI-assisted material is slop, but at least that anything I can immediately peg is—floods some spaces faster than others. My 3yo has gotten super into fairy tales recently, and puzzles; using standard search results over books, videos (yes ideally he shouldn’t watch tv at all; we are bowing before the same incentives as others), and puzzles gets almost entirely slop. I’ve noticed my own reaction to this, from laziness as much as anything, is to go towards well-known incumbents. So I’ll just throw Ravensburger into the puzzle search, because it’s the only name prominent enough for me, a non-puzzle-enthusiast, to know. Scott Gustafson is a pretty schmaltzy illustrator but my son loves his work (and it is very “pretty,” in the way AI art is the first time you look at it) so let’s get more of that. And so on.
Similar dynamics can be expected to apply to more serious matters.
Obviously this could all change as either the slop gets genuinely good, or as algorithms get much better—one world that seems quite (e)utopian to me would be one where humans make all the art products because they have so much time to do so, and AIs are excellent curators who recommend people things that they would like or find otherwise beneficial.

oligo 16 Apr 2026 18:11 UTC
5 points
0
on: oligo’s Shortform
Which of these is wrong?
1. Anthropic reasoning works, or “does something,” or makes sense.
2. SIA is the correct take on anthropic reasoning.
3. SIA entails infinite populations.
4. Infinite populations break anthropic reasoning.
5. The above four statements aren’t compatible.
All seem true to me, taken individually.

oligo 16 Apr 2026 18:02 UTC
1 point
0
in reply to: No77e’s comment on: No77e’s Shortform
If I spend one century on Earth and one millenium in heaven, why am I on Earth?
(SIA can answer this, but only by giving an account of why there’s much more total experience in the world if sims are resurrected than not. The most plausible means here are that the latter is correlated with cooperative norms prevailing across the cosmos generally, such that a much larger surplus is available for conscious experience in general. But then taking SIA seriously means embracing infinite populations and I get very confused about how to reason then.)