Decaeneus

Karma: 170

Decaeneus 14 Jul 2025 21:44 UTC
1 point
0
in reply to: Vladimir_Nesov’s comment on: Decaeneus’s Shortform
Indeed! I meant “we” as a reference to the collective group of which we are all members of, without requiring that every individual in the group (i.e. you or I) share in every aspect of the general behavior of the group.

To be sure, I would characterize this as a risk factor even if you (or I) will not personally fall prey to this ourselves, in the same way that it’s a risk factor if the IQ of the median human drops by 10 points, which this effectively might be equivalent to (net-of-distractions).

Decaeneus 14 Jul 2025 21:02 UTC
1 point
0
in reply to: Vladimir_Nesov’s comment on: Decaeneus’s Shortform
I suppose there are varying degrees of the strength of the statement.
- Strong form: sufficiently compelling entertainment is irresistible for almost anyone (and of course it may disguise itself as different things to seduce different people, etc.)
- Medium form: it’s not theoretically irresistible, and if you’re really willful about it you can resist it, but people will large will (perhaps by choice, ultimately) not resist it, much as they (we?) have not resisted dedicating an increasing fraction of their time to digital entertainment so far.
- Weak form: it’ll be totally easy to resist, and a significant fraction of people will.
I guess I implicitly subscribe to the medium form.

Decaeneus 14 Jul 2025 18:27 UTC
6 points
−2
on: Decaeneus’s Shortform
For me, a crux about the impact of AI on education broadly is how our appetite for entertainment behaves at the margins close to entertainment saturation.
Possibility 1: it will always be very tempting to direct our attention to the most entertaining alternative, even at very high levels of entertainment
Possibility 2: there is some absolute threshold of entertainment above which we become indifferent between unequally entertaining alternatives
If Possibility 1 holds, I have a hard time seeing how any kind of informational or educational content, which is constrained by having to provide information or education, will ever compete with slop, which is totally unconstrained and can purely optimize for grabbing your attention.
If Possibility 2 holds, and we get really good at making anything more entertaining (this seems like a very doable hill to climb as it directly plays into the kinds of RL behaviors we are economically rewarded for monitoring and encouraging already) then I’d be very optimistic that a few years from now we can simply make super entertaining education or news, and lots of us might consume that if it gets us our entertainment “fill’ plus life benefits to boot.
Which is it?

Decaeneus 14 Jul 2025 14:06 UTC
1 point
0
on: Decaeneus’s Shortform
Maybe there’s a deep connection between:
(a) human propensity to emotionally adjust to the goodness / badness our recent circumstances such that we arrive at emotional homeostasis and it’s mostly the relative level / the change in circumstances that we “feel”
(b) batch normalization, the common operation for training neural networks
Our trailing experiences form a kind of batch of “training data” on which we update, and perhaps we batchnorm their goodness since that’s the superior way to update on data without all the pathologies of not normalizing.

Decaeneus 7 Jul 2025 14:30 UTC
3 points
0
in reply to: Kabir Kumar’s comment on: Kabir Kumar’s Shortform
One can say that being intellectually honest, which often comes packaged with being transparent about the messiness and nuance of things, is anti-memetic.

Decaeneus 7 Jul 2025 14:29 UTC
42 points
5
on: Decaeneus’s Shortform
Having young kids is mind bending because it’s not uncommon to find yourself simultaneously experiencing contradictory feelings, such as:
- I’m really bored and would like to be doing pretty much anything else right now.
- There will likely come a point in my future when I would trade anything, anything to be able to go back in time and re-live an hour of this.

Decaeneus 7 Jul 2025 14:21 UTC
1 point
0
in reply to: RedMan’s comment on: Decaeneus’s Shortform
This is a plausible rational reason to be skeptical of one’s own rational calculations: that there is uncertainty, and that one should rationally have a conservativeness bias to account for it. What I think is happening though is that there’s an emotional blocker than is then being cleverly back-solved by finding plausible rational (rather than emotional and irrational) reasons for it, of which this is one. So it’s not that this is a totally bogus reason, it’s that this actually provides a plausible excuse for what is actually motivated by something different.

Decaeneus 7 Jul 2025 14:17 UTC
1 point
0
in reply to: leogao’s comment on: Decaeneus’s Shortform
Thank you. I think, even upon identifying the reasons for why the emotional mind believes the things it does, I hit a twofold sticking point:
- I consider the constraints themselves (rarely in isolation but more like the personality milieu that they are enmeshed with) to be part of my identity, and attempting to break them is scary in both a deep existential loss of self sense, and in a “this may well be load bearing in ways I can’t fully think through” sense
- Even orthogonal to the first bullet, it’s somehow hard to change them even though with my analytical mind I can see what’s going on. It’s almost like the emotional Bayesian updating has brought these beliefs / tendencies to a very sharp peak long ago, but now circumstances have changed but the peak is too sharp to belief it away with new experience.
If it sounds like I’m trying to find reasons to not make the change, perhaps that’s another symptom of the problem. There’s a saboteur in the machine!

Decaeneus 4 Jul 2025 17:08 UTC
5 points
0
on: Decaeneus’s Shortform
This is both a declaration of a wish, and a question, should anyone want to share their own experience with this idea and perhaps tactics for getting through it.
I often find myself with a disconnect between what I know intellectually to be the correct course of action, and what I feel intuitively is the correct course of action. Typically this might arise because I’m just not in the habit of / didn’t grow up doing X, but now when I sit down and think about it, it seems overwhelmingly likely to be the right thing to do. Yet, it’s often my “gut” and not my mind that provides me with the activation energy needed to take action.
I wish I had some toolkit for taking things I intellectually know to be right / true, and making them “feel” true in my deepest self, so that I can then more readily act on them. I just don’t know how to do that—how to move something from my head to my stomach, so to speak.
Any suggestions?

Decaeneus 25 Jun 2025 16:49 UTC
6 points
0
on: Decaeneus’s Shortform
Something that gets in the way of my making better decisions is that I have strong empathy that “caps out” the negative disutility that a decision might cause to someone, which makes it hard to compare across decisions with big implications.

In the example of the trolley problem, both branches feel maximally negative (imagine my utility from each of them is negative infinity) so I have trouble comparing them, and I am very likely to simply want to not be involved. This makes it hard for me to perform the basic utility calculation in my head, perhaps not in the literal trolley problem where the quantities are obvious, but certainly in any situation that’s more ambiguous.

Decaeneus 4 Jun 2025 17:45 UTC
11 points
0
on: Decaeneus’s Shortform
There’s a justifiable model for preferring “truthiness” / vibes to analytical arguments, in certain cases. This must be frustrating to those who make bold claims (doubly so for the very few whose bold claims are actually true!)

Suppose Sophie makes the case that pigs fly in a dense 1,000 page tome. Suppose each page contains 5 arguments that refer to some of / all of the preceding pages. Sophie makes the claim that I am welcome to read the entire book, or if I’d like I can sample, say, 10 pages (10 * 5 = 50 arguments) and reassure myself that they’re solid. Suppose that the book does in fact contain a lone wrong argument, a bit flip somewhere, that leads to the wrong result, but is mostly (99.9%) correct.

If I tell Sophie that I think her answer sounds wrong, she might say: “but here’s the entire argument, please go ahead, and show me where any of it is incorrect!”

Since I’m very unlikely to catch the error at a glance, and I’m unlikely to want to spend the time to read and grok the whole thing, I’m going to just say: sorry but the vibes are off, your conclusion just seems too far off my prior, I’m just going to assume you made a difficult-to-catch mistake somewhere, but I’m not going to bother finding it.

This is reasonable on my part since I’m forced to time-ration, but must be very frustrating for Sophie, in particular if she genuinely believes she’s right (as opposed to just being purposefully deceptive.)

There’s also the possibility of a tragedy-of-the-commons here, whereby spendign my time on this is selfishly not in my best interest, but has positive externalities.

Decaeneus 22 May 2025 16:19 UTC
1 point
0
on: Decaeneus’s Shortform
I wonder if the attractor state of powerful beings is a bipole consisting of:
a. wireheading / reward hacking, facing one’s inner world
b. defense, facing one’s outer world
As we’ve gotten more and more control over our environment, much of what we humans seem to want to do resembles reward hacking: video games, sex-not-for-procreation, solving captivating math problems, etc. In an ideal world, we might just look to do that all day long, in particular if we could figure out how to zap our brains into making every time feel like the first time.
However, if you spend all day wireheading and your neighbor doesn’t, your neighbor will outpace you in resource generation and may be able to, one way or another, melt you down for scrap (and repurpose your resources for their own wireheading, possibly).
Much human culture (e.g. social customs, religion) can be understood as an attempt to temper some of the wireheading in favor of more defense, i.e. it’s discouraged as immoral to over-indulge yourself on video games, you should be out working hard instead.
Perhaps this, or something akin to it, could be expected to hold for the behavior of advanced AI systems. The end state of superintelligences may be perfect wireheading hidden behind the impenetrable event horizon of a black hole so that nobody can disturb its reverie.^[1]
1. ^
  Of course, it would be bad news if the epitome of defense is wiping out anything else that may surprise it, a la the List of Lethalities.

Decaeneus 21 May 2025 14:40 UTC
7 points
0
on: Decaeneus’s Shortform
Epistemics vs Video Generation

Veo 3 released yesterday serves as another example of what’s surely coming in terms of being able to generate video that’s indistinguishable from reality. We will be coming off a many-decades period where we could just believe video as a source of truth: what a luxury that will have been, in hindsight!
Assuming it’s important to have sources of truth, I see the following options going forward:
1. we will have to just accept that our world has permanently become epistemically worse. sad!
2. we will have to trust real life more than we do external evidence. this seems very limiting: my real life friends don’t have much insight into what’s happening in Ukraine, for example
3. we will need to pair video with a sort of “chain of custody” whereby the video itself will not be worth anything (epistemically) unless it comes with someone, whose identity is established in some other way, vouching for it being true. this is, in a way, how we presently treat written evidence.
4. lacking a trusted source, but making up for it by combining multiple videos of the same thing. imagine multiple angles of the same event (as confirmed by hard-to-spoof video metadata) corroborating each other (perhaps this can even be AI-enabled, in a privacy-preserving way). hoping that this is easy to verify but hard to fake (hard to generate mutually consistent multiple angles of the same thing) though this might be wishful thinking since this seems like something that can be trained eventually, but maybe buys us a few years
5. the physics solution: from first principles, video had the property that it was much easier to verify than it was to fake (a la certain encryption). video is, reductively, just a time series of photons. perhaps we can build The Truth Camera, which captures lots of other particles (or maybe captures photons at a frame rate and resolution that is costly, and thus there isn’t much data of, and thus is hard to credibly fake). essentially perhaps we can snapshot more aspects of reality in such a way that we re-create the condition that made video credible: easy to verify, hard to generate.
nb: There will likely be a long period where video is fake but still believed, much like people still believe things that are written down over things that are spoken, despite the fact that anyone can freely write down anything at any point.

Decaeneus 16 May 2025 15:35 UTC
6 points
0
on: Decaeneus’s Shortform
Inspired by this Tweet by Rohit Krishnan https://x.com/krishnanrohit/status/1923097822385086555
One thing LLMs can teach us is that memorisation of facts is, in fact, a necessary part of reasoning and intelligent behaviour
There’s a very simple model under which memorization is important:
- if good reasoning resembles test-time compute, judgement involves iterating through lots of potential hypotheses and validating them
- the speed with which you validate each hypothesis is partly a function of the time it takes to fetch the data necessary to assess it
- when that data is memorized (i.e. stored “in your brain’s weights”) access is really fast
- when that data is stored externally (i.e. have to run a google search to find it) validation is OOMs slower
- so you’ll be OOMs slower if you haven’t internalized the context of the problem
- further, there are upper bounds on the slower process (because you get bored, or you fatigue, etc.) so this difference in speed may also end up being a difference in capability. it’s not just that it’s OOMs slower, it may not even get there
This doesn’t please me as I generally dislike memorization, but the logic seems plausible.

Decaeneus 6 May 2025 15:11 UTC
5 points
1
on: The Ukraine War and the Kill Market
Combining kill markets with mis-optimized (incompletely specified, etc) RL agents is the stuff of nightmare.

Decaeneus 6 May 2025 14:19 UTC
1 point
0
on: Playing in the Creek
Some of this feels like partly what is going on with modernity & apathy, at least among those of us priviledged enough to have abundant material comfort (which is not everyone).
The “collect playthings” game has been beyond won, is no longer fun, and we haven’t collectively managed to choose our next game yet, for which we’d find ourselves in the more fun part of the curve.

Decaeneus 2 May 2025 16:46 UTC
2 points
0
on: Decaeneus’s Shortform
Absence of evidence is the dark matter of inference. It’s invisible yet it’s paramount to good judgement.
It’s easy to judge X to be true if you see some evidence that could only come about if X were true. It’s a lot more subtle to judge X to be false if you do see some evidence that it’s true, but you can also determine that there are lots of evidence that you would expect to have if it were true, but that is missing.
In a formalized setting like a RCT this is not an issue, but when reasoning in the wild, this is the norm. I’m guessing this leads to a bias of too many false positives on any issue where you care to look deeply enough to find and cherry pick the positive evidence.

EDIT: Correcting the opening sentence to say “absence of evidence” rather than the original “negative evidence”.

Decaeneus 1 May 2025 13:41 UTC
1 point
0
on: Decaeneus’s Shortform
Feels like there’s a missing deep learning paradigm that is the equivalent of the human “go for a walk and think about stuff, absent new stimuli”. There are some existing approaches that hint at this (generative replay / dreaming) but those feel a bit different than my subjective sense that I’m “working things out” when I go for a walk, rather than generatively dreaming, as I do at night.

Relatedly: it reduces my overall cognitive output when I go through periods of depriving myself of these idle periods by jamming them full of stimuli (e.g. podcasts). I do better when I bite the bullet, accept the boredom, and allow my mind to work out whatever it needs to.

Speculatively, perhaps the two are related: overstimulated ML researchers have a blind spot to the fact that understimulating a neural net might be helpful.

Decaeneus 16 Apr 2025 17:07 UTC
5 points
0
on: Decaeneus’s Shortform
There’s a particular type of cognitive failure that I reliably experience, which seems like a pure kind of misconfiguration of the mind, and which I’ve found very difficult to will myself to not experience, which feels like some kind of fundamental limitation.
The quickest way to illustrate this is with an example: I’m playing a puzzle game that requires ordering 8 letters into a word, and I’m totally stuck. As soon as I look at a hint of what the first letter is, I can instantly find the word.
This seems wrong. In theory, I expect I can just iterate through each of the 8 letters, tell myself that’s the first one, and then look for the word, and move on if the word isn’t there. If it took me 2 seconds to “see the word” for a given starting letter (so, not quite insta-, budgeting some context switching time) then I ought to be able to reliably find the word within 16 seconds. Yet, I find that I can’t do it even with a much greater time budget. It’s almost like the “not being sure” if each particular letter I’m iterating through is actually the starting one reduces my mental horsepower and prevents me from earnestly trying to find the word, almost like I expect to feel some mental pain from whole-heartedly looking for something that may not be there.
So I’m truly stuck, and then when I get the hint, I’m insta-unstuck. I’ve had the same feeling when solving other puzzles or even math problems: I (with high confidence) know the superset from which the beginning of the solution is drawn, yet iterating through the superset does not make the solution pop to mind. However, once I get a hint that tells me *for sure* what the beginning of the solution is, I can solve the rest of it.
What gives? How to fix? Does this have implications outside of puzzles, in the more open-ended world of e.g. looking to make new friends, or to find a great business model?
(Putting this into Grok tells me it doesn’t have a name but it’s a commonly known thing in the puzzle-solving community, of which I’m not part.)

Decaeneus 3 Dec 2024 17:50 UTC
4 points
−10
on: Decaeneus’s Shortform
Proposal: if you’re a social media or other content based platform, add a long-press to the “share” button which allows you to choose between “hate share” and “love share”.

Therefore:
* quick tap: keep the current functionality, you get to send the link wherever / copy to clipboard
* long press and swipe to either hate or love share: you still get to send the link (optionally, the URL has some argument indicating it’s a hate / love share, if the link is a redirect through the social media platform)

This would allow users to separate out between things that are worth sharing but that they hate / love and want to see less / more of, and it might defang the currently powerful strategy (with massive negative social externalities) of generating outrage content just to get more shares.

Social media companies can, in turn, then use this to dial back the viraility of hate share vs love share content, if they choose to do so.