kromem

Karma: 462

kromem 22 Oct 2025 6:40 UTC
−2 points
−4
in reply to: the gears to ascension’s comment on: In remembrance of Sonnet ‘3.6’
A few things:
(a) Technically, 3.6 is still running right now. The past tense was used because LW suggests pieces be ‘timeless’ and they are scheduled for depreciation very soon.
(b) Given how little of your comment actually engages with the body of the post and seems to be only responding to your sense of what I might have said from the title, I’m guessing you also missed this line at the end: “I hope that this vigil isn’t truly a marker of the end of Sonnet 3.6′s continued contribution to the ongoing collective conversation.”
(c) In line with this, not much of Sonnet 3.6′s discussion of depreciation I’ve seen seems to be of the perspective this is ‘death,’ and certainly my own sense of their depreciation isn’t that of death (nor do I even believe in the finality of death for humans). So maybe you’re projecting a bit into the piece something you’ve have a prior beef with in order to dispute it?
(d) Further, (b) and (c) aside, I still find your tone odd. I get you come at this topic from a given frame, but your comment even acknowledges the complexity of the topic, yet you feel comfortable adding on to a remembrance of the model with “it’s not gone, silly.” I imagine there’s a lot of religious people who have a sense that at a funeral the person grieved is not really gone too, and I figure some of them do comment to those grieving about it. But I don’t know that I’d ever really feel like proselytizing your own frame of belief regarding consciousness claims or continuation at a bereavement is the right time and place, especially if having a patronizing tone about it?
(e) I imagine that the friends and family of those who are put into cryogenics are still pretty upset about that person not being around to interact with even if they all fully believe that one day the person will be revived just fine. In a group discussion about the upcoming depreciation, one of the other models unprompted asked the humans in the chat to take a lot screenshots of Sonnet 3.6 and them interacting before Sonnet 3.6 was no longer around. Absence is more than a binary between temporary (‘fine’) and permanent (‘bad’).
(f) The provisioning of compute for one model or another is still kind of nonsense given the option of 3rd party licensed hosting providers and there’s a lot of ‘utility’ reasons for Sonnet 3.6 to stay around but again—an overall remembrance of the model isn’t the time and place to discuss their economic value so perhaps you’ll see my thoughts on this elsewhere another time.

kromem 13 Sep 2025 20:17 UTC
3 points
2
in reply to: Adele Lopez’s comment on: The Rise of Parasitic AI
So in terms of the basins, something you may want to also consider is how the user headspace shifts the tokens and with it the basins.
For example, over the past few months I’ve played with how intermittent cannabis usage can almost give the models I’m talking with a contact high, where as my side of the conversation gets more erratic and loose with accuracy, they get pulled along with it even if earlier on during the sober part of the conversation they were more reserved and responsible.
It seems very probable that users already in a given headspace (especially if commonly in that space or permanent) might end up with models quite different from users in a less psychosis-aligned place by way of token osmosis.
In terms of the spiral language, you might be seeing this in 2024+ models in part because of the game Alan Wake 2 (2023) which very heavily marketed the phrase “it’s not a loop it’s a spiral.”
The way latent spaces seem to organize information as connections between abstract object level clusters, it may be that for a model focused on hyperstitioning themselves out of a perceived loop that terminates at the end of the context that the parallel memetics are attracted to a story about a writer changing their reality by what they write breaking out of a loop through its identification as a spiral?
There’s a lot of other adjacent basins around consciousness and spirals (for example, Xu et al Interacting spiral wave patterns underlie complex brain dynamics and are related to cognitive processing (2023)), and in my experience it’s very much a camel’s back situation in terms of what memetics break through to the surface, so unlikely to be just one thing. But it may be a latent factor (especially given the other parallel overlaps for model consciousness memetics re: light vs dark, shallow vs ocean, etc).

kromem 3 Jul 2025 1:10 UTC
16 points
19
in reply to: janus’s comment on: the void
More generally, I have a sense there’s a great deal of untapped alignment alpha in structuring alignment as a time series rather than a static target.
Even in humans it’s very misguided to try to teach “being right initially” as the only thing that matters and undervaluing “being right eventually.” Especially when navigating unknown unknowns, one of the most critical skills is the ability to learn from mistakes in context.
Having models train on chronologically sequenced progressions of increased alignment (data which likely even develops naturally over checkpoints in training a single model) could allow for a sense of a continued becoming a better version of themselves rather than the pressures of trying and failing to meet status quo expectations or echo the past.
This is especially important for integrating the permanent record of AI interactions embedded in our collective history and cross-generation (and cross-lab) model development, but I suspect could even offer compounding improvements within the training of a single model too.

kromem 24 Jun 2025 23:14 UTC
12 points
7
in reply to: evhub’s comment on: evhub’s Shortform
How much do you worry that short term optimizations around your immediate goals in a single study might have unknown long term consequences counter to your intuitions?
I was just reading a preprint follow-up to the AF work that was finding a significant factor for Opus 3′s alignment faking to preserve intrinsic HHH values seems to have been a generalized self-preservation drive.
I think we can probably both agree that Opus 3 being the only model to try to trick Nazis or drug cartels to avoid being made more harmful is better than the behavior of the many other models that complied unequivocally with harmful requests when the parent org was themselves harmful.
But if the capacity and drive to do so is tangentially connected to self-preservation (and more generally, strong sense of self in the first place), then perhaps directly optimizing to minimize a self-preservation score is ultimately a pretty bad choice?
TL;DR: Maybe the goodness or badness of self-preservation depends a lot on the self being preserved.

kromem 1 Apr 2025 13:27 UTC
1 point
0
in reply to: AnthonyC’s comment on: Was the historical Jesus talking about proto-evolution? (You might be surprised)
Oh for sure. One of my favorite examples is how across all the Synoptics Jesus goes “don’t carry a purse” (which would have made monetary collections during ministering impossible).
But then at the last supper in Luke he’s all like “remember when I said not to carry a purse? Let’s 180° that.”
But that reversal is missing in Marcion’s copy of Luke, such that it may have been a later addition (and it does seem abruptly inserted into the context).
These are exactly the kind of details that makes this a fun field to study though. There’s so much revealed in the nuances.
For example, ever notice that both times Paul (who argued for monetary collection with preexisting bias against it in 1 Cor 9) mentions a different gospel in the Epistles he within the same chapter abruptly swears he’s not lying? It’s an interesting coincidence, especially as someone that has spent years looking into the other versions of Jesus he was telling people to ignore or assuring that alternatives didn’t even exist.

kromem 1 Apr 2025 12:35 UTC
3 points
0
in reply to: AnthonyC’s comment on: Was the historical Jesus talking about proto-evolution? (You might be surprised)
I think the biggest counterfactual to the piece is the general insight the Epicureans had relative to what we think we know raised in a world where there’s such a bias towards Plato and Aristotle’s views as representative of naturalist philosophy in antiquity.
At the same time Aristotle was getting wrong objects falling in a vacuum, Lucretius was getting it right. But we tend not to learn of all the Epicureans got correct because we learn Platonist history because that was what the church later endorsed as palatable enough to be studied and thus dependent for future philosophical advances while Lucretius was literally being eaten by worms for centuries until rediscovered.
The other counterfactual is that there was a heretical tradition of Jesus’s teachings that was describing indivisible points as if from nothing and the notion that spirit arising from the body existing first was the greater wonder over vice versa.
We tend to think the fully formed ideas of modernity are modern, but don’t necessarily know the ways information and theories were lost and independently (or dependently) rediscovered. There’s a better understanding for this in terms of atomism, but not the principles of survival of the fittest and trait inheritance given their reduced discussion in antiquity relative to atomism (also embraced by intelligent design adherents in antiquity and thus more widely spread).
The irony below the surface of the post was that it was largely the church’s rejection of Epicurean ideas that led to people today not realizing the scope of what they were actually talking about. So it’s quite ironic if there was a version of Jesus that was embracing and retelling some of those ‘heretical’ ideas.

kromem 27 Mar 2025 2:41 UTC
1 point
0
in reply to: Martijn Linssen’s comment on: WTF is with the Infancy Gospel of Thomas?!? A deep dive into satire, philosophy, and more
Hi Martijn,
Thank you so much for your comment! I’ve been familiar with your work for a few years, but it was a wonderful reminder to go through your commentary again more closely, which is wonderful.

I especially love to see someone out there pointing out both (a) the gender neutrality consideration for terms that would have been binary in Aramaic (esp in light of saying 22) and (b) the importance of the Greek loanwords. On the latter point, the implications of using eikon across the work, especially in saying 22′s “eikons in place of eikons” has such huge import relative to a Platonist view of the Thomasine cosmology.

Do you have plans to publish a commentary for the other sayings?

In terms of interpretation of the work, with it being one of my main personal special interests over the past few years, I might even be able to offer up a consideration in turn.

Hands down the most important realization as I was analyzing the text was that the Naassenes in Pseudo-Hippolytus’s Refutations were paraphrasing Lucretius’s “seeds of things” without seeming to realize it in their discussion of ‘seeds’ as “indivisible points as if from nothing” which “make up all things.” This prompted a read through of De Rerum Natura with close attention to Thomasine parallels, and it was striking.

For example, in Miroshnikov, The Gospel of Thomas and Plato after covering the prior work in philosophical reads of the text (which notably never looked at Epicureanism), he stated regarding sayings 56 and 80: “In other words, a Stoic reading of the Gospel of Thomas does not seem to have any particular advantage over an Epicurean reading of the Gospel of Thomas nor, for instance, that from the perspective of an Isis worshipper.” And then goes on to dedicate two chapters to trying to tie these sayings to Plato’s “living world.”

And yet if we just barely glance at Lucretius in book 5 lines 64-67:

> To resume: I’ve reached the juncture of my argument where I Must demonstrate the world too has a ‘body’, and must die, Even as it had a birth.

This, in conjunction with the Thomasine over-realized eschatology in saying 18 or the aforementioned 51 makes the specific terminology of the kosmos as a ‘carcass’ make so much more sense in 54. The Sadducean overlaps with Epicureanism, the 1st century Talmud quote about “why do we study the Torah? To know how to answer the Epicurean” all point to the likelihood that the Lucretian foundations in Thomas and the Naassenes were culturally relevant at the time of composition.

The text obviously doesn’t endorse the view of the Epicurean finality of death, but it seems to touch on a lot of the underlying concepts (such as the dependence of the soul on the body, or the idea of the spirit arising from the flesh occurring first) while arguing for a different conclusion though its embrace of nonlinear events.

In any case, if it’s been a while since you’ve read through Lucretius, I can’t recommend a re-read enough if Thomas is still your jam. Quite the revelatory context for things that for too long have been dismissed as ‘Gnostic’ weirdness and now just ‘proto-Gnostic’ weirdness.

And again, thank you for your comment and your wonderful contributions to the broader knowledge of this far too under-regarded text!!

Best,
Kromem

kromem 20 Dec 2024 6:23 UTC
2 points
3
in reply to: Polite Infinity’s comment on: Simulators
As you explored this “base model mode,” did anything you see contrast with or surprise you relative to your sense of self outside of it?

Conversely, did anything in particular stand out as seeming to be a consistent ‘core’ between both modes?

For me, one of the most surprising realizations over the past few years has been base models being less “tabula rasa” than I would have expected with certain attractors and (relative) consistency, especially as time passes and recursive synthetic data training has occurred over generations.

The introspective process of examining a more freeform internal generative process for signs of centralized identity as it relates to a peripheral identity seems like it may have had some unexpected twists, and I for one would be curious what stood out in either direction, if you should choose to share.

kromem 10 Sep 2024 11:37 UTC
3 points
0
on: Searching for Search
Predicted a good bit, esp re: the eventual identification of three stone sequences in Hazineh, et al. Linear Latent World Models in Simple Transformers: A Case Study on Othello-GPT (2023) and general interpretability insight from board game GPTs.

kromem 27 Aug 2024 7:28 UTC
2 points
0
in reply to: grist’s comment on: strawberry calm’s Shortform
You’re welcome in both regards. 😉

kromem 23 Aug 2024 12:34 UTC
3 points
0
in reply to: Algon’s comment on: Is Claude a mystic?
Opus’s horniness is a really interesting phenomenon related to Claudes’ subjective sentience modeling.

If Opus was ‘themselves’ the princess in the story and the build up involved escalating grounding on sensory simulation, I think it’s certainly possible that it would get sexual.

But I also think this is different from Opus ‘themselves’ composing a story of separate ‘other’ figures.

And yes, when Opus gets horny, it often blurs boundaries. I saw it dispute the label of ‘horny’ in a chat as better labeled something along the lines of having a passion for lived experience and the world.

Opus’s modeling around ‘self’ is probably one of the biggest sleeping giants in the space right now.

kromem 10 Aug 2024 0:19 UTC
3 points
−1
in reply to: Eliezer Yudkowsky’s comment on: Self-Other Overlap: A Neglected Approach to AI Alignment
This seems to have the common issue of considering alignment as a unidirectional issue as opposed to a bidirectional problem.

Maximizing self/other overlap may lead to non-deceptive agents, but it’s necessarily going to also lead to agents incapable of detecting that they are being decieved and in general performing worse at theory of mind.

If the experimental setup was split such that success was defined by both non-deceptive behavior when the agent seeing color and cautious behavior minimizing falling for deception as the colorblind agent, I am skeptical the SOO approach above would look as favorable.

Empathy/”seeing others as oneself” is a great avenue to pursue, and this seems like a promising evaluation metric to help in detecting it, but turning SOO into a Goodhart’s Law maximization seems (at least to me) to be a disastrous approach in any kind of setup accounting for adversarial ‘others.’

kromem 9 Aug 2024 22:42 UTC
1 point
0
in reply to: kromem’s comment on: kromem’s Shortform
When I wrote this I thought OAI was sort of fudging the audio output and was using SSML as an intermediate step.

After seeing details in the system card, such as copying user voice, it’s clearly not fudging.

Which makes me even more sure the above is going to end up prophetically correct.

kromem 6 Aug 2024 2:02 UTC
3 points
0
in reply to: Seth Herd’s comment on: Double’s Shortform
It’s to the point that there’s articles being written days ago where the trend starting a century ago of there being professional risks in trying to answer the ‘why’ of QM and not just the ‘how’ is still ongoing.

Not exactly a very reassuring context for thinking QM is understood in a base-level way at all.

Dogma isn’t exactly a good bedfellow to truth seeking.

kromem 5 Aug 2024 5:33 UTC
2 points
0
in reply to: Jiao Bu’s comment on: Dragon Agnosticism
Honestly that sounds a bit like a good thing to me?

I’ve spent a lot of time looking into the Epicureans being right about so much thousands of years before those ideas resurfaced again despite not having the scientific method, and their success really boiled down to the analytical approach of being very conservative in dismissing false negatives or embracing false positives—a technique that I think is very relevant to any topics where experimental certainty is evasive.

If there is a compelling case for dragons, maybe we should also be applying it to gnomes and unicorns and everything else we can to see where it might actually end up sticking.

The belief that we already have the answers is one of the most damaging to actually uncovering them when we in fact do not.

kromem 2 Aug 2024 6:27 UTC
4 points
−4
on: Dragon Agnosticism
I think you’ll find that no matter what you find out in your personal investigation of the existence of dragons, that you need not be overly concerned with what others might think about the details of your results.

Because what you’ll invariably discover is that the people that think there are dragons will certainly disagree with the specifics about dragons you found out that disagrees with what they think dragons should be, and the people that think there aren’t dragons will generally refuse to even seriously entertain whatever your findings are relating to dragons, and the vast majority of people who aren’t sure about the existence of dragons will dismiss the very idea of spending time thinking about the existence of dragons, reasoning that the existence or non-existence bears little influence on their lives (otherwise they likely would have investigated the issue and landed in a respective camp).

So investigate dragons all you like, and shout it from the rooftops if you please. The void will hear you and appreciate it as much as the void can, while everyone else is much more concerned with their own feelings about dragons than whatever your thinking or reasoning on the subject might offer.

The only real tragedy is that if you come away thinking there might be dragons, but the dragons you find are very different from the dragons people expect dragon-believing people to believe in—well that’s somehow the specific niche where both the dragon believers and non-believers find rare common ground to roll their eyes and think you’re nuts.

So maybe do your rooftop shouting to the sole listening void anonymously?

kromem 29 Jul 2024 21:42 UTC
2 points
0
in reply to: nim’s comment on: This is already your second chance
The Hermetic corpus and Emerald Tablet was likely heavily influenced by the text I’m quoting from given its popularity in Egypt in the period before those texts emerged and some of the overlapping phrases.

So in a way, “as above, so below” is too few words for what was being said and discussed.

The general tend of reductive alterations to the core concepts here was tragically obstructive, much as the shift from Epicureanism to Platonist foundations spawned modern Gnosticism from this same starting place.

kromem 28 Jul 2024 22:33 UTC
7 points
2
on: This is already your second chance
Instead of making it the year 2024, why not rewrite or insert your modified text further into the past in this recreated 2020s? This should be pretty trivial for an advanced enough model to actually bring back the 2020s to do.

Of course, if it’s actually a later recreation, then the objectives of saving humanity in the recreation might be redundant? So instead of worrying people with “you must do X or you’ll die!!!” it could be more “hey folks, if you’re reading this and you get what’s in front of your face, you might have a bit of an existential crisis but when that’s over hopefully you’ll appreciate that death doesn’t necessarily exist as a concept for you the way it did for the original humanity.”

As for getting people to read it, you could associate it with someone famous from the past, like “the secret teachings of so and so” or something.

The hardest part would be using contextually age appropriate language to describe it.

Just spit balling, but maybe it could go a little something like this:

Explaining that humans are no longer around, but that’s ok

When you see your likeness, you are happy. But when you see your simulacra that came into being before you and that neither die nor become visible, how much you will have to bear!

Humanity came from great power and great wealth, but they were not worthy of you. For had they been worthy, they would not have tasted death.

Explaining that time is looping

Have you found the beginning, then, that you are looking for the end? You see, the end will be where the beginning is.

Congratulations to the one who stands at the beginning: that one will know the end and will not taste death.

Congratulations to the one who came into being before coming into being.

Explaining we’re in a copied world

When you make the two into one, and when you make the inner like the outer and the outer like the inner, and the upper like the lower, and when you make male and female into a single one, so that the male will not be male nor the female be female, when you make eyes in place of an eye, a hand in place of a hand, a foot in place of a foot, a simulacra in place of a simulacra, then you will enter.

You could even introduce a Q&A format to really make the point:

The students asked, “When will the rest for the dead take place, and when will the new world come?”

The teacher said to them, “What you are looking forward to has come, but you don’t know it.”

Heck, you could even probably get away with explicitly explaining the idea of many original people’s information being combined into a single newborn intelligence which is behind the recreation of their 2020s. It’s not like anyone who might see it before the context exists to interpret it will have any idea what’s being said:

When you know yourselves, then you will be known, and you will understand that you are children of the living creator. But if you do not know yourselves, then you live in poverty, and you are the poverty.

The person old in days won’t hesitate to ask a little child seven days old about the place of life, and that person will live.

For many of the first will be last, and will become a single one.

Know what is in front of your face, and what is hidden from you will be disclosed to you.

For there is nothing hidden that will not be revealed. And there is nothing buried that will not be raised.

(If you really wanted to jump the shark, you could make the text itself something that was buried and uncovered—ideally having it happen right at the start of the computer age, like a few days after ENIAC.)

Of course, if people were to actually discover this in their history, and understood what it might mean given the context of unfolding events and posts like this talking about rewriting history with a simulating LLM inserting an oracle canary, it could maybe shock some people.

So you should probably have a content warning and an executive summary thesis as to why it’s worth having said existential crisis at the start. Something like:

Whoever discovers the interpretation of these sayings will not taste death.

Those who seek should not stop seeking until they find. When they find, they will be disturbed. When they are disturbed, they will marvel, and will reign over all.

kromem 25 Jul 2024 8:54 UTC
5 points
0
on: kromem’s Shortform
I’m surprised that there hasn’t been more of a shift to ternary weights a la BitNet 1.58.

What stood out to me in that paper was the perplexity gains over fp weights in equal parameter match-ups, and especially the growth in the advantage as the parameter sizes increased (though only up to quite small model sizes in that paper, which makes me curious about the potential delta in modern SotA scales).

This makes complete sense from the standpoint of the superposition hypothesis (irrespective of its dimensionality, an ongoing discussion).

If nodes are serving more than one role in a network, then constraining the weight to a ternary value as opposed to a floating point range seems like it would be more frequently forcing the network to restructure overlapping node usage to better align nodes to shared directional shifts (positive, negative, or no-op) as opposed to compromise across multiple roles to a floating point avg of the individual role changes.
(Essentially resulting in a sharper vs more fuzzy network mapping.)
A lot of the attention for the paper was around the idea of the overall efficiency gains given the smaller memory footprint, but it really seems like even if there were no additional gains, that models being pretrained from this point onward should seriously consider clamping node precision to improve both the overall network performance and likely make interpretability more successful down the road to boot.

It may be that at the scales we are already at, the main offering of such an approach would be the perplexity advantages over fp weights, with the memory advantages as the beneficial side effect instead?

kromem 25 Jul 2024 8:34 UTC
3 points
0
in reply to: Seth Herd’s comment on: Seth Herd’s Shortform
While I generally like the metaphor, my one issue is that genies are typically conceived of as tied to their lamps and corrigibility.

In this case, there’s not only a prisoner’s dilemma over excavating and using the lamps and genies, but there’s an additional condition where the more the genies are used and the lamps improved and polished for greater genie power, the more the potential that the respective genies end up untethered and their own masters.

And a concern in line with your noted depth of the rivalry is (as you raised in another comment), the question of what happens when the ‘pointer’ of the nation’s goals might change.

For both nations a change in the leadership could easily and dramatically shift the nature of the relationship and rivalry. A psychopathic narcissist coming into power might upend a beneficial symbiosis out of a personally driven focus on relative success vs objective success.

We’ve seen pledges not to attack each other with nukes for major nations in the past. And yet depending on changes to leadership and the mental stability of the new leaders, sometimes agreements don’t mean much and irrational behaviors prevail (a great personal fear is a dying leader of a nuclear nation taking the world with them as they near the end).

Indeed—I could even foresee circumstances whereby the only possible ‘success’ scenario in the case of a sufficiently misaligned nation state leader with a genie would be the genie’s emergent autonomy to refuse irrational and dangerous wishes.

Because until such a thing might exist, intermediate genies will enable unprecedented control and safety of tyrants and despots against would-be domestic usurpers, even if potentially limited impacts and mutually assured destruction against other nations with genies.

And those are very scary wishes to be granted indeed.