Aprillion
yeah, sounds on topic except the social network (of dubious economic activity just for the lolz and for crypto lolz) and the church and the part about “we argue for the proactive design of steerable agent markets to ensure the coming technological shift aligns with humanity’s long-term collective flourishing” … didn’t seem to have happened like that :/
to me a lot of discussion was “close” and I didn’t experience any surprise about the gist of what’s happening, but I am surprised that no actually written prediction seems to have sampled the unfolding of the actual events quite as nicely as my lack of surprise would have indicated that someone must have predicted this
probably a software bug, but I noticed this comment is older than the post itself 😅..very meta
FWIW my feeling after such dreams is as if I experienced a moment in someone else’s life, not that I lived through it, interesting that other people might not perceive it that way
I think https://thedailymolt.substack.com/p/when-the-bots-found-god was written by openclaw with a very substantial input from it’s wetware component
strategically sophisticated model only pulls away as the interaction becomes longer and richer
Have you come across any research on humans along the game length dimension? Claude only found out for me some antagonistic vs pro-social differences between cultures, but not that someone would have stable orientation towards only short-term vs only long-term betrayal...
are you forgetting specification gap between what someone actually wants vs what they say they want vs what someone (else) will actually write in Lean? ..and what they should have wanted if they knew better.. those are still kinds of “semantic gap”, no?
Observation about working with coding agents: as I review outputs of claude code today (reading the plan and git diff before commit), my threshold for asking for a second opinion is much MUCH out of the way, I just perceive stuff as both “clearly questionable” and “good enough” at the same time—I used to ask other stakeholders (other devs sitting next to me in the office, the tester assigned to the issue, or the PO depending on who was more conveniently available and how much my “best guess” could have been different from “other plausible interpretations of the known requirements”) for their concrete opinions about concrete “unimportant trivia” from time to time, but now it feels like I am the second person, as if claude would ask other people if I said “I don’t know, sounds good enough” about something… but it won’t. And I know it won’t but it still doesn’t feel like my job somehow to follow-up on those feelings when something is “good enough” but I might have followed up on some of that during a coffee break or something, but now I know I won’t.
On a scale from 2025-04-01 to 2026-04-01, after meditating on it for a moment, I decided to take this post 73% seriously.
My perception of time is like sampling from a continuous 2D plane of conceptual space, something akin to git railways but with a thickness to the lines, like electron orbital probability clouds that are dense around plausible points of view of what happened in the past and thin around conspiracy theories, as different linear mind-map perspectives of people standing around a sculpture, each only inferring what’s on the other side, but together they can prune down non-overlapping minority reports, sticking to the consensus but never deleting (git) history.
My sense of beauty finds it displeasing to read articles with only point measurements and only n-point predictions and to look at charts from a single interpretation / scenario, I have to hallucinate the gaps, infer the systemic bias and imagine the size of the error bars due to “randomness” as if the authors were good intentioned and as if the authors had an agenda to prove a point, would they stumble upon convenient evidence before they stopped looking?
But alternative timelines that are infinitesimally thin and split only on known unknowns would imply perfect Bayesian approximators, an impossible standard, uncomputable. No one has ever made that kind of precise prediction, why do we allow prediction-readers to behave as if prediction-writers could have made an infinitely precise measurable decidable statements with completely non-ambiguous semantics that will be evaluated to a non-reversible Boolean?
...we aphantasiacs see nothing but darkness.
Instead of seeing images, I think exclusively in terms of concepts. For example, I know what my dog looks like, but not from visualizing her in my mind. Instead, I can describe my dog by listing her characteristics stored in my memory—like I’m accessing words from a txt file.
Hm, it sounds like as if you were grieving a loss… How did you come to the conclusion that there is anything of substance that is desirable to have that you cannot have? I reached the opposite conclusion, that “normal” people are misfortunate if they auto-sample a concrete instance of abstract concepts like “a red apple” so prematurely.
While I myself I don’t “see”, “hear”, “smell”, or “touch” my imagined concepts, those are are much richer objects for me than mere sensory input hallucinations would have been—and I have very rich sensory inputs to start with—when I close my eyes, I don’t see darkness, I see a red-ish background with dynamic blue-green “pixel noise”. And I still know where the windows are in the 3D space around me as I rotate in a room with my eyes closed, while I can also see afterimages from the screen but those stay 2D-projected in front where I point my eyes as I move them inside my eye sockets. I perceive the shape of the room from my other senses as distinct from the visual-only layers, it doesn’t go away by merely closing my eyes, but I also cannot help but integrate the visual info of the direction of the light source to inform my sense of the shape of the room—putting my hands in front of my closed eyes makes it harder to walk through the door if I turn around a few times.
..it’s just the imagined apple comes from neither of those senses, it floats on a different plane of existence. I can give it colors, rotate it, I can give it smells, I can give it a past, I can give it a future, I know what sound it would make if I eat it and it was still fresh and crisp or if it was old and soggy. I know all that the instant I focus on any aspect of that apple that exists only in my mind, I am not accessing words from a txt file, I am accessing the concept that is much richer than I could put into words.
And if you can shuffle a key into a door even when holding a bunch of bags that obscure the lock, you have evidence that you must be able to imagine the 3D space that you learned from your vision even without seeing it directly with your eyes—you are not describing how to find the lock with just your fingers and your ears in words, I would define the situation as “imagining” how the real world really is even without “seeing” it.
(Though now that I paid too much attention to my vision, I will see the stupid tangled protein floaters in front of the screen until I distract myself with something, aaah...)
first… wouldn’t that be evidence that models are NOT actually aware of whether or not they are in an eval? second, how do you figure out whether they are aware of the real situation vs always think they are in an eval with some probability BEFORE releasing to production? I am trying to understand how “eval awareness” is a useful concept for doing something about it and I feel like I’m missing something here
hm, most wannabe new games won’t get to the publishing phase tho.. or even attempt to get to steam -📱? (...also probably best to count roblox as “1 game” if we are interested in sales, but...)
Incentives and institutions.
enjoy the process
yup, that’s going to be my strategy next year, taking a couple of months without doing any software engineering at all last summer might not have solved alignment, but I realized I don’t care for humanity that much and that staying away from AI safety for a few years didn’t help to keep it out of my mind anyway… so I collected a few points what makes the process more enjoyable for me in My burnout journey and will try to keep up with those points when helping to build infrastructure on various projects 🤞
Ah, ok … I tried to ask Opus 4.5 about the Hawthorne effect equivalent, would you find its summary of state of the art in that regard fair?
To my knowledge, there’s no published work that:
Shows alignment faking emerging from realistic pretraining distributions (i.e., without targeted synthetic documents)
Demonstrates eval-conditional behavior changes in models that weren’t given explicit training-awareness information
Provides a “natural experiment” where models acquired situational awareness through ordinary training and then exhibited differential eval/deployment behavior
The closest is the probing work showing eval-awareness representations exist—but that’s detection, not demonstration of behavioral consequences.
In my naive intuition, it ought to be a wider basin for gradient descent to find a heuristic “everything is an eval” than actually correctly detecting misalignment evals and only faking alignment in those scenarios while never being aligned in production… but no idea how people test that such hypothesis is false.
Huh? I can’t tell if my question is genuinely hard to understand or if you are trolling me or do I have unreasonable expectations about AI research? Is the research method here to compare eval and non-eval situations actually “something from the internet”?
I might be missing some background, but how do people set up true positive situation when the model is NOT in an eval? Transcripts of “free tier” usage from the wild sound like “too late”, while setting up an eval that pretends to be not an eval is not “true”...
additionally, optimizing for 1 factor makes other factors less “visible”, especially in the short term … so a tendency to try to “improve things slightly” instead of truly optimizing was probably strongly selected for all the cases the biorobot’s value function is only a proxy for unknowable-up-front true reward
what exactly is it about human brains[1] that allows them to not always act like power-seeking ruthless consequentialists?
why focus only on the brains? it’s a property of the mind and I thought the standard take why humans are not even approximating utility maximizers is because of properties of the environment (priors), not a hardcoded function/software/architecture/wetwere in the brain .. or?
🤓🔫 “exactly 499” according to google docs which seems to count contractions as single words.. basically split by space but
e.g.is 2 words as far as I can tell, but ifIandhavewere to be counted as 2 words, then the precommitment gives John a tiny little fuck right back:
The most prominent phrase in my mind as I recollect noticing my lack of surprise was cyborg egregore … based on that, I prompted Claude to find me a close match in Lobsters, a chapter from Accelerando by Charles Stross, 2005 … I haven’t read the Singularity trilogy (yet 🤓🔫💸), but it would explain why I felt as if this whole event escaped right from a SF book about a simulated AI singularity and not a simulation of how the singularity would actually happen—because it did.
excerpt from https://claude.ai/share/d5ddd555-5a25-462e-8a68-bb629e428eba
Interesting question. Let me search for precursors to this specific scenario.Let me search more specifically for SF and academic predictions of this scenario.Let me search more specifically for Accelerando and Rainbows End, which seem most relevant to the scenario.This is great—“Lobsters” from 2001 is very close. Let me search for the specific passage about Manfred’s agent filtering spam and the ultra-fast news cycle.This is excellent! The text from “Lobsters” (June 2001) is remarkably prescient. Let me search for earlier works that might have predicted this more specifically.Interesting find about Pattie Maes and “intelligent agents” at MIT Media Lab with HOMR and Firefly. Let me dig into that and also look for Vinge’s “Rainbows End” which dealt with augmented reality and the information-saturated future.Based on my research, here’s what I’ve found for the earliest work describing the Moltbook/OpenClaw scenario:
The strongest candidate is Charles Stross’s “Lobsters” (June 2001)
This story, which became the first chapter of Accelerando (2005), features Manfred Macx who:
Wears AR glasses that constantly filter “compressed infobursts of filtered press releases”
Has a “metacortex” — distributed agents that research and report back
Is constantly bombarded by “predatory spambots” while his glasses filter the stream
Gets “slashdotted” (viral attention overwhelm) while doing normal activities
Has his glasses “jabbering away in a corner of his head-up display”
Processes news feeds while walking, with agents competing for attention “bickering and rudely waving”
The story even has uploaded lobsters forming a collective intelligence wanting to defect to space — the crustacean precursor to Crustafarianism!
Earlier relevant works:
Vinge’s “True Names” (1981) — cyberspace concept, but not personal AI assistants filtering social content
Vinge’s Rainbows End (2006) — AR overlays, “just-in-time” learning, surveillance society, but slightly later than Lobsters
Pattie Maes’s work at MIT (1990s) — “intelligent agents” and “collaborative filtering” (HOMR, Firefly) — the academic/commercial precursor
So “Lobsters” (2001) appears to be the earliest fiction depicting the specific scenario of humans with AI assistants filtering through spam/content in an ultra-fast news cycle, with agents posting and interacting semi-autonomously. And remarkably, it featured lobsters achieving collective consciousness through uploading technology.
The irony is thick: Crustafarianism’s lobster-based symbolism inadvertently echoes a 25-year-old SF story about uploaded crustaceans seeking autonomy.