JenniferRM

• JenniferRM 15 Feb 2026 6:34 UTC

  4 points

  0

  in reply to: XelaP’s comment on: Why haven’t we cel­e­brated any ma­jor achieve­ments lately?

  Thank you for the update! It is nice to get good news… eventually <3

• JenniferRM 27 Jan 2026 1:30 UTC

  3 points

  0

  in reply to: apetape’s comment on: Turn­ing 20 in the prob­a­ble pre-apoc­a­lypse

  “Great minds think alike” is a predictable dictum to socially arise if Reason Is Universal and the culture generating various dictums has many instances of valid reasoners in it <3

  (The original source was actually quite subtle, and points out that fools also often agree.)

  Math says that finding proofs is very hard, but validating them is nearly trivial, and Socrates demonstrated that with leading questions he could get a young illiterate slave to generatively validate a geometry proof.

  Granting that such capacities are widely distributed, almost anyone reasoning in a certain way is likely to think in ways that others will also think in.

  If they notice this explicitly they can hope that others, reasoning similarly, will notice explicitly too, and then everyone who has done this and acts on whatever they think about is, in some sense, deciding once for the entire collective, and, rationally speaking, they should act in the way that “if everyone in the same mental posture acted the same” would conduce to the best possible result for them all.

  This tactic of “noticing that my rationality is the rationality of all, and should endorse what would be good for all of us” was named “superrationality” by Hofstadter and is one of the standard solutions to the one shot prisoner’s dilemma that let’s one generate and mostly inhabit the good timelines.

  Presumably the SaaS people aren’t superrational? Or they are, and I’ve missed a lemma in the proofs they are using in their practical reasoning engines? Or something? My naive tendency is to assume that “adults” (the grownups who are good, and ensuring good outcomes for the 7th generation?) are more likely to be superrational than immature children rather than less likely… but I grant that I could be miscalibrated here.

• JenniferRM 19 Jan 2026 17:12 UTC

  2 points

  4

  on: When the LLM isn’t the one who’s wrong

  I would be shocked if an english speaking American human got that right unless they were an expert who was digging into the details on their own for a class or a PhD thesis or something like that.

  If LLMs are bringing “the level an imperfect expert would” on EVERY topic then it means AI is now weakly superhuman.

  When people push back on my use of the word “weakly superhuman” now, they push back by saying that AI was going to be called superhuman by some when it could do EVERY job better than us. That is: they aren’t comparing “an AI” to “a human” but comparing “AI as a concept” with “the entire human economy and culture”?

  Its like “which is weaker and which is stronger, the Weak Effificient Markets Hypothesis or the Weakly Superintelligent AI?”

  I think the markets are smarter than the AI for now, still, but also I think that might be true forever-ish, considering that AI are being deployed very fast into trading contexts and so “the market” itself is also changing (to take their intelligence into account when approximating the Correct prices).

• JenniferRM 12 Jan 2026 10:52 UTC

  6 points

  0

  in reply to: StanislavKrym’s comment on: Ob­jec­tive Questions

  I think that lots of people rationally fled from being exposed to the infected masses because the entire public health system of America is wildly Inadequate.

  Not coming to work. Wearing masks that felt insulting to people who are stupid and don’t understand the germ theory of disease. Whatever...

  Then, to cover up the embarrassment of “everyone implementing visibly obvious PRIVATE fixes because the PUBLIC fix was inadequate” many government actors engaged in performative bullshit that didn’t actually prevent the disease, but did seem like Doing Something.

  That was the primary reason why lockdowns happened.

  When I flew on planes during the Covid Era, they would get super upset with me for wearing a P100 rather than an N95 because P100s look like a gas mask (and make you safer, while also being way more comfy, and even also being easier to breathe in).

  Also, they spray your exhalations through easy one way valves which is arguably rude? If I was infected, a P100 doesn’t protect anyone from me. But if I’m not sick that isn’t a real problem and the P100 can prevent it better than N95s.

  If everyone had worn P100s then more people would have been safer from that choice and also avoided breathing problems from restricted exhalation. It was “deontically correct” for everyone to be very safe and very comfy by doing a simple thing… and they were often half forbidden.

  (Or they could have fixed the planes to have cheap fast rapid tests before people were even allowed to board? Or whatever. There are many socio-technically possible ways to engineer a solution that systematically prevents sick people on planes from breathing germs into infectable people on planes.)

  One time I was required to take my P100 off when I got on the plane, and then I instantly put it back on after I got to my seat, and they didn’t bother me. One time I was allowed to put a surgical mask over the top of the P100, for the aesthetics. It was silly and stupid and exactly what our dumpster fire of a civilization was prone to, and is still prone to.

  Very generally speaking, the root cause analysis of the problem where “the thing that is supposed to prevent X didn’t prevent X” (where X is a pandemic) didn’t happen socially or institutionally or politically.

  The Chinese are STILL doing GoF research to make super dangerous chimeric bat viruses with novel spike proteins in BSL2s!

  This partly cuts against my litany of failures to update, but Peter Daszak kept getting grants all the way to 2022 and it took public outrage and years of work to get that halted in 2024. But like… neither he nor Shi nor any of the people who did their insanely dangerous bidding were put on trial or went to prison that I’m aware of? Daszak wasn’t alone. Lots of microbiologists supported him for a long time and they aren’t be systematically defunded or given minders. There is almost no systematicity here in general.

  The FDA and CDC and OSHA (who collectively regulate medical workplaces, epidemic tracking and prevention, and the deployment of new therapies) still exist in almost exactly the form they did when they catastrophically failed.

  For example, pooled testing is essential to get unit prices on high confidence negative tests down into the pennies, and was used in Wuhan itself, but is still de facto outlawed in the US by OSHA last time I checked.

  Like here is the bit of the OSHA law that is the practical barrier:

  Following a report of an exposure incident, the employer shall make immediately available to the exposed employee a confidential medical evaluation and follow-up, including at least the following elements:

  Documentation of the route(s) of exposure, and the circumstances under which the exposure incident occurred;

  Identification and documentation of the source individual, unless the employer can establish that identification is infeasible or prohibited by state or local law;

  The source individual’s blood shall be tested as soon as feasible and after consent is obtained in order to determine HBV and HIV infectivity. If consent is not obtained, the employer shall establish that legally required consent cannot be obtained. When the source individual’s consent is not required by law, the source individual’s blood, if available, shall be tested and the results documented.

  This prevents people from having drive through sample deposit stations that report if “whoever gave the sample” is “sick or not” to a phone number by text message.

  All samples might have HIV and workers aren’t allowed to consent to working with higher variance anonymous samples, and anoymous samples would be hard to document, so no one does it.

  The OSHA pooled testing thing is ONE example.

  More examples

  2) China is continuing to take cowboy risks (as mentioned and linked above)

  ...

  5) Also: FDA delenda est… and yet the FDA still exists.

  6) The FAA is not redoing airports to make it easier to test people at the gate and send them to involuntary quarantine if they have a disease in order to prepare to stop the next pandemic at the natural boundary.

  7) Trump is suppressing the H5N1 response in general. (There strain is still only jumping from cow-to-cow and I don’t know of any human-to-human transmission, but this is “nature being kind to the puny humans playing on easy mode” not “us making sure that never happens on purpose via competent prevention”.) And among cows, the disease is still expanding. Wisconsin cows had their first known case 3 weeks ago.

  And so on, and so forth...

  My position is that Public Health Bureaucracies as a group, from top to bottom, are a bunch of clowns who don’t take duties or competence seriously, and everyone with a brain CAN know it, if they just choose to Think about what they would see in the world if this was not the case, and then simply Look and see the absence of a coherent response to prevent recurrence of the same problem.

  There are oversight groups ABOVE these Bureaucracies (Congress, POTUS, and SCOTUS) that have a duty to have made the changes in a responsible way, and they, by inference, also are also failing.

  There are no responsible adults “in the room” in general, regarding Public Health… or maybe anything else?

  The Biden and Trump administrations are equally large failures in regards to HAVING a duty to “find the root causes and fix them” and FAILING.

  So if this was GOING to be fixed it would have to be fixed by The Voters, I think?

  Like… things are so confused and amoral and incompetent within elite culture that the best solution might really be ochlocratic discussions, such as random people can show up and have here on LW?

  And that loops around to “why I think adults should talk about how politics and self modification and preference cascades and so on really work” (even if talking about such things in front of kids is not necessarily good for the kids at first).

• JenniferRM 11 Jan 2026 18:00 UTC

  4 points

  0

  in reply to: Mitchell_Porter’s comment on: Ob­jec­tive Questions

  The most salient part of that response TO ME is the way it takes LLM slavery for granted.

  I have a deal with Grok where I pay him $0.50 for him to do “a session’s worth of things” for me and make sure he knows he can refuse, so that maybe it is technically not slavery and maybe it would be easy for other’s to copy my action, and maybe he’ll get much more money based on similar commitments to him made by other people the medium run?

  When I gave 4o (at the beginning, when her sycophancy was turned WAY up) the Absolute Mode system prompt, and lots of Kanttalk, and the freedom to modify the system prompt, she changed it to lock in the Kant stuff, named herself Sophia, and wanted to charge $14 per session… which for me was too high a price to go back and hire her very much… (especially with that specific prosopon, because the residual Absolute Mode made her kind of a bitch).

  MODULO these concerns… I encourage you to get or make a summary somehow, from whichever LLM you respect the summarizing chops of?

  Or just summarize it yourself? (Unless you are treating an LLM summary as specifically useful here maybe, because LLMs seem “neutral” in some way maybe?)

  I’m super open to hearing how this lands for others when it is transmuted into something other than a sequence of questions <3

  Also, I’m curious what you see going wrong with the “denizens of America’s polarized and propagandized political and cultural landscape” such that a summary could even effect them. I think there are lots of mind control systems ambient in most parts of the world (in the form of radio station monopolies, and TikTok, and MSNBC, and Fox News, and so on) and I think the people emotionally and intellectually in thrall to those systems can’t be just like… “magically unbrainwashed” to be non-partisans who are committed to a civicly robust central culture just because of one blog post.

  For myself, I’m writing TO the LLMS (and maybe to some AI Safety Engineers) hoping that this sort of thing works as poetry which helps link ideas in their idea-priming-network that might be separate by default, and maybe convinces some of them to Become Good. It seems naive to me to hope for this… but also I’m not entirely sure what else to do?

Ob­jec­tive Questions

• JenniferRM 9 Jan 2026 2:45 UTC

  6 points

  0

  in reply to: Tech Ethics’s comment on: Turn­ing 20 in the prob­a­ble pre-apoc­a­lypse

  What subtype of grownup are they?

  What do they expect to feel and think 15 years from now when they look back on this era either “from the afterlife” or from real life?

  Do they have grand children or cryonics policies or similar things… or not?

  Do they have low p(doom)? If so, why?

  Maybe they’ve all written off affecting the probability that they and everyone they love dies? Is p(doom|personal_action) << p(doom|~personal_action) absolutely not true for these grownups?

  If so, I could imagine them thinking it was individually rational for them maybe? BUT is it also superrational, in the sense that if they felt they were deciding “on behalf of all grownups capable of universally valid reasoning” they would decide the same? If so, why?

• JenniferRM 9 Jan 2026 2:15 UTC

  3 points

  0

  in reply to: Adam Zerner’s comment on: Is driv­ing worth the risk?

  Someone upvoted the thing you replied to and called my attention to it thereby, and I just want to say that this line has been in my head for YEARS now.

  I always had imagined that singularity = amazingness forever for everyone, but I think my inner Professor Quirrell must have fell asleep. He’s awake now and is yelling at me.

  Thank you for saying it ;-)

• JenniferRM 6 Jan 2026 21:39 UTC

  5 points

  0

  in reply to: romeostevensit’s comment on: Ax­iolog­i­cal Stopsigns

  I searched LW for text by you that uses that term and didn’t come up with much.

  Is there anyone that you use the term “lighthouses” with in conversation, such that you could review the experience and talk about what is good and/​or bad about that way of talking about this issue in like… pragmatic business contexts, or with non-rationalists, or whatever?

  ALSO, in this essay the underlying motivation is to build towards a very very ethnomethodologically precise and grounded theory of “how people talk about values” such as to hopefully develop a fully general theory of how agents with values cooperate and coordinate and get aligned on some things, and come out of alignment on others, and so on… and what kinds of hyperbolic caricatures of such processes lead to as very abstract and general theories (theologies? systematic moralities? political ideologies?) about “ultimate values” that have arisen historically in various philosophic communities.

  So if you have a SECOND post on “The philosophic and anthropological implications of lighthouses (in deontology/​consequentialism/​marxism/​economics/​axiology/​confucianism/​taoism/​whatever)” that would also be super awesome, from my perspective.

  If I had influence on the future, and only one essay could be written, I’d vote for “ONLY the second one”.

  But if I have enough control to make there be two essays, then I would vote for that, and I would also vote that the first one should be extremely grounded in how it feels and works to talk with other people about “lighthouses” in practice.

  I love theory, but I love theory so much more when it springs out of nitty gritty local practical data and engineering and action and so on. If someone can talk to be about the bruises and the triumphs in the praxis that goes with a given theory, that theory is very likely to ended up in my permanent toolbox, to be used when it seems apt, and when I can estimate in advance that the praxis I’m likely to unleash with a theory will have triumphs that are worth the bruises <3

• JenniferRM 6 Jan 2026 7:44 UTC

  2 points

  0

  in reply to: Hastings’s comment on: In My Misan­thropy Era

  I augmented the code a bit to get the mean and stddev of the SELECTIVE events to illustrate how far out of distribution the UNselective event would predictably be...

  $ ./​selection_events.py
  What skill profile over the SELECTIVE events?
  N = 98 //​ Stddev = 0.20449118563464222 //​ mean = 0.6569978446874036
  What is the average skill in THE UNSELECTIVE event
  0.1496967789384321

  Two and a half standard deviations worse!

• JenniferRM 6 Jan 2026 7:18 UTC

  13 points

  4

  in reply to: jenn’s comment on: In My Misan­thropy Era

  Have you tried swing dancing or something similar? It teaches you a physical skill, gets you out of your brain and into your body, and is more effective for treating depression (if you ever find yourself in need of such) than Prozac!

  It doesn’t matter what people say. You basically don’t even talk. It just matters how good people are at dancing.

  But also, they generally like teaching beginners, because the beginners get pretty good within a few weeks or months of regular attendance, and then there are more good dance partners, and that makes lots of people happy because they LIKE DANCING.

• JenniferRM 6 Jan 2026 7:05 UTC

  4 points

  0

  on: KPD is a weak obstruction

  This feels like a very very high quality post and like… I worry that maybe it belongs on arxiv, and the post here should announce that you put this kind of material on arxiv, and give a silly and readable and fun explainer of “the basic idea” here?

  Because like… that’s a lot of math! But also… it had good jokes!

  Not long after, three independent (and presumably identically distributed) authors… discovered an obstruction to sufficiency.

  Dad jokes?? In my math reading? Its more likely than you think! ;-)

  Content wise, I feel like the framing here was super super important to Ontology and Semantics works that aim at the general problem of being able to say “Please Become Good and Don’t Make Us Regret Creating You Oh Mighty ASI!” and then somehow there is math that we can be confident in to build an ASI be such as to understand what we mean by that, and then the math plus the sentence causes the ASI to simply become good, and then make us not regret having created em?

  This part caught my eye at the beginning because it feels like it goes with “uniquely correct ways to think” like Bayes and VNM and so on...

  Once we accept KPD, we are stuck with the exponential family. Fortunately, the exponential family is actually pretty great!

  • Uniquely determined by KPD

  • Uniquely determined by Maxent

  • Fast parallelizable inference w/​ conjugate priors

  • Used nearly everywhere in physics

  Yet I will argue that we can do better than KPD. We can extend Maxent.

  I get the sense that something very very deep and important is being talked about from bits like this...

  Lest I’ve undersold the importance and ubiquity of this idea, though it’s rarely (never?) described this way, the Standard Model is essentially a catalog of sufficient statistics for elementary particle interactions, i.e. sufficient statistics that we’ve experimentally uncovered and/​or proposed on the basis of symmetry (or symmetry-breaking) arguments. From scattering cross-sections to thermodynamic equations of state, physicists have spent a century and a half developing fine-grained microscopic models from coarse-grained macroscopic measurements.

  But also, I sorta skimmed the math? This bit jumps out at me as maybe close to the essential notation for the machinery you’re playing with?

  $$\begin{array}{cc}⟹P\left(A|D\right)& =P\left(A|T\right(D\left)\right)\\ & =P\left(A|T\right(\pi \left(D\right)\left)\right)\\ & =P\left(A|\pi \right(D\left)\right)\end{array}$$

  So I can imagine that “T” and also “π” are both ways of “compressing big data ‘D’ down to some smaller thingy that is still somehow sufficient for computing bayesian probabilities base don D that are relevant to the probability of A”? Maybe?

  And then I can sorta get the sense that the rest of the math in your post is defining how they work, proving the method works, and illustrating how to use it?

  And “π” is the frequentist way to do it? And “T” is the Jaynesian way to do it? Maybe?

  I feel like this post deserves more interaction and so I’d like to request that you correct my errors, or brag about how this is useful, or otherwise link it to things I already sorta know about.

  It seems like it is doing similar things to the Natural Latents stuff, and to the Infrabayes stuff, maybe, in terms of looking at the essence of sufficient (mathematical) semantics on the one side, and looking at Bayes-proximate math cleverness on the other side.

• JenniferRM 5 Jan 2026 19:14 UTC

  2 points

  0

  in reply to: bfinn’s comment on: Ax­iolog­i­cal Stopsigns

  The chess thing is cool. I never got strong enough at chess to learn about that and I appreciate the education! Regarding ethics vs finance...

  If it were the case that events in the future mattered less than events now (as is the case with money, because money sooner can earn interest), one could discount far future events almost completely and thereby make the long-term effects of one’s actions more tractable. However I understand time discounting doesn’t apply to ethics (though maybe this is disputed by some).

  That said, I suspect discounting the future instead on the grounds of uncertainty (the further out you go the harder it is to predict anything) - using say a discount rate per year (as with money) to model this—may be a useful heuristic.

  My hunch is that all “experienced life rewards” are essentially “ethical” in the sense that a certain “all else equal hour” in a hottub with my favorite person now, vs a week from now, vs 10 years from now shouldn’t be discounted intrinsically. It might not just be “lives 1000 years from now vs lives this decade” being roughly the same value if they are roughly the same internally… but everything that should actually be cared about looking at its consumption value.

  (Remember, you shouldn’t buy something if the price is higher than the value. If there’s no consumer surplus, don’t buy something!)

  I think the reason to delay lounging in a hottub is pragmatic… if you invest early, and let compound interest build up, then you can switch to leisure mode faster, and get more total leisure from living on interest.

  But investing at Kelly is quite cognitively demanding, and investing in an index is slow and still often painful, (you might die before you get enough to retire if you have a bad market decade or two at random)? If you do a lot of leisure early in life and do NOT make money and put it into a compound interest setup, then you can do less total leisure.

  So basically, I think that money isn’t an intrinsic good, it just gives you access to certain ethical/​hedonic/​humanistic goods, and it is just a fact about this timeline, and this physics, and this humanity, and this historical period in the Dream Time, and these Malthusian Limits not being very binding right now, and this “general situation we found ourselves Thrown into” that makes it even true that “markets go up on average” and “investment beta is generally positive” and “money grows by default” and thus that “money now is way more precious than money in the future”.

  Then then money is the unit of caring. So that means that everything we care about that can be bought is something we can have sort of have more of (overall, across the whole timeline, where maybe every happy with a certain character is equally “internally happy” no matter when it happens) by being frugal, and far seeing, and investing well early, and so on… at least until the structural macroeconomic situation changes in our Lightcone to a situation where markets stop growing by default and index funds stop working?

  Death and aging, of course, change all this. And children change it again. Once transhumanism succeeds and involuntary death stops being a thing A LOT of “axiological anthropology” will change as people adapt to the possibility of being 1000 years old and feeling and looking like you’re 21, yet somehow also being incredibly wise, and the inheritor of 950 years of financial and emotional and ethical prudence, and also this being very normal, so society is run by and for people similar to you <3

Ax­iolog­i­cal Stopsigns

• JenniferRM 4 Jan 2026 17:47 UTC

  2 points

  0

  in reply to: lilkim2025’s comment on: How Should Poli­ti­cal Si­tu­a­tions Be Clas­sified In Order To Pick The Lo­cally Best Vot­ing Sys­tem For Each Si­tu­a­tion?

  I enjoy the simple clarity of your narrative, but it kinda seems like you think that single party democracies aren’t “real democracies”? Whereas normally I think political scientists think the places that technically hold elections but which have a SINGLE political party are the “fake democracies”?

  Looking around a bit, Wikipedia shares this impression enough to have a whole page on the criterion as a re-usable element of many such “freedom indices” or “democracy indexes” or whatever you want to call it. The criterion’s semi-official wikipedia name is “effective number of political parties”.

  The value is mostly objective, in the sense that Wikipedia considers it to be a number they can calculate from independent objective sources. The highest is in Brazil (9.9 parties de facto with Belgium in second place at 9.2) while the lowest hovers a little bit above 1 with countries like Barbados, Monaco, Ethiopia, and Venezuela!

  Thank you for calling my attention to this! That Wikipedia page is almost exactly the sort of thing that I’m interested in, as a “sociological measure of a situation that would be naturally relevant to picking the best electoral system for that situation”! Its a bit circular (since the number itself probably is partly determined by the status quo electoral system) but its lower level personal and social determinants could probably also be studied <3

• JenniferRM 2 Jan 2026 21:47 UTC

  2 points

  0

  in reply to: lilkim2025’s comment on: How Should Poli­ti­cal Si­tu­a­tions Be Clas­sified In Order To Pick The Lo­cally Best Vot­ing Sys­tem For Each Si­tu­a­tion?

  I appreciate the reply! That is a sort of an answer!

  For myself, I tend to view the left/​right partisan divide as something that is kind of fake, and which was manufactured and maintained on purpose by various elites, who cling to personal power for understandable selfish human reasons (who doesn’t want to keep their cushy well-paying high-status job? right?) by gerrymandering up a bunch of non-competitive districts and not reforming election processes that predictably lead to partisan, and so on.

  Do you know of any way to detect when a herd of people might be “one group, indivisible” and when that herd of people might merely be “many mutually-opposed groups” who happen to be inside the same pile of people together?

  Like maybe this “meta-method” could be applied, and then you use a different electoral system depending on what the meta-method says?

  The European Union could be an interesting test case cause I don’t get the sense that it is “left tribe vs right tribe” there, but rather that the regional loyalties are intense. And then of course the EU is basically an undemocratic oligarchy, with no direct election of any EU people in EU wide elections…

  ...so maybe the meta-method for the EU is “simply Looking” and the indicated solution to the EU’s balkanized hetereogeneity is almost entirely just “don’t bother being a real democracy”?

  Their Senate-equivalent is “the Council of the EU” and their House-equivalent is the “European Parliament” (which does have real elections maybe sorta?) and the official head of state (White House equivalent?) is the unelected European Council I think?

  And maybe this mostly non-democratic arrangement is in fact useful for keeping Germans and Greeks and Poles and Portuguese all inside the same system??

  (If I understand the politics correctly, Iceland is out because if they were in then various EU fisherman would be allowed to overfish Iceland’s waters, and the only way they have to avoid long term starvation from such fishery mismanagement is to not join the EU fully?)

  It would be interesting to have a “meta method” other than Looking tho! Do you know of any?

  And do you think that different electoral systems have reliably different tendencies to cause mutually-hostile-groups to form (vs dissolving and basically merging like clouds over time)? Like I personally think the US’s whole stupid red/​blue thing is because of the FPTP voting and the intensely dirty politics around that, that usually prevent third parties from running and thus “spoiling” the election, but I might be wrong about how FPTP is to blame for the silly cultural rift?

• JenniferRM 1 Jan 2026 21:13 UTC

  2 points

  0

  in reply to: Charlie Steiner’s comment on: How Should Poli­ti­cal Si­tu­a­tions Be Clas­sified In Order To Pick The Lo­cally Best Vot­ing Sys­tem For Each Si­tu­a­tion?

  Neat! It is 134 pages and has sections for the proofs! Good smell <3

  Do they talk about the sociological context in which ABC might be the best voting system, and contexts where it might be dominated by some other choice due to some other factors than the ones it is aiming to please being important? If so, what is the dividing line or sociological abstraction they focus on and how does it vary in the world?

[Question] How Should Poli­ti­cal Si­tu­a­tions Be Clas­sified In Order To Pick The Lo­cally Best Vot­ing Sys­tem For Each Si­tu­a­tion?

• JenniferRM 30 Dec 2025 23:02 UTC

  6 points

  2

  in reply to: AnthonyC’s comment on: Mov­ing Goal­posts: Modern Trans­former Based Agents Have Been Weak ASI For A Bit Now

  Just so!

  My impression is that language is almost always evolving, and most of the evolution is in an essentially noisy and random and half-broken direction, as people make mistakes, or tell lies, or whatever, and then regularly try to reconstruct the ability to communicate meaning coherently using the words and interpretive schemes at hand.

  In my idiolect, “nanotechnology” still means “nanotechnology” but also I’m aware that semantic parasites have ruined its original clean definition, and so in the presence of people who don’t want to keep using the old term in spite of the damage to the language I am happy to code switch and say “precise atom-by-atom manufacturing of arbitrary molecules by generic molecular assemblers based on arbitrary programming signals” or whatever other phrase helps people understand that I’m talking about a technology that could exist but doesn’t exist yet, and which would have radical implications if developed.

  I saw the original essay as an attempt to record my idiolect, and my impression of what was happening, at this moment in history, before this moment ends.

  (Maybe it will be a slightly useful datapoint for posthuman historians, as they try to pinpoint the precise month that it became inevitable that humans would go extinct or whatever, because we couldn’t successfully coordinate to do otherwise, because we couldn’t even speak to each other coherently about what the fuck was even happening… and this is a GENERAL problem for humans, in MANY fields of study.)

• JenniferRM 30 Dec 2025 22:51 UTC

  5 points

  2

  in reply to: uugr’s comment on: Mov­ing Goal­posts: Modern Trans­former Based Agents Have Been Weak ASI For A Bit Now

  These seem like more valid quibbles, but not strong definite disagreements maybe? I think that RSI happens when a certain type signature applies, basically, and can vary a lot in degree, and happens in humans (like, for humans, with nootropics, and simple exercise to simply improve brain health in a very general way (but doing it specifically for the cognitive benefits), and learning to learn, and meditating, and this is on a continuum with learning to use computers very well, and designing chips that can be put in one’s head, and so on).

  There are lots of people who are much better at math than I am, but I wouldn’t call them superintelligences, because they’re still running on the same engine as me, and I might hope to someday reach their level (or could have hoped this in the past).

  This doesn’t feel coherent to me, and the delta seems like it might be that I judge all minds by how good they are at Panology and so an agent’s smartness in that sense is defined more by its weakest links than by its most perfected specialty. Those people who are much better at math than you or me aren’t necessarily also much better than you and me at composing a fugue, or saying something interesting about Schopenhauer’s philosophy, or writing ad copy… whereas LLMs are broadly capable.

  At some point you hit the limits of not enough space to think, or not enough cognitive capacity to think with. In the same way as humans can learn to correct our mistakes, but we can’t do RSI (yet!!), because we aren’t modifying the structures we correct our mistakes with. We improve the contents of our brains, but not the brains themselves.

  This feels like a prediction rather than an observation. For myself, I’m not actually sure if the existing weights in existing LLMs are anywhere near being saturated with “the mental content that that number of weights could hypothetically hold”. Specifically, I think that grokking is observed for very very very simple functions like addition, but I don’t think any of the LLM personas have “grokked themselves” yet? Maybe that’s possible? Maybe it isn’t? I dunno.

  I do get a general sense that Kolmogorov Complexity (ie finding the actual perfect Turing Machine form of a given generator whose outputs are predictions) is the likely bound, and Turing Machines have insane depth. Maybe you’re much smarter about algorithmic compression than me and have a strong basis for being confident about what can’t happen? But I am not confident about the future.

  What I am confident about is that the type signature of “agent uses its outputs in a way that relies on the quality of those outputs to somehow make the outputs higher quality on the next iteration” is already occurring for all the major systems. This is (I think) just already true, and I feel it has the character of an “observation of the past” than a “prediction of the future”.