Test Bio Please Ignore
Charlie Sanders
Karma: 128
You Should Go Vote for the MAGA-Rebranded Name for AI
Many fields of science were infeasible to develop until the corresponding engineering work had already been performed. The canonical example is the steam engine, which predates thermodynamics by over a century. Those early, commercially implemented steam engines allowed the scientists of the day to benchmark and measure the properties of actual examples of their field of study. There would be no sense in castigating the scientists of the day for not having worked enough on theory prior to the engineering work. The downside of this reality, the inefficiency of early steam engines, was simply a consequence of how reality constructed itself.
Similarly, real-world examples of misalignment such as the HuggingFace incident could end up being a necessary input into the science of AI alignment. If so, then it doesn’t make sense to decry the work that has thus far went into capabilities—the window of opportunity where the engineering has advanced sufficiently to allow for scientific progress may only just now be opening.
Here’s a second-order effect worth considering: Germline modification for intelligence is statutorily prohibited in most countries, the relevant professional societies are currently calling for a 10 year moratorium on the topic, and the public polling is uniformly and staunchly opposed. What if all that is for a good reason? Might some bright young person wanting to work on AI Safety end up having a counterfactually reduced impact if they choose to pursue such a devastatingly disfavored research path?
The reason why China would go for Plan A is mostly that the status quo is terrible for them, and as the AIs get more capable and the US stays ahead, we expect them to wake up to this reality.
Seb’s assertion is that your consideration of the geopolitics of negotiations with China is naïve, and this response doesn’t seem to refute that. A negotiated settlement is only one of many options in China’s toolbox. Some potential considerations:
What about the optics of this deal being reminiscent of the Century of Humiliation? Won’t that make this politically infeasible for Xi?
What about if China instead invests in espionage and exfiltration of model weights? Are any labs currently hardened against state-level cyberattacks, and if not does your plan explain how they would become so?
What if China instead puts their full backing behind companies like Moonshot and relies on catching up to the frontier? K3 seems to make this a promising approach from a Chinese perspective.
China and Russia have failed to come to an agreement on the Power of Siberia 2 pipeline for years despite it clearly being in both of their best interests. What does this say about the prospects of reaching a deal on AI, even if it is in the best interests of everyone involved?
Trump’s president through 2028. How well do you trust his negotiating skills? How have his recent negotiations with China gone?
China is unique among nuclear superpowers in refusing to use a military deescalation hotline, going so far as to weaponize the hotline’s very existence. What does that say about their expected future behavior on x-risk mitigation?
Getting China onboard for this proposal would be somewhere on the order of the greatest geopolitical coup in the history of the world. Treating it like an afterthought that will just naturally work itself out “because AGI/ASI” feels like skipping the hard part of the conversation. I would have wanted to see drastically less sci-fi speculation and drastically more engagement with on-the-ground facts of current geopolitical reality, since that’s what will actually make or break this plan.
Surely there are third parties with authority over the labs who would not permit this scenario to occur? Mechanical Interpretability averting a critical failure is obviously going to bring down the hammer of every regulatory agency in a 10,000 mile radius.
As an example, Mythos is currently being de facto barred from deployment by the US federal government after it demonstrated a hypothetical ability to cause minor amounts of harm. It strains credulity to argue that, after narrowly averting a world-ending catastrophe and with direct evidence of the existence of that risk, the AI labs will simply be permitted to return to business as usual. We have direct experience to say that that’s not how society works.
Charlotte, USA—ACX Spring Schelling 2026
We Need to Be Able to Talk About AI Use
Agreed. There’s really a continuum here, where the rate of self-improvement can range from sublinear to fully exponential. Processes that lack a mechanism for feeding improvements back into the system’s underlying intelligence are unlikely to achieve exponential growth; they’ll tend to plateau or progress linearly at best.
One takeaway is that an effective regulatory regime aimed at limiting fast-takeoff risk might focus precisely on this distinction — restricting processes that iteratively improve at greater than a linear rate.
Self-play is the foundational element of recursive self-improvement — it’s what allowed AI systems to bootstrap to superhuman performance in Chess and Go through effectively unbounded iteration. That said, there’s likely a meaningful spectrum within self-play that depends on the independence and algorithmic sophistication of the evaluating role.
Not all self-play is created equal in this regard. Purely rivalrous, symmetric self-play (like white versus black in Go) represents one end of that spectrum, while asymmetric arrangements with structurally distinct roles (like author and critic) may occupy a qualitatively different position — particularly when the critic brings independent evaluative criteria rather than simply optimizing against the same objective from the opposing side. The degree to which the feedback signal is decoupled from the generative process likely matters a great deal for how “recursive” the improvement truly feels.
In case you’re curious, here’s the current output:
February 15th Story
The chart said Adaeze Okonkwo, fifteen months, and Dr. Rao already knew what was coming before Patty leaned into the doorframe of her office.
“Mrs. Okonkwo is in Room 2. She’s here for the MMR.”
Patty said it the way she’d been saying things lately, with the emphasis shifted just enough to signal trouble. She held a printout against her chest, the state medical board letterhead visible at the top.
“I know what it says,” Dr. Rao told her.
“I’m just making sure you saw the updated memo. ‘Administering non-recommended vaccines without documented clinical justification may constitute grounds for license review.’” Patty read it like a weather report. “‘Physicians are advised to exercise caution consistent with the revised federal schedule.’”
Dr. Rao took the paper. She’d read the original version three weeks ago when the CDC’s advisory committee had moved MMR from “routine” to “optional, parental discretion,” a category that hadn’t existed six months prior. The language was new. The meaning was not.
“She’s my last appointment,” Dr. Rao said. “I’ll handle it.”
Patty nodded and pulled the door shut behind her. Through the wall, Dr. Rao could hear the muffled syllables of a toddler’s voice, the particular pitch of a child discovering the crinkle of exam table paper.
She set the memo on her desk, squared it against the corner, and stood.
Grace Okonkwo sat in the molded plastic chair with her daughter on her lap. She wore a navy peacoat and had her purse on the floor between her feet, both hands occupied with keeping Adaeze from lunging toward the cartoon frog poster. The child had Grace’s wide forehead and her father’s dimpled chin, and she was reaching for the frog with the absolute confidence of someone who had never been told no about anything that mattered.
“Dr. Rao.” Grace’s voice was steady, rehearsed. “I’m here for her MMR. I called ahead. Your office confirmed you still carry it.”
“We do.” Dr. Rao sat on the rolling stool and opened the chart on her tablet. She could feel the shape of the conversation before it started, the way you feel weather change in your knees. “Mrs. Okonkwo, I want to be straightforward with you. The federal schedule was updated last month. MMR is no longer on the recommended list for this age group. It’s been moved to an optional category.”
“I know what they did.”
“If I administer it without a documented clinical justification, my license could be reviewed. That’s new as of this month.”
Grace shifted Adaeze to her other hip. The child grabbed a fistful of her mother’s braids and pulled. Grace didn’t flinch.
“Emeka was three,” she said. “My son. He got measles at his daycare in Silver Spring. A place with a waiting list. A place I thought was safe.” She paused, not for effect but because the next part still cost her something. “He had a hundred-and-five fever for four days. The infection spread to his ears. He lost thirty percent of the hearing in his left ear. He wears a hearing aid now. He’s seven.”
Dr. Rao set the tablet down.
“I am not here to argue about policy,” Grace continued. “I am not confused. I am not hesitant. I want my daughter vaccinated. I’ll pay out of pocket. I’ll sign whatever you need me to sign.”
“It’s not about the money, Mrs. Okonkwo.”
“I know it’s not about the money.” Grace looked at her directly. “It’s about whether you’ll do it.”
The room was quiet except for Adaeze, who had found the crinkle paper again and was tearing a strip of it with great satisfaction. The fluorescent light buzzed at a frequency Dr. Rao had stopped noticing fifteen years ago and was noticing now.
The government shipment of vaccines hadn’t come. It was 1987, and measles was moving through the slum east of her father’s clinic faster than anyone had predicted. Her father drove to a veterinary supply depot in Secunderabad and bought what he could, then spent three days sterilizing, reconstituting, dosing by hand with a chart he’d drawn himself on the back of an invoice. No authorization. No documentation anyone would accept. Fourteen children. No deaths.
He never talked about it. Her mother described it only once, late at night after his funeral.
Dr. Rao looked at Adaeze, who had stopped tearing the paper and was now trying to fit her entire fist into her mouth. She had four teeth, the front ones, and she was grinning around her knuckles.
The supply cabinet was three steps away. The MMR vials were on the second shelf, behind the varicella, exactly where they’d been for twenty-two years.
“Lay her down for me,” Dr. Rao said.
Grace exhaled and eased Adaeze onto the table. The child kicked once, twice, then went still as Grace placed a hand on her belly. Dr. Rao opened the cabinet, pulled the vial, checked the lot number and expiration. She drew the dose into a syringe, flicked it, pressed the plunger until a single bead appeared at the needle’s tip.
She swabbed the child’s left thigh. Adaeze looked at the cartoon frog.
Dr. Rao gave the shot, withdrew the needle, and pressed a cotton ball to the puncture. Adaeze’s face crumpled for two seconds, then reset. Grace scooped her up.
Dr. Rao dropped the syringe into the sharps container, heard it click against the plastic, and wrote the lot number in the chart.
It’s Not That Hard to Build Recursively Self-Improving AI
This story was inspired by considering what different possibilities the use of AI models might unlock. One potential area is ludicrously deep technical expertise, of the kind that eclipses the potential of even the most in-depth research. I think of these stories as having “hyperdetails”, details so specific that they transcend the fourth wall with the reader.
Pygmalion’s Wafer
I was struck by the disparity in inflation results shown by https://www.visualcapitalist.com/cp/g20-inflation-tracker-august-2025/, with China being a clear outlier in global inflation performance. However, I struggled to take that data and visualize why deflation is generally regarded as economically destructive, or how. I think fiction is an underappreciated way to convey complicated topics, so here’s a story that tries to demonstrate the underlying mechanisms of deflationary spirals, also known as “Involution”.
LLM Disclaimer: LLM assistance was used in drafting this story.
Communism By Another Name
Newcomber
In Darkness They Assembled
Of Loving Grace
Thank you! I think there’s a lot of value to be explored in increasing peoples’ awareness of AI progress via fiction.
It’s worth mentioning that the principal reason given for why we cannot coordinate on AI is because the American AI industry currently understands itself to be representing a version of the future where transformative AI is developed in a free and democratic society instead of a repressive authoritarian dictatorship. Everything is downstream of that conclusion.
Incidentally, the Trump-Xi summit is in two weeks And AI is expected to feature heavily in discussions.