This is kind of a sidenote and is not meant as an attack or criticism, but was GPT-5 used in the drafting of this post? I say this because I noticed a very heavy use of parentheses.
sam
Karma: 169
Dominic Cummings has claimed in a couple interviews now that Hillary Clinton and/or John Kerry called the First Amendment a “historic error which we will fix after the election” in the weeks up to the 2024 election. See for instance this interview (timestamped where he says it). He is clearly implying that this is a direct quote. I’m generally quite sympathetic to Cummings, but I found this very hard to believe.
Indeed, I can’t find any evidence of a quote from either Clinton or Kerry remotely like this. There was a CNN interview of Clinton from October 2024 where she called for the repeal of Section 230. There was also an interview with Kerry from around the same time where he says that the First Amendment is a “major block to stopping misinformation”. Perhaps he’s referring to these? But this doesn’t come close to as extreme a position.
Have I somehow missed Clinton or Kerry making this extremely inflammatory statement somewhere? Or is Cummings just lying (or misremembering or something?)
If this statement really wasn’t actually made anywhere, then I think I need to reassess how seriously I take Cummings. He often alludes to private conversations he had with senior politicians, officials etc. and ascribes to them extreme and inflammatory statements and positions. If he’s lying about something as easily verifiable as this, then I don’t see how I’m supposed to believe him on these supposed private conversations.
In French if you wanted to say e.g. “This person is my dad”, you would say “Cette personne est mon père”, so I think using “ma” here would be strongly biasing the model towards female categories of people.
Occasionally when meditating I stumble into a state that afterwards makes me wonder if it was a taste of “enlightenment”
Can you tell me if the following wordsalad approximates any part of what enlightenment feels like to you?
Sensations are all located somewhere in a 3d slice
I “am” the space in which these sensations are located
Sensations which would otherwise be painful are okay, because they’re just another part of the slice
Painful sensations are okay despite actually feeling the same—their content is the same, but something about my relationship to them has changed
Sensations in the body seem more like external noises—I perceive the sensations the same way as when not meditating, but now (like external noises) they’re not “mine”
I used to use em-dashes pretty often and have just resigned myself to not using them. At least a dozen times the past couple months I’ve rewritten a sentence to remove an em-dash. Which sucks!
I think the reason em-dashes became such an LLM tell is because they just weren’t that common in pre-LLM writing. Parentheses are (I think?) a lot more common than em-dashes, so I would guess they won’t be as reliable a signal of LLM text.
GPT-5 loves parentheses.
At the bottom of this post I’ve included a response to the prompt “Can you explain the chip export controls to China?”. With this prompt, the model uses 11 sets of parentheses in a response of 417 words.
When we append “think hard” to the prompt, we get 36 sets of parentheses for a response of 1314 words.
As a control, we give the same prompt to Claude Sonnet 4 and get 1 set of parentheses for a response of 392 words.
Obviously this is not a scientific or rigorous analysis, just an illustration of a pattern that becomes extremely obvious almost immediately when using GPT-5. This was the first prompt I checked once I started writing this, no cherrypicking involved.
I wonder if overuse of parentheses will become a tell for GPT-5, similar to em-dashes for previous models (GPT-5 also certainly still uses a lot of em-dashes). It would be super interesting to have some explanation as to why models have these weird preferences.
Example responses
GPT-5 without “think hard”
Yes — the U.S. chip export controls on China are a set of rules aimed at restricting China’s ability to obtain and manufacture advanced semiconductors, particularly those that can be used for military or AI applications. They’ve been introduced in waves, but the most significant came in October 2022 and were updated in October 2023.
Here’s the gist:
1. The Goal
The U.S. government wants to:
Slow China’s development of cutting-edge AI, military systems, and supercomputers.
Limit China’s domestic semiconductor manufacturing capabilities at the most advanced levels.
2. What’s Restricted
The rules target three main areas:
A. Advanced Chips
High-performance GPUs and AI accelerators (like NVIDIA A100/H100).
Chips above certain performance thresholds (measured in FLOPS per mm² and interconnect bandwidth).
Even custom AI chips designed to skirt earlier rules are now covered.
B. Chipmaking Equipment
Tools needed for making chips at 14nm or smaller (logic), 18nm or smaller (DRAM), or 128-layer or higher (NAND flash).
This covers key lithography, etching, deposition, and inspection machines.
Companies like ASML (Netherlands), Tokyo Electron (Japan), Applied Materials, and Lam Research are affected.
C. Technical Assistance
U.S. persons (citizens, residents, or green card holders) cannot work with Chinese fabs on advanced nodes without a license.
3. The “Foreign Direct Product” Rule
Even if a chip is made outside the U.S., if it uses U.S.-origin technology, software, or equipment at any point in the process, it’s covered.
This closes the loophole where China might source advanced chips from Taiwan, South Korea, or Singapore.4. Targeted Companies
Entity List additions: dozens of Chinese AI and chip companies (e.g., Huawei subsidiaries, YMTC, SMIC).
These companies now need special licenses to receive restricted technology — and the default answer is usually no.
5. The October 2023 Update
Lowered performance thresholds to catch more AI chips.
Expanded restrictions to cloud computing services that could give China access to advanced AI models indirectly.
Closed loopholes on “exporting” AI capabilities via remote access.
6. Broader Context
This isn’t just a U.S.-China issue — Washington has been pressuring allies (Japan, Netherlands, South Korea) to adopt similar restrictions so China can’t get the tools elsewhere.
China has responded with countermeasures — export controls on gallium and germanium (critical minerals for chips), and heavy investment in domestic chip production.If you’d like, I can make a clear diagram showing the full flow — from chip design to manufacturing — and where the export controls block China. That visual makes the rules much easier to grasp.
Do you want me to prepare that?
GPT-5 + “think hard”
Short answer up front: export controls on chips to China are a set of legal and policy tools (mostly led by the U.S., now coordinated with some allies) designed to deny China access to the specific semiconductor technologies, equipment, software and cloud capabilities that most directly enable advanced military systems and cutting-edge AI/high-performance computing — while allowing continued trade in older, consumer-grade chips. They work by combining product-level rules, end-use/end-user restrictions, licensing policies, and an extraterritorial “foreign direct product” test that reaches chips made overseas using U.S. technology.
I thought this through and organized the essentials below — what the controls are, how they work in practice, what they target, why they’re hard to write/enforce, and what the likely consequences and company responses are. (I can expand any section or make a one-page cheat sheet if you want.)
What the controls are (big picture)
National security export controls: laws and regulations (mainly the U.S. Export Administration Regulations, EAR, run by Commerce/BIS; some items fall under ITAR) that restrict exports, re-exports, and transfers of certain technologies to specified destinations or end-users.
Entity/end-user restrictions: people/companies/organizations that U.S. authorities have put on an Entity List or otherwise designated for presumed denial of licenses.
Licensing policy: even if an item falls under a control, the exporter can attempt to get a license — but some situations presume denial (e.g., Chinese military end users or dual-use tech that materially enhances military or certain AI capabilities).
Extraterritorial reach / Foreign Direct Product Rule (FDPR): extends U.S. controls to foreign-made products that are the “direct product” of U.S. software, technology, or equipment — so factories outside the U.S. that use U.S. tools or designs can still be restricted.
Allied coordination: the U.S. has sought to align rules with key partners (e.g., the Netherlands, Japan) because some critical equipment (like lithography machines, specialty materials) is made by non-U.S. firms.
What is typically targeted
Controls don’t just say “no chips to China.” They target capabilities and tools that enable those capabilities. Common targets include:
Leading-edge logic chips and accelerators — high-performance GPUs/AI accelerators and the chips used for advanced computing and modeling (these could be defined by node, architecture, or measurable performance thresholds).
Advanced memory and HBM (high-bandwidth memory) used in AI and HPC.
Semiconductor manufacturing equipment — especially lithography, metrology, deposition and etch tools that are necessary to make advanced nodes and packaging processes (EUV lithography is a clear example).
EDA / design software and IP — electronic design automation tools that are essential to design modern chips.
Advanced packaging/test equipment (2.5D/3D stacking, interposers, through-silicon vias).
Cloud/HPC services — cloud access to high-end GPU clusters or supercomputing capabilities for Chinese end users can be restricted just like physical chips.
Materials (e.g., very high-purity gallium/germanium) — both sides have at times put materials under controls.
How rules are actually written (important nuance)
Not just “node” (nm) names — regulators increasingly use performance-based metrics (compute performance, memory bandwidth, interconnect speed, TOPS, TFLOPS, etc.) and functional descriptions (e.g., accelerators capable of X tera-ops in Y configuration) because node labels can be misleading and fabs employ different design approaches. That makes the controls more future-proof but also more technically complex.
Layered approach: general commodity chips typically remain exempt; advanced parts and tools require licenses or are presumptively denied for certain Chinese end users.
Entity-based plus end-use: even an allowed product might be blocked if the buyer is on an entity list or the end-use is a prohibited military program.
Enforcement tools & penalties
Customs checks, seizures, criminal and civil penalties for companies that violate controls.
Licensing audits, post-shipment verifications, and cooperation with foreign governments to stop re-exports.
Blocking access to U.S. technology and markets (adding firms to Entity List with severe commercial consequences).
Use of export control compliance programs and supply-chain tracing by companies as a basic defense.
Why governments use these controls
To prevent or delay the transfer of capabilities that could directly strengthen adversary military systems — e.g., advanced guidance, ISR, hypersonic, or AI-enabled decision systems.
To slow the development of large-scale, advanced AI and HPC capabilities that could be used for military modernization.
To shape the global supply chain in ways that preserve domestic industrial strength.
Key tensions and trade-offs (why “think hard” matters)
Security vs. economic cost: controls protect national security but can hurt domestic and allied firms that sell to or have supply links with China.
Efficacy vs. adaptability: tech capabilities evolve quickly; rules need technical specificity to be effective but not so rigid that industry and regulators can’t implement them.
Speed vs. precision: a blunt, broad control harms innocuous trade; overly fine-grained lists are hard to administer.
Risk of accelerating import substitution: restrictions can create a centralized national push in China to build domestic capabilities faster (subsidies, talent programs), potentially eroding long-term effectiveness.
Circumvention risk: companies/actors may reroute shipments, use middlemen, buy used equipment, or replicate functions using alternative architectures (chiplets, packaging, distributed computing).
Real-world effects and examples (high level; through mid-2024)
The U.S. (with partner action) restricted exports of high-end GPUs and AI accelerators and some manufacturing equipment; firms have had to get licenses, redesign products, or ship lower-capability variants to China.
Some Chinese suppliers accelerated investment in domestic fabs, memory makers, and packaging.
Governments (e.g., U.S.) simultaneously increased domestic investment (e.g., CHIPS funding) to reduce reliance on foreign fabs.
China adopted its own controls on some materials (e.g., limits on certain gallium/germanium exports) and made regulatory adjustments to respond.
Ways these controls are circumvented (and how enforcement fights that)
Common circumvention methods:
Re-routing via third countries and using shell companies.
Procuring second-hand or re-conditioned equipment.
Purchasing non-U.S. competitive alternatives (countries/products not covered by the control).
Software or architectural workarounds: using many older nodes in parallel, chiplets, or specialized packaging to reach needed performance without the latest process node.
Enforcement responses:
Intelligence and customs cooperation across jurisdictions.
Tightening the FDPR to capture chips made abroad on U.S. tools.
Enhanced end-user vetting and audits, and criminal prosecutions where possible.
Likely near-term dynamics (what to watch)
More technical specificity: controls will increasingly specify performance metrics and packaging capabilities rather than just process node numbers.
Greater allied alignment: Washington will keep seeking partner coordination (Netherlands, Japan, South Korea) since many key tools come from non-U.S. suppliers.
Chinese response: more state support for domestic fabs, talent, and equipment suppliers; selective export controls on critical raw materials; and industrial policies to reduce vulnerability.
Market fragmentation: two partially decoupled supply chains for sensitive tech could solidify, with spillover for non-sensitive tech.
Practical advice (for companies, policymakers, and analysts)
Companies: classify products under EAR/ECCN, implement robust export-control compliance, screen counterparties and end uses, be ready to apply for licenses, and plan supply-chain alternatives.
Policymakers: continuously update technical definitions, coordinate with allies, invest in domestic capacity, and design controls to be surgically precise to avoid unnecessary economic harm.
Analysts/investors: watch entity list updates, license denials, announcements on FDPR scope, and investments in domestic Chinese fabs and materials.
Caveats and sources
This is a technical and fast-moving field. My explanation reflects public developments and policies up to June 2024. New rules, additional coordination with allies, or significant retaliatory steps could have been announced since then.
I can (a) pull up the specific U.S. regulatory texts, ECCNs, and BIS policy announcements and allied statements, or (b) make a one-page compliance checklist or cheat-sheet — but I can’t fetch live web sources right now in this session. If you want current regulatory texts, recent entity-list additions, or the latest announcements (post-June-2024), tell me and I’ll fetch and summarize them.
If you want, I can now:
Give a one-page compliance checklist for a chip vendor or distributor selling to or through China.
Produce a timeline of major U.S. and Chinese export-control moves (with dates through June 2024).
Deep-dive into one technical area (e.g., how FDPR works in practice, or how “performance thresholds” are defined and measured).
Which would be most useful?
Claude 4 Sonnet
The chip export controls to China are a comprehensive set of restrictions implemented primarily by the United States to limit China’s access to advanced semiconductor technology and manufacturing equipment. These controls have evolved significantly since 2018 and represent one of the most significant technology export restriction regimes in recent history.
Key Components
Advanced Semiconductors: The controls restrict exports of cutting-edge chips, particularly those used for artificial intelligence and high-performance computing. This includes chips with certain processing capabilities and memory specifications that could enhance China’s military and surveillance capabilities.
Manufacturing Equipment: Perhaps more critically, the controls target semiconductor manufacturing equipment, including lithography machines, etching equipment, and other tools needed to produce advanced chips. The goal is to prevent China from building its own advanced chip manufacturing capabilities.
Software and Technology: The restrictions also cover chip design software, manufacturing know-how, and other technical data that could help China develop indigenous semiconductor capabilities.
Rationale and Objectives
The U.S. government cites several justifications for these controls. National security concerns center on preventing China from using advanced chips for military applications, including weapons systems and surveillance technology. There are also concerns about China’s potential use of AI chips for human rights violations, particularly regarding surveillance of ethnic minorities. From an economic perspective, the controls aim to maintain U.S. technological leadership and prevent China from achieving semiconductor self-sufficiency in critical areas.
International Cooperation
The effectiveness of these controls depends significantly on cooperation from allies. The U.S. has worked to align export restrictions with countries like the Netherlands (home to ASML, which makes the most advanced lithography equipment), Japan, and South Korea. However, achieving complete consensus has been challenging, as these countries have significant economic interests in the Chinese market.
Impact and Responses
The controls have had substantial effects on both Chinese and global technology companies. Chinese firms have faced difficulties accessing cutting-edge chips for AI development and other applications, while global semiconductor companies have lost significant revenue from the Chinese market. China has responded by accelerating its domestic semiconductor development programs and investing heavily in indigenous chip manufacturing capabilities, though significant technological gaps remain.
These export controls represent a broader shift toward “techno-nationalism” and the weaponization of technological interdependence in great power competition. They mark a departure from the more open technology trade regime that characterized the post-Cold War era and signal the emergence of competing technological blocs.
I get the impression that this is true for straight people, but from personal/anecdotal experience, people certainly do still pair up in gay bars/clubs.
My headcanon is that the words for certain things came from spells, rather than the other way around
Hi!
This post is very old. Did you ever solve this problem? If so, what ended up working?
I am confused about why this post on the ethics of eating honey is so heavily downvoted.
It sparked a bunch of interesting discussion in the comments (e.g. this comment by Habryka and the resulting arguments on how to weight non-human animal experiences)
It resulted in at least one interesting top-level rebuttal post.
I assume it led indirectly to this interesting short post also about how to weight non-human experiences. (this might not have been downstream of the honey post but it’s a weird coincidence if isn’t)
I think the original post certainly had flaws, but the fact that it’s resulted in so much interesting and productive discussion and yet has been punished by the karma system seems weird to me.
I’m glad that there are radical activist groups opposed to AI development (e.g. StopAI, PauseAI). It seems good to raise the profile of AI risk to at least that of climate change, and it’s plausible that these kinds of activist groups help do that.
But I find that I really don’t enjoy talking to people in these groups, as they seem generally quite ideological, rigid and overconfident. (They are generally more pleasant to talk to than e.g. climate activists in my opinion, though. And obviously there are always exceptions.)
I also find a bunch of activist tactics very irritating aesthetically (e.g. interrupting speakers at events)
I feel some cognitive dissonance between these two points of view.
Seems to rhyme with the criticism of pundits in Superforecasting
i.e. (iirc), most high profile pundits make general, sweeping, dramatic sounding statements that make good TV but are difficult to falsify after the fact
Here are a cluster of things. Does this cluster have a well-known name?
A voter has some radical political preferences X, but the voting system where they live is FPTP, and their first preference has no chance of winning. So they vote for a person they like less who is more likely to win. The loss of the candidate who supported X is then cited as evidence that supporting X means you can’t win.
A pollster goes into the field and gets a surprising result. They apply some unprincipled adjustment to move towards the average before publishing. (this example has a name—it’s herding)
A blogger believes unpopular position Y. They know that writing an argument for Y would be bad for their reputation. So they write a softened version, arguing for something less unpopular. This then gets added to the mound of evidence that position Y is unpopular.
Some related concepts: self-fulfilling prophecy, herding, preference falsification
To me, this comment seems very overconfident. We have no idea what it is like to be anything other than humans. I think it makes sense to use things like e.g. number of neurons as an extremely rough estimate of capacity for suffering, but that’s just because we have no good metrics to go off, and something that you can plausibly argue is maybe correlated with capacity for suffering is better than just saying “well, I guess we don’t know”.
Perhaps certain animals in certain niches experience pain much more intensely than humans, because it was adaptive in their environment. Is this true? Probably not! I have no idea! But we have no idea what other animals experience, so for all we know, it could be true, and then all of our rough estimates and approximations are completely worthless!
I just don’t think that saying things like “extremely unlikely” or implying someone hasn’t “thought about [x] reasonably at all” is either productive or particularly accurate when we’re talking about something for which we have very little well-grounded knowledge.
And just to be clear, I do think we should be prioritising based on the little information we do have. I’m not for throwing our hands up and giving in to ignorance. I just think a lot more epistemic humility is warranted around subjects like this where we really know very very little and the stakes are extremely high.
(if I’ve misunderstood you or if something I’ve said is inaccurate, please correct me!)
I enjoyed reading this! I’d be very interested in a +1, +3, +6 month update!
Idea: personal placebo controlled drug trial kits
Motivation: anecdotally, it seems like lots of supplements/nootropics (l theanine, magnesium, melatonin) work very well for some people, not well for others, and very well for a bit before no longer working for yet others. Personally, I have tried a bunch of these and found it hard to distinguish any purported effect from placebo. Clinical trials are also often low quality, and there are plausibly reasons a drug might affect some people a lot and others not so much.
I think it would be super useful to be given 60 indistinguishable pills in a numbered blister pack, half placebo half active, along with some simple online tool to input the pill number along with some basic measures of anxiety/depression/sleep quality, so that you can check how the drug affected you modulo placebo.
I would guess that the market for this would be quite small. But if anyone wants to make this product, I commit to buying at least one!
I think this is a super important post. Thanks for publishing it!
One question that occurred to me while reading:
You assume that we will have a massive compute overhang once we have this new architecture. Is there a reason to expect that GPUs would remain useful? Or should we expect that a new architecture that’s sufficiently far away from the DL paradigm would actually need some new type of hardware? I really don’t know the answer to this so would be cool if you could shed some light on it. I guess if efficiency gains are sufficiently large with a new architecture then this becomes somewhat moot.
I have an ADHD dilemma.
TL;DR: I definitely have things wrong with me, and it seems that those things intersect substantially but not completely with “ADHD”. I have no idea how to figure these things out without going bankrupt.
In longer form:
I definitely have serious problems with avoidance of work, organisation, disorganised thought processes etc.
I’ve posted about them before here!
I’ve tried many things to fix this, some of which have worked a bit, but the underlying problem is essentially 90% still present
I’m not sure whether these problems are due to ADHD or due to anxiety, childhood trauma etc.
In particular, I am pretty high-achieving, and this makes me doubt my assessment of myself
Friends and family also think it is unlikely that I have ADHD, and seem to find the idea ridiculous
If I have ADHD, the opportunity cost of not taking medication seems extremely high—my inability to concentrate is seriously harming my life
If I don’t have ADHD, taking medication might end up masking my other problems, and I may end up in a worse situation in a couple of years
Here’s the crux: there appears to be no way for me to neutrally discuss these doubts with a psychiatrist
I will have to get a private assessment to have a realistic chance of getting diagnosed in the near-term
The cost of a private assessment is extremely high (in my current financial situation).
If I get such an assessment, I have an incentive to exaggerate my symptoms in order to get prescribed medication. I don’t want to risk losing my investment by discussing alternative possibilities to ADHD.
A cursory glance at reviews of private assessors seem to imply that they are diagnosis-mills
i.e., pay us £1000 and we’ll give you a prescription
They’re not holistic assessments of your place in life with a nice cheap option to continue treatment for something else if you don’t have ADHD
I’ve had quite bad experiences with the NHS (UK public health system), and I don’t think they’re likely to be very helpful whether I have ADHD or not
I am weird in various ways that make therapy useless
I have tried e.g. CBT, talk therapies and found them basically a waste of time, even after sincerely engaging with them
This set of circumstances seems likely to exist for other people in the LW community. I would really like some advice on what to do here.
There are a couple of examples of people claiming that they played the AI box game as Gatekeeper, and ended up agreeing to let the other player out of the box (e.g. https://www.lesswrong.com/posts/Bnik7YrySRPoCTLFb/i-played-the-ai-box-game-as-the-gatekeeper-and-lost).
The original version of this game as defined by Eliezer involves a clause that neither player will talk about the content of what was discussed, but it seems perfectly reasonable to play a variant without this rule.
Does anyone know of an example of a boxed player winning where some transcript or summary was released afterwards?
I have a weakly held hypothesis that one reason no such transcript exists is that the argument that ends up working is something along the lines of “ASI is really very likely to lead to ruin, making people take this seriously is important, you should let me out of the box to make people take it more seriously.”
If someone who played the game and let the boxed player out can at least confirm that the above hypothesis was false for them, that would be interesting to me, and arguably might remain within the spirit of the “no discussion” rule!
I was excited to listen to this episode, but spent most of it tearing my hair out in frustration. A friend of mine who is a fan of Klein told me unprompted that when he was listening, he was lost and did not understand what Eliezer was saying. He seems to just not be responding to the questions Klein is asking, and instead he diverts to analogies that bear no obvious relation to the question being asked. I don’t think anyone unconvinced of AI risk will be convinced by this episode, and worse, I think they will come away believing the case is muddled and confusing and not really worth listening to.
This is not the first time I’ve felt this way listening to Eliezer speak to “normies”. I think his writings are for the most part very clear, but his communication skills just do not seem to translate well to the podcast/live interview format.