Interesting.
One complication here—the working language of the Vatican is ordinarily Italian and this document was probably drafted first in Italian (or possibly in a mish-mash of languages with intermediate translation steps). The clearest tells are probably in the Italian version. I don’t know how much LLM-based translation ends up rendering things in LLM voice, but it seems possible that part of the explanation comes from translation rather than drafting steps.
This seems like a reasonable forecast, and if you could buy Anthropic stock, I’d buy a lot of it.
But, I think there are some additional reasons for skepticism:
1) At some point, open weights models (or perhaps cheaper closed ones) will eat a lot of Anthropic’s business for use cases that don’t require being at the frontier. We’re only just reaching the point where this matters—Anthropic models started having real economic value only very recently and so being on the frontier has meant a lot. But, once the open weights models catch up to what, say, Opus 4.6 could do, then how much does that peel off? This is a worse problem for Anthropic if the frontier is kinda lumpy (and it might well be) where instead of delivering continuous economic value improvements you have to cross particular thresholds to open the next set of tasks ahead while open weights eat you from behind.
2) Is their current lead likely to be durable? Best argument in favor is that Claude itself is a big speedup but the counterargument is that this is a relatively young, relatively small company without any obvious moat and with financial resources that are, for the moment, way behind those of, say, Google. So it can’t really be that hard for Google to do what has to be done to eat their lunch. Likewise, there’s an argument that moving to the next level financially requires a kind of expertise in working with big corporate clients that they lack as compared to Google/Microsoft.
3) Subjectively, it seems like Claude excels at coding but ChatGPT is often better at factual stuff. It may well turn out that OpenAI has the edge when it comes to non-coding knowledge work.
4) Similar to your #2 above, the AI boom so far has generated massive market cap gains, which have mostly been captured by the semiconductor stack. So, to suppose that Anthropic will surpass Nvidia, Broadcom, and TSMC, you have to suppose that for some reason the current trends for value accruing to semiconductors vs. model builders will flip. Is something different now than over the last few years to explain why that would happen?
You could try some sort of soft extortion of the potential targets of the attacks (i.e., demanding big money to reveal the vulnerabilities) in a way that’s legal and fairly lucrative.
Is there any reason to believe Trump has ever chatted with an LLM? He interacts with plenty of AI-generated content via other people, but given what we know about his use of technology, I’d be pretty surprised if he’s ever actually interacted with an LLM directly.
Fair enough. Probably no way to come to terms then.
I’m not sure exactly how I’d define a coup but I’d say it has to be clear cut enough that, “Was it really an attempt at a coup?” is not really in contention in the aftermath.
Just out of curiosity, why is that? And what counts as “clear cut”?
After any failed coup (or, for that matter, any failed crime) you’ll always be able to find some apologists who say that the actions/intentions of the accused plotters are being misunderstood and it wasn’t really an attempted coup (or, as they case may be, an attempted murder, robbery, kidnapping, whatever) and/or had some kind of valid legal justification.
On #3, the net effect of the Iran War (at least in conjunction with other recent events) is likely to increase rather than decrease proliferation in general and maybe even with respect to Iran specifically.
From the perspective of an American adversary:
Iran/Venezuela vs. DPRK shows just how vulnerable you are to an American attack if you don’t have nuclear weapons.
Yet, Iran has fought to (as of now) at least a draw and arguably a long-run strategic win against an extremely punishing American air assault while also demonstrating (as everyone has suspected since Iraq/Afghanistan) an American unwillingness to put boots on the ground.
No one has seriously alleged that Iran was engaged in any nuclear work since last summer, and no one has alleged (even unseriously) that Venezuela had nuclear ambitions. So, you’re not safe even if you don’t build nuclear weapons.
So, for most adversaries, you look at that and see a pretty clear upside to going nuclear (you’re safe) vs. a limited downside (an American air attack that might happen even if you’re not building nuclear weapons). Why not build?
From the perspective of a country that’s somewhere between neutral and allied with the US, you look out at the world and see:
The Iran War reflects (along with Ukraine) a new local maximum of predatory great power behavior.
Iran seems to have split the fault line in the US relationship with NATO into a full blow schism, so the situation looks much more tenuous for anyone relying on American protection.
And the US complete aversion to doing anything that actually endangers the lives of non-trivial numbers of US troops make the US look like a complete paper tiger even if you think they might want to protect you.
The US positioning on the Strait of Hormuz has suggested a complete indifference to the fate of Asian allies.
Many international actors are more concerned with what the US has done than what Iran was doing.
Once again, all signs point towards building nuclear weapons. I think we’re rapidly approaching the point where a technologically sophisticated American ally that could build nuclear weapons very fast (Taiwan, Japan) would be a fool not to.
Not OP, but potentially interested in betting on this with you. Before haggling over precise details, do you agree that the events following the 2020 election count as trying to execute a coup?
There are very believable paths where AI reduces your income, but the current big AI companies don’t capture that value, or don’t return it to shareholders.
I suppose it depends on exactly what you do, but if the threat to you is something along the lines of “AI broadly replaces white collar workers” then it’s very hard to come up with a scenario where AI can replace most white collar workers but can’t find a way to make money for the companies building it.
You have to tell a really contorted story for that work—I guess it’s “believable” to imagine something like: AI gets good enough to replace most white collar workers, but there’s such intense commoditization and competition that there is no pricing power so the companies make no money, and the one white collar thing that AI can’t do is design semiconductors, so all the economic gains flow to Nvidia. Could happen but you have to rig the assumptions to get that result. And aside from that, most of the forms I can come up with are of the “money won’t matter anymore” variety in which case, there is no financial hedge.
Also quite believable that it’ll IPO with all that growth already priced in.
The point I’m trying to make is that Anthropic shares are worth more to your average upper middle class white collar worker than they are to the current ultra-wealthy or institutional shareholders. So, let’s assume Anthropic is currently valued in a way that perfectly captures it’s future growth prospects, it should still IPO at a higher valuation (and raise tons of money in the process) because retail investors will rationally put a higher value on the stock than institutions given that it is useful to retail but not institutions as a hedge.
I’m sure institutional shareholders, to the extent they can, are doing some amount of arbitrage in expectation of that dynamic. Everyone knows retail is desperate to buy AI. So, maybe, a fair secondary market valuation of Anthropic already accounts for that premium multiplied by some assessment of the probability they’ll actually list.
But, either way, the point is that the average company is worth about the same to retail vs. institutional investors, and AI companies have very different valuations for those two groups.
It’s widely rumored that both OpenAI and Anthropic will go public within the next year or so, triggering massive capital inflows. I think this unleashes an underrated financial dynamics.
An optimal portfolio hedges your personal risks. Unless you’re independently wealthy, the risk of job loss due to AI automation is one your largest personal financial risks, and it makes a lot of sense to buy insurance against that (i.e., to buy into AI companies). Right now, individual investors can’t do that. And the investors that have access to OpenAI/Anthropic on the private market are precisely the ones that don’t need to hedge that risk. Once you open up to retail, a rational investor should park a sizable percentage of their net worth in those shares, opening up a capital gusher.
This is especially true for Anthropic—there’s basically no way for a retail investor to get meaningful Anthropic exposure if they want it. It’s a little different for OpenAI given that SoftBank is kinda an OpenAI proxy at this point (albeit in ways that make it a less attractive investment than the real thing).
Seems like this could lead to substantial acceleration of progress. It may also change the politics. If stopping AI progress mostly means financially soaking billionaires, then it’s politically a lot more palatable than if it will also screw up the 401ks of the mass affluent.
Rule of law is mostly response to incomplete trust in government and a little bit because predictability is desirable. Because we can’t trust that the government will always act properly, we want to constrain its actions with rules and limit abusable discretion. And because we’re concerned that exceptions to rules will privilege favored groups, we want the rules to be the same for everyone. Same goes for due process.
But that’s not really a first-best solution. Adherence to fixed rules inevitably misfires in some concrete cases where we all agree that it would be better to make exceptions. So, if you have a benevolent and super-intelligent overlord, why constrain its actions based on rules? Why not let it just always do what is best? A super-intelligence can also solve the predictability problem by just directly answering your question about whether you can do X without getting into trouble (whereas that’s administratively infeasible for existing governments).
As to protected rights, I guess it depends on whether you think these are goods in themselves or means to an end. A lot of existing rights probably don’t survive on the basis of being inherently good. For example, most Americans seem to agree that things like hate speech or misinformation are not inherently good but ought to be protected as free speech in order to avoid a slippery slope. But, with a benevolent super-intelligence, we needn’t worry about that because it can just draw the line (and make the needed exceptions).
Personally, I have a pretty fatalistic attitude about this. If we all die, we all die. There’s nothing I can do about it. Unless you happen to be a billionaire or a person of extraordinary influence, there’s nothing you can do about it either. People I’ve known with untreatable terminal diagnoses have roughly that attitude—there’s nothing you can do about it, and you just make your peace and use whatever time is left. In a lot of ways, it’s crueler to have a treatable disease that is probably terminal but where you can try to fight. It’s better to detach.
I do feel an anxiety about the tension between the existential scenario and the scenario where AI doesn’t kill us all but takes over all the white collar jobs. There are things I could to prepare for that, but I don’t really want to do them. They’re in tension with the things I want to do if we’re all going to die. and also the ambiguity of timelines makes it impossible to know what strategies make sense (e.g., if I quit my white collar job to go to nursing school, how much time does that actually buy me? Does nursing get automated by robots quickly enough that I end up behind economically versus just milking what’s left of my career?) So, that sucks.
Another thing I find helpful is remembering that many other people have coped with alternative versions of this and managed to live their lives. At points in the Cold War, the threat of annihilation via nuclear war was probably at least as emotionally pressing for your parents/grandparents as whatever you’re feeling now. And there are plenty of examples of more discrete groups of people staring down certain or near-certain death with their faculties intact.
Could be true, but the API price doesn’t vary by time of day, so there’s no particular incentive to do that for power users and so I’d be kinda surprised if they did anything active to smooth their usage.
Bicameral systems can respect one person one vote, and they even do in the 49 states with bicameral legislatures.
The US Senate is a weird kludge that was necessary to secure support for the constitution. No one would arrive at it from first principles.
The best argument you can make for the Senate is that it’s necessary to protect vulnerable minorities (residents of small states). And democracies tend to trade off in various ways to protect vulnerable minorities at the expense of democratic purity (e.g., via constitutional provisions). That’s a pretty silly argument so far as it goes but does have a little more historical punch. You can’t make it for a hypothetical system with a chamber to protect the interests of the rich who are very obviously not a vulnerable minority at all. And there’s a much higher level of normative repugnance to “we’ve got to look out for the rich” than for most other groups you can fill in the blank with there.
Anthropic recently announced higher Claude usage limits for off-peak hours, which is everything except weekdays from 8 AM − 2 PM ET.
I was surprised by the specific times here. This is night time in Asia, the tail end of the workday in Europe, and the start of the work day in North America. The beginning of the window is too early for much West Coast activity, and it winds down at 11 AM in California. That seems inconsistent with the idea that coding is driving power use?
The timing seems most consistent with the period when you’d see overlapping demand from the US East Coast and from Western Europe—neither of which are super tech heavy regions. Maybe this timing implies a lot of the power users are financial firms working on New York/London time?
Sure, as I said it’s approximate and there are exceptions.
But entry into the peerage also wasn’t closed off, and sufficient wealth was a fairly common pathway. Get wealthy enough, make a few connections or well-placed donations, and you’d wind up in the House of Lords (e.g., the Rothschilds or William Lever—the “lever” in Unilever).
Historically, the British House of Lords at least approximated representation based on wealth (albeit somewhat indirectly) and various other arrangements existed to guarantee the political power of the aristocracy who, before industrialization, were the wealthy.
This is not a very popular thing to do because it’s anathema to the normative basis of democracy (one person, one vote). If one believes that the wealthy deserve increased de jure political representation, then why have a democracy at all and not some sort of oligarchy? And, of course, democratic systems de facto do give the wealthy extra power via various channels.
My understanding is that giving “substantive medical advice” is “unauthorized practice” and as such is banned.
It depends on what exactly you mean by “substantive medical advice” but you have to read these statutes against the backdrop of robust free speech protections when applied to giving advice.
This is a somewhat unsettled legal area and the Supreme Court is currently considering a case (Chiles v. Salazar) that may reshape the doctrine, but if you’re talking about professional practice that purely involves speech then it gets a lot of protection. That quickly recedes once any kind of conduct is involved (i.e., any kind of physical interaction), so the government usually wins unauthorized practice cases by pointing to something other than talking that was done.
Presumptively, if you’re only talking the government can’t regulate the content of that speech without clearing an exceptionally high legal bar and so someone giving you purely verbal medical advice without any physical component is likely in the clear. That’s why it’s not unauthorized practice of medicine if you tell me you have a headache and I suggest you take an aspirin, and all of the various websites out there that let you submit a list of symptoms and then suggest a diagnosis are fine too.
To crack down on something that doesn’t involve fraud/impersonation, the government would have to rely on the so-called “professional speech” exception to the first amendment, used in some circuit courts. It’s not clear this is even good law; the Supreme Court’s majority has never endorsed it, and they cast significant doubt on in in NIFLA v. Becerra (2018).
To the extent the professional speech exception exists at all, it applies when there is a “personal nexus” between the professional and the client. The threshold inquiry is whether or not “a speaker … purport[s] to be exercising judgment on behalf of any particular individual with whose circumstances he is directly acquainted.” So, to the Tweet above, a chatbot is definitely allowed to answer generic questions related to these professions as much as it likes. A chatbot answering “what are the symptoms of a heart attack” is no more acting as a professional than WebMD is by publishing an article on that.
Even if the chatbot is giving particularized advice, it probably isn’t practicing a profession because the courts still are looking for a narrow construction that avoids the first amendment issue; scholars have suggested that the right way of thinking about is whether the advisor has a fiduciary duy. Rosemond v. Markham is a good model for thinking about that in the chatbot context, holding that even if you’re giving personalized advice in response to someone’s specific question about themself, that does not necessarily establish such a relationship. (Rosemond was a columnist who gave psychological advice to people who wrote in to him, describing himself as a “family psychologist.” Someone in Kentucky complained that, because Rosemond was not a licensed professional in Kentucky, this amounted to unauthorized practice. Applying the professional speech doctrine pre-NIFLA, the court held that just responding to questions without establishing some further relationship doesn’t qualify as professional speech and thus Rosemond gets first amendment protection).
There’s a fairly large (1,800 forecaster) market on AGI timelines at Metaculus (albeit with a contestable definition).
The market has had a fairly stable forecast of mid-to-late 2033 over the last six months or so. In the last couple of days, the median has shifted from May 2033 (Tuesday) to October 2032 this morning.
I wonder what that’s about. The date of the move doesn’t really align with any new releases or any breakthroughs that I’m aware of (unless it was partially an anticipatory move on GPT-5.4 rumors). Maybe it’s just noise?
Just to be clear, this is from an Twitter account for someone named Peter Girnus. He is not the CEO of Palantir.
I don’t totally follow the shtick of the account, but he frequently sends out tweets of the form “I am [important person]” that read about like this and are probably some kind of parody.
The “permanent underclass” idea from the tweet seems to be mostly a concern of American white collar workers—and so I imagine the claim here is more or less that this lets relatively wealthy American white collar workers who are vulnerable to being automated away buy some insurance against that. Post-IPO, a white collar worker with a couple of tens of thousands of dollars available can probably buy that insurance via leveraged positions in the stocks, thus escaping the “permanent underclass.”