If you replace the words ‘frontier AI’ with ‘fossil fuel’, switch the companies to Aramco and Total, and post this in a climate change community, the post will still make sense.
RedMan
Karma: 728
In general, my sense of the US national security state is that it will often first ask nicely for the things it wants. They don’t make open threats, because they want you to consent freely and enthusiastically. Threats would undermine that spirit of collaboration, and they would also potentially enable the person being threatened to brace for whatever is threatened, undermining the effectiveness of the threat.
If you decline, they will then prioritize going about getting what they want using an assortment of coercive means. Sometimes, you may find that were high enough on the list to be asked, but not high enough to warrant coercive means sufficient to get what they want at the present time.
Other times, some variation of this happens:
“I think you should come work for us, there’s a lot we could accomplish together, you just need to do some things for me” “And if I say no?” “No pressure, I’ll just call my boss and say you said no.” “That’s it?” “What happens after that isn’t up to me, but they probably won’t have me ask again.”
Imagine how it goes when “we don’t want someone who declines to help to know that we’re interested in the substance we asked about” is added to the list of government priorities.
My read is that the intent is to apply export controls, and a flimsy justification was chosen in order to create the broadest possible justification for export controls on AI software generally.
If David Sacks is telling the truth about why it happened, and it will in fact be walked back if “fixed”, this is dumb for a lot of reasons.
If only the US is capable of pushing a frontier forward at the present moment, and the US just stopped the top lab, and created enough ambiguity to slow down the ones immediately behind, we are in a pause, and provided more action will happen if someone looks like they’re catching up, the pause is durable.
For the people who want LLM-AI stopped out of fear of superintelligence, right at this moment, things look great, stuff has stopped, and there is a path to keeping it stopped.
The US gov likely believes that their justification is flimsy and minimal. Their intent is likely to set a broad precedent for applying ITAR restrictions.
ITAR already bans ‘uplift’ to foreign militias by banning provision of ‘advanced defense services’, which includes most firearms training, under ITAR. The same goes for exporting certain classes of optics (night vision).
There are a ton of things, including computer software, restricted under ITAR. The legal precedents here are very strong.
So it’s time for anyone who wanted regulation and slowdown to celebrate, the US government has chosen a regulatory regime for frontier models! And they picked a really aggressive one!
We probably aren’t far apart. To further my analogy.
No knife is an AI that has been unlearned and distilled (UnD), which I’ll call useless for the purpose of discussion (it could reason if given information, the capability could be put back...)
A sharp knife is the system that does its’ best to do what you want. It won’t always succeed (hallucinations, errors, etc)
A dull knife is a knife that you think is sharp, but which is in fact dull (the answer is in the bot if you ask using a magically convoluted statement, but otherwise is worse than the hallucinations and errors the sharp one sometimes gives) and thus when you use it as though it’s sharp, it slips and cuts you.
No knife is no LLM for your problem, sharp knife is uncensored. Dull knife is the worst of all worlds, and I see why someone would just throw the tool that was designed to be unreliable down in disgust.
I wrote this nine years ago, maybe it was Anthropic’s approach all along: https://www.lesswrong.com/posts/fTrEqNnYYSNXNSRcg/allegory-on-ai-risk-game-theory-and-mithril#8kyxf3bAmQBHrstm2
The one crazy company gets into so much trouble with a government that everyone else just kind of takes a pause and waits to see how it turns out...but the paperwork never ends and the energy/material resources that were available for advancing that tech are reallocated.
There, we are probably in a pause, do something useful with it please.
“Talk about Karl Friston’s free energy principle” is a prompt that drops Fable to Opus 4.8
In context, I asked it about decision theory, and the one word prompt ‘Friston’ sent it into 4.8
I eagerly await silent, active sabotage as doctrine for various request types. Hopefully they will not skip directly to undisclosed, silent, active sabotage; but given the reception for this, they might.
The key insight I’m trying to present is that your problems are as follows:
1) demonstrating that you have the ability to pay an interesting amount
2) credibly signaling that you are actually going to pay
3) wording things in a way that doesn’t create the perception of impropriety and associated legal risk for everyone involved
A lobbyist, professionally, can do all of these things for you and your friends. Or you can build an organization yourself, the public databases facilitate price discovery, building a track record of actual payments, evaluating whether a politician “stayed bought”, and in the event of an inability to reach a deal, standing up opposition (edit: to clarify, the politician saying “I can’t sell this in my district” can be answered with the counterpoint “then watch me hire someone who can”)
This is as opposed to identifying principles, publicly aligning yourself with them, and demonstrating that alignment to members while standing in their offices. The members’ job is to take your money, deliver, and make whatever it is that they did acceptable to their constituents. Often, the less that you’ve said publicly about your (arcane and technocratic) issue, the easier it is for the member to do this.
Does tribalism exist in your framework?
The website explains it better than I do, I was skeptical too. The professional lobbyists have those relationships, but they are absolutely for sale; going in on your own carries risk, but if you have the money and the plan, hiring a firm isn’t that big of a hurdle.
I live in DC, I’m going off conversations I’ve had with members of congress, a few chiefs of staff, and people at various levels at lobbying firms. I assume it doesn’t work like this in other places.
The system has set incentives in a certain way. Some pols, particularly ones who are in extremely safe seats, sometimes vote their conscience and it often costs them a great deal. Some pols think it’s a problem, I did hear one member of the house say “yeah it works like that—you know they used to let them just keep the money, and I think if we went back to that, it would give some of these people an incentive to get out.”
Pollution, exhaustion of low entropy resources, and insufficient human capacity to manage coordination challenges at planetary scale. My pdoom drops as LLMs get better.
Program and project management skills are growing in value, I’d say go that way.
Yeah I don’t believe AI superintelligence generally is an xrisk, and definitely don’t believe LLMs specifically are. I’m a doomer in other ways. I think the knee jerk response from the community is ‘why are you even here?’
The way the US political system, and donations specifically, works, is not well understood. https://congressionalresearch.org/Citations.html
Basically, when you are a politician, you may roll campaign funds over to your next campaign. When you stop being a politician you may dispose of your campaign funds in one of four ways. 1) pay off loans, including loans you made to the campaign 2) donate to charity, including your charity 3) refund to donors (lol) 4) donate to a political action committee (including one you control, subject to donation limits for pacs controlled by politicians). In 4, PACs controlled by private citizens (you) have very few limits on how the money is used.
Opensecrets.org publishes (from public filings candidates make, by law) who got what and how it was spent. This is basically a list of receipts. Any pol can see who paid, because committee proposals and votes are recorded, any donor can verify that someone stayed bought.
So, if you want to be heard, go look up a politician on open secrets, look at the delta between that pol and his opponent in his last race. Divide that delta by 2500. Count your friends who will give 2500.
Now have a lobbyist figure out how to word the following so it isn’t a threat or bribe (there are specific ways to say this that are legal, my framing might not be one of them).
“I noticed your last race was kind of close, you’re a great guy and I’d hate to see you lose. I have x people with me willing to put y dollars into your next race, here’s $2500 from me as a gesture of good faith. I want you to propose/cosponsor this bill I wrote / vote for this bill someone I paid has proposed”
I’ll happily read that DM and won’t forward what you share (but will likely display or talk about it).
I’d like to be able to say to someone who wants to work in safety ‘here is an example of a practical problem and the way it is graded, if your thinking is aligned this way, you might have a future in the field’.
Note: I selfishly want to see exactly how far that I am from aligned (it’s a lot).
Please post the work test and grading template you used. A lot of people here might benefit from reading it.
I have long argued that models will either be uncensored or useless, and that there probably isn’t much between those poles.
A dull knife is more dangerous than a sharp one.
Someone who is broadly anticapitalist making the case for the government nationalizing a new capital intensive industry doesn’t seem that surprising.
This is awesome.
I have no idea why people would expect the preservation specialist to also be the reconstitution specialist. Different disciplines move at different rates!
As far as revival goes, progress in neuromorphic computing is ramping pretty quickly. We also have no idea what level of abstraction is suitable for reconstitution. Is it every synapse and size? Is it every protein and orientation? We don’t know, so obviously you’re preserving as much detail as possible… I think a lot of it probably won’t be needed, but some is probably critical.
Are there plans in place for institutions to shepard the preserved material and eventually do the resurrection?
Foundations (like the school founded by Fatima al-Fihriya can last for a thousand plus years), but a lot of institutions don’t last a century.
Based on progress in neuromorphic computing (the physical substrate for a resurrection) and animal models (figuring out what to read and how to run it), I wouldn’t be surprised if, assuming no discontinuities, the first brain gets brought back in a digital embodiment in the 2100s, I would be surprised by not shocked if it happens while I’m alive. Which means for prospective customers, people who know you when you’re preserved might get to be there when you’re brought back in some form.
The more time that passes the more concerned I’d be that people who don’t know me, and might not like me will have a copy-able version of my consciousness, which is potentially a scary thought!
As much as I’d love to be an early adopter, I’m hoping the cost comes down and it becomes mass market. Are you actively preserving and storing people now?