Akash

Karma: 5,773

Akash 11 Sep 2024 3:24 UTC
3 points
1
in reply to: Ben Pace’s comment on: AI forecasting bots incoming
Hm, good question. I think it should be proportional to the amount of time it would take to investigate the concern(s).

For this, I think 1-2 weeks seems reasonable, at least for an initial response.

Akash 10 Sep 2024 23:23 UTC
25 points
22
in reply to: Adam Scholl’s comment on: AI forecasting bots incoming
Hendryks had ample opportunity after initial skepticism to remove it, but chose not to.
IMO, this seems to demand a very immediate/sudden/urgent reaction. If Hendrycks ends up being wrong, I think he should issue some sort of retraction (and I think it would be reasonable to be annoyed if he doesn’t.)
But I don’t think the standard should be “you need to react to criticism within ~24 hours” for this kind of thing. If you write a research paper and people raise important concerns about it, I think you have a duty to investigate them and respond to them, but I don’t think you need to fundamentally change your mind within the first few hours/days.
I think we should afford researchers the time to seriously evaluate claims/criticisms, reflect on them, and issue a polished statement (and potential retraction).
(Caveat that there are some cases where immediate action is needed– like EG if a company releases a product that is imminently dangerous– but I don’t think “making an intellectual claim about LLM capabilities that turns out to be wrong” would meet my bar.)

Akash 9 Sep 2024 4:42 UTC
7 points
−3
in reply to: davekasten’s comment on: Zach Stein-Perlman’s Shortform
I think it’s bad for discourse for us to pretend that discourse doesn’t have impacts on others in a democratic society.
I think I agree with this in principle. Possible that the crux between us is more like “what is the role of LessWrong.”
For instance, if Bob wrote a NYT article titled “Anthropic is not publishing its safety research”, I would be like “meh, this doesn’t seem like a particularly useful or high-priority thing to be bringing to everyone’s attention– there are like at least 10+ topics I would’ve much rather Bob spent his points on.”
But LW generally isn’t a place where you’re going to get EG thousands of readers or have a huge effect on general discourse (with the exception of a few things that go viral or AIS-viral).
So I’m not particularly worried about LW discussions having big second-order effects on democratic society. Whereas LW can be a space for people to have a relatively low bar for raising questions, being curious, trying to understand the world, offering criticism/praise without thinking much about how they want to be spending “points”, etc.

Akash 8 Sep 2024 1:32 UTC
5 points
2
in reply to: Zach Stein-Perlman’s comment on: Zach Stein-Perlman’s Shortform
Is this where we think our pressuring-Anthropic points are best spent ?
I think if someone has a 30-minute meeting with some highly influential and very busy person at Anthropic, it makes sense for them to have thought in advance about the most important things to ask & curate the agenda appropriately.
But I don’t think LW users should be thinking much about “pressuring-Anthropic points”. I see LW primarily as a platform for discourse (as opposed to a direct lobbying channel to labs), and I think it would be bad for the discourse if people felt like they had to censor questions/concerns about labs on LW unless it met some sort of “is this one of the most important things to be pushing for” bar.

Akash 6 Sep 2024 15:12 UTC
9 points
0
on: What is SB 1047 *for*?
It seems to me like the strongest case for SB1047 is that it’s a transparency bill. As Zvi noted, it’s probably good for governments and for the world to be able to example the Safety and Security Protocols of frontier AI companies.
But there are also some pretty important limitations. I think a lot of the bill’s value (assuming it passes) will be determined by how it’s implemented and whether or not there are folks in government who are able to put pressure on labs to be specific/concrete in their SSPs.
More thoughts below:
Transparency as an emergency preparedness technique
I often think in an emergency preparedness frame– if there was a time-sensitive threat, how would governments be able to detect the threat & make sure information about the threat was triaged/handled appropriately? It seems like governments are more likely to notice time-sensitive threats in a world where there’s more transparency, and forcing frontier AI companies to write/publish SSPs seems good from that angle.
In my model, a lot of risk comes from the government taking too long to react– either so long that an existential catastrophe actually occurs or so long that by the time major intervention occurs, ASL-4+ models have been developed with poor security, and now it’s ~impossible to do anything except continue to race (“otherwise the other people with ASL4+ models will cause a catastrophe”.) Efforts to get the government to understand the state of risks and intervene before ASL4+ models seem very important from that perspective. It seems to me like SSPs could accomplish this by (a) giving the government useful information and (b) making it “someone’s job” to evaluate the state of SSPs + frontier AI risks.
Limitation: Companies can write long and nice-sounding documents that avoid specificity and concreteness
The most notable limitation, IMO, is that it’s generally pretty easy for powerful companies to evade being fully transparent. Sometimes, people champion things like RSPs or the Seoul Commitments as these major breakthroughs in transparency. Although I do see these as steps in the right direction, their value should not be overstated. For example, even the “best” RSPs (OpenAI’s and Anthropic’s) are rather vague about how decisions will actually be made. Anthropic’s RSP essentially says “Company leadership will ultimately determine whether something is too risky and whether the safeguards are adequate” (with the exception of some specifics around security). OpenAI’s does a bit better IMO (from a transparency perspective) by spelling out the kinds of capabilities that they would consider risky, but they still provide company leadership ~infinite freedom RE determining whether or not safeguards are adequate.
Incentives for transparency are relatively weak, and the costs of transparency can be high. In Sam Bowman’s recent post, he mentions that detailed commitments (and we can extend this to detailed SSPs) can commit companies to “needlessly costly busy work.” A separate but related frame is that race dynamics mean that companies can’t afford to make detailed commitments. If I’m in charge of an AI company, I’d generally like to have some freedom/flexibility/wiggle room in how I make decisions, interpret evidence, conduct evaluations, decide whether or not to keep scaling, and make judgments around safety and security.
In other words, we should expect that at least some (maybe all) of the frontier AI companies will try to write SSPs that sound really nice but provide minimal concrete details. The incentives to be concrete/specific are not strong, and we already have some evidence from seeing RSPs/PFs (and note again that I think that the other companies were even less detailed and concrete in their documents.)
Potential solutions: Government capacity & whistleblower mechanisms
So what do we do about this? Are there ways to make SSPs actually promote transparency? If the government is able to tell that some companies are being vague/misleading in their SSPs, this could inspire further investigations/inquiries. We’ve already seen several Congresspeople send letters to frontier AI companies requesting more details about security procedures, whistleblower protections, and other safety/security topics.
So I think there are two things that can help: government capacity and whistleblower mechanisms.
Government capacity. The FMD was cut, but perhaps the Board of Frontier Models could provide this oversight. At the very least, the Board could provide an audience for the work of people like @Zach Stein-Perlman and @Zvi– people who might actually read through a complicated 50+ page SSP with corporate niceties but be able to distill what’s really going on, what’s missing, what’s misleading, etc.
Whistleblower mechanisms. SB1047 provides a whistleblower mechanism & whistleblower protections (note: I see these as separate things and I personally think mechanisms are more important). Every frontier AI company has to have a platform through which employees (and contractors, I think?) are able to report if they believe the company is being misleading in its SSPs. This seems like a great accountability tool (though of course it relies on the whistleblower mechanism being implemented properly & relies on some degree of government capacity RE knowing how to interpret whistleblower reports.)
The final thing I’ll note is that I think the idea of full shutdown protocols is quite valuable. From an emergency preparedness standpoint, it seems quite good for governments to be asking “under what circumstances do you think a full shutdown is required” and “how would we actually execute/verify a full shutdown.”

Akash 4 Sep 2024 21:01 UTC
5 points
4
in reply to: ryan_greenblatt’s comment on: The Checklist: What Succeeding at AI Safety Will Involve
@ryan_greenblatt one thing I’m curious about is when/how the government plays a role in your plan.
I think Sam is likely correct in pointing out that the influence exerted by you (as an individual), Sam (as an individual), or even Anthropic (as an institution) likely goes down considerably if/once governments get super involved.
I still agree with your point about how having an exit plan is still valuable (and indeed I do expect governments to be asking technical experts about their opinions RE what to do, though I also expect a bunch of DC people who know comparatively little about frontier AI systems but have long-standing relationships in the national security world will have a lot of influence.)
My guess is that you think heavy government involvement should occur for before/during the creation of ASL-4 systems, since you’re pretty concerned about risks from ASL-4 systems being developed in non-SL5 contexts.
In general, I’d be interested in seeing more about how you (and Buck) are thinking about policy stuff + government involvement. My impression is that you two have spent a lot of time thinking about how AI control fits into a broader strategic context, with that broader strategic context depending a lot on how governments act/react.
And I suspect readers will be better able to evaluate the AI control plan if some of the assumptions/expectations around government involvement are spelled out more clearly. (Put differently, I think it’s pretty hard to evaluate “how excited should I be about the AI control agenda” without understanding who is responsible for doing the AI control stuff, what’s going on with race dynamics, etc.)

Akash 3 Sep 2024 22:51 UTC
26 points
4
on: The Checklist: What Succeeding at AI Safety Will Involve
I liked this post (and think it’s a lot better than official comms from Anthropic.) Some things I appreciate about this post:
Presenting a useful heuristic for RSPs
Relatedly, we should aim to pass what I call the LeCun Test: Imagine another frontier AI developer adopts a copy of our RSP as binding policy and entrusts someone who thinks that AGI safety concerns are mostly bullshit to implement it. If the RSP is well-written, we should still be reassured that the developer will behave safely—or, at least, if they fail, we should be confident that they’ll fail in a very visible and accountable way.
Acknowledging the potential for a pause
For our RSP commitments to function in a worst-case scenario where making TAI systems safe is extremely difficult, we’ll need to be able to pause the development and deployment of new frontier models until we have developed adequate safeguards, with no guarantee that this will be possible on any particular timeline. This could lead us to cancel or dramatically revise major deployments. Doing so will inevitably be costly and could risk our viability in the worst cases, but big-picture strategic preparation could make the difference between a fatal blow to our finances and morale and a recoverable one. More fine-grained tactical preparation will be necessary for us to pull this off as quickly as may be necessary without hitting technical or logistical hiccups.
Sam wants Anthropic to cede decision-making to governments at some point
[At ASL-5] Governments and other important organizations will likely be heavily invested in AI outcomes, largely foreclosing the need for us to make major decisions on our own. By this point, in most possible worlds, the most important decisions that the organization is going to make have already been made. I’m not including any checklist items below, because we hope not to have any.
Miscellenaous things I like
- Generally just providing a detailed overview of “the big picture”– how Sam actually sees Anthropic’s work potentially contributing to good outcomes. And not sugarcoating what’s going on– being very explicit about the fact that these systems are going to become catastrophically dangerous, and EG “If we haven’t succeeded decisively on the big core safety challenges by this point, there’s so much happening so fast and with such high stakes that we are unlikely to be able to recover from major errors now.”
- Striking a tone that feels pretty serious/straightforward/sober. (In contrast, many Anthropic comms have a vibe of “I am a corporation trying to sell you on the fact that I am a Good Guy.”)
Some limitations
- “Nothing here is a firm commitment on behalf of Anthropic.”
- Not much about policy or government involvement, besides a little bit about scary demos. (To be fair, Sam is a technical person.Though I think the “I’m just a technical person, I’m going to leave policy to the policy people” attitude is probably bad, especially for technical people who are thinking/writing about macrostratgy.)
- Not much about race dynamics, how to make sure other labs do this, whether Anthropic would actually do things that are costly or if race dynamics would just push them to cut corners. (Pretty similar to the previous concern but a more specific set of worries.)
- Still not very clear what kinds of evidence would be useful for establishing safety or establishing risk. Similarly, not very clear what kinds of evidence would trigger Sam to think that Anthropic should pause or should EG invest ~all of its capital into getting governments to pause. (To be fair, no one really has great/definitive answers on this. But on the other hand, I think it’s useful for people to start spelling out best-guesses RE what this would involve & just acknowledge that our ideas will hopefully get better over time.)
All in all, I think this is an impressive post and I applaud Sam for writing it.

Akash 3 Sep 2024 20:36 UTC
2 points
0
in reply to: Neel Nanda’s comment on: Akash’s Shortform
Thanks for sharing! Why do you think the CA legislators were more OK pissing off Big Tech & Pelosi? (I mean, I guess Pelosi’s statement didn’t come until relatively late, but I believe there was still time for people in at least one chamber to change their votes.)
To me, the most obvious explanation is probably something like “Newsom cares more about a future in federal government than most CA politicians and therefore relies more heavily on support from Big Tech and approval from national Democratic leaders”– is this what’s driving your model?

Akash 2 Sep 2024 23:39 UTC
11 points
1
on: Akash’s Shortform
Why do people think there’s a ~50% chance that Newsom will veto SB1047?
The base rate for vetoes is about 15%. Perhaps the base rate for controversial bills is higher. But it seems like SB1047 hasn’t been very controversial among CA politicians.
Is the main idea here that Newsom’s incentives are different than those of state politicians because Newsom has national ambitions? So therefore he needs to cater more to the Democratic Party Establishment (which seems to oppose SB1047) or Big Tech? (And then this just balances out against things like “maybe Newsom doesn’t want to seem soft on Big Tech, maybe he feels like he has more to lose by deviating from what the legislature wants, the polls support SB1047, and maybe he actually cares about increasing transparency into frontier AI companies?)
Or are there other factors that are especially influential in peoples’ models here?
(Tagging @ryan_greenblatt, @Eric Neyman, and @Neel Nanda because you three hold the largest No positions. Feel free to ignore if you don’t want to engage.)

Verification methods for international AI agreements

Akash31 Aug 2024 14:58 UTC

14 points

1 comment4 min readLW link

(arxiv.org)

Akash 30 Aug 2024 14:01 UTC
7 points
3
in reply to: M. Y. Zuo’s comment on: Would catching your AIs trying to escape convince AI developers to slow down or undeploy?
Despite the disagree votes, I actually think this is a reasonable suggestion– and indeed more people should be asking “why doesn’t X policy person go around trying to explain superintelligence risks?”
I think the disagree votes are probably coming from the vibe that this would be easy/simple to do. Short (in length) is not the same as easy.
My own answer is some mix of fear that I’m not the right person, some doubts about which points to emphasize, and some degree of “I may start working more deliberately on this soon in conjunction with someone else who will hopefully be able to address some of my blind spots.”

Akash 28 Aug 2024 12:43 UTC
10 points
4
on: SB 1047: Final Takes and Also AB 3211
Supposing SB1047 passes, what are the main ways in which you think it will contribute to (increasing or decreasing) existential risks?
Idea #1: Transparency through SSPs
It seems to be like the main way it could reduce AI risk is by making it more likely that governments and/or members of the public are able to notice (a) signs of dangerous capabilities or (b) problems in comapnies’ safety and security plans.
One counter to this is that companies will be able to write things that sound good but don’t actually give governments or members of the public the ability to meaningfully understand what’s going on. For example, there are ways to write “we have an AI that is able to do novel AI R&D tasks” in ways. that underplay the extent or nature of the risks.
The counter to that seems to me like “well yes, we still have to deal with the fact that companies can try to make vague/unhelp SSPs [CC the discourse about RSPs] and we have to deal with the fact that governments will need to have sufficient risk context to understand the SSPs and understand when they need to intervene. You can’t expect one bill in 2024 to solve all of the major challenges, but it pushes in the right direction and on the margin more transparency seems good.”
Idea #2: Whistleblower mechanisms
The whistleblower protections seem great– and perhaps even better than “whistleblower protections” are “whistleblower mechanisms.” I am worried about the amount of friction RE whistleblowing– who do you go to, what channels do you use, what are you supposed to say, etc etc. The fact that companies would have to provide “a reasonable internal process” and employees are explicitly told they can use this channel to disclose “misleading statements related to its safety and security protocol” or “failure to disclose known risks to employees” seems excellent.
The counter is that a whistleblower could say “I’m worried about X” and the most likely outcome is that the government ignores X because it doesn’t have enough time/attention/technical capacity/risk context to understand why it should care or what it should do. But of course the counter to that counter is once again “yes, this doesn’t fix everything, but surely we are better off in a world where clear whistleblower mechanisms exist, and hopefully there will be other efforts to make sure the government is equipped with the personnel/resources/processes it needs to notice and prepare for AI risks.”
(Meta: After the political battles around SB1047 end, I hope more people end up writing about if/how they think SB1047 will/would affect AI risks. So much of the discourse got [perhaps rightfully] hijacked by political fights about claims that seem pretty ridiculous. It’s plausible that this was the right call– political battles need to be fought sometimes– but on the margin I wish there was more “Here’s the affirmative/positive case for why SB1047 is good”, “Here are some concerns I have about SB1047 from the perspective of reducing existential risk, not from the perspective of responding to industry talking points”, and things like “SB1047 seems to help but only if governments also develop X and Y in the upcoming years, and here are some thoughts on how they could do that.”

Akash 27 Aug 2024 19:19 UTC
10 points
5
on: Soft Nationalization: how the USG will control AI labs
I’m a big fan of work that imagines plausible future scenarios in which governments become more concerned about AI risks. Thank you for this!
I find myself least convinced by the sections about why “hard nationalization” isn’t going to happen. I don’t even necessarily disagree with the conclusion, but I think these sections treat AI like a “normal technology” and many of the reasons listed don’t seem like they would be particularly important if the US was super concerned about AI.
IMO, the main factor determining whether or not hard nationalization is a feasible option is the extent to which the US government is concerned about superintelligence risks. I think the main cruxes will be things like “how concerned is the US about AI risks”, “how concerned is the US about AI misalignment risks”, and “to what extent does the US understand believe that superintelligence is truly a civilization-defining technology”. In the worlds where the US government gets Very Concerned, I suspect the economic/legal challenges would be figured out, and there would either be full nationalization or something that looks pretty indistinguishable from full nationalization for all practical purposes.
Nonetheless, I support this work overall and find myself very supportive of the following statement:
Rather than committing to a specific model of the future, we believe the most effective analysis today will consider a wide range of scenarios that describe actions the US government will take in response to global circumstances. By enumerating many of the plausible scenarios regarding soft nationalization, we believe AI governance researchers can better ground our research in likely futures and design better interventions.

Akash 27 Aug 2024 2:22 UTC
73 points
31
on: Would catching your AIs trying to escape convince AI developers to slow down or undeploy?
The main reason I’d be more freaked out is that I already think egregious misalignment is fairly plausible; if I thought it was very unlikely, I wouldn’t change my mind based on one weird observation.
I agree with this perspective, and I think it has some important implications for AI policy/comms.
My understanding is that there are very few individuals who are actually trying to help policymakers understand misalignment concerns or loss-of-control concerns. There has been a widespread meme that in order to appear respectable and legitimate to policymakers, you need to emphasize CBRN concerns, misuse concerns, and China concerns and you don’t want to come across as “weird” by mentioning misalignment concerns.
(I think these concerns have been overstated– my own experiences suggest that many policymakers appear to be very interested in better understanding loss of control scenarios. And in fact some policymakers see this as the main “value add” from AI people– they tend to rely on existing CBRN experts and existing China experts for the misuse topics).
Regardless, even if we were in a world where the best way to gain status was to avoid talking about misalignment risks, this would come at a major cost: policymakers would not understand misalignment risks, and their actions would reflect this lack of understanding/context/previous conceptual understanding.
I think one of the most important things for AI policy folks to do is to find concise/compelling ways to convey concerns about misalignment.
To be clear, I think this is extremely challenging. One of the reasons why so few people are explaining misalignment concerns to policymakers is that it is very difficult to actually answer questions like “I understand that this is possible, but why is this likely?” or “Why won’t the companies just develop safeguards to prevent this from happening?”, or “As the science gets better, won’t we just figure out how to avoid this?”
But I think a year ago, I expected that a significant fraction of the AI policy community was regularly giving presentations about misalignment. I think I substantially overestimated the amount of effort that was going into this, and I think this is one of the most neglected/important topics in AI safety.
(And while I don’t want to call anyone out in particular, I do feel like some folks in the AI safety community actively discouraged people from trying to have frank/honest/clear conversations with policymakers about misalignment risks. And I think some AI policy groups continue to avoid talking about misalignment risks and systematically empower people who are going to appear more “normal” and systematically avoid hiring people who are inclined to try to explain misalignment/loss-of-control scenarios. It does seem like the culture has been changing a bit in the last several months, but I think this “don’t talk about misalignment concerns with policymakers” gatekeeping is among the worst mistakes made by the AI policy community.)
What kinds of resources would be helpful? Some examples IMO:
- Short presentations that summarize key reasons to be concerned about misalignment risks
- People who can have 1-1s with policymakers and actually feel like they can answer basic questions about misalignment risks
- Pieces like this and this.
- Attempts at taking things like List of Lethalities and Without Specific Countermeasures and explain them in short/concise/clear ways.
- Statements from labs, lab employees, and other respected AI experts that they are concerned about misalignment risks (IMO Bengio is a good example of this; he seems to do this more frequently than ~everyone else and still maintain high quality and high clarity.)
(Also, to be clear, I’m not saying that policy groups should never “meet people where they’re at” or never discuss things like CBRN risks/China/misuse risks. I do think, however, that most LW readers would likely be shocked/disappointed at just how little % of time is spent explaining misalignment risks relative to everything else.)

Akash 23 Aug 2024 0:21 UTC
15 points
19
in reply to: aysja’s comment on: Zach Stein-Perlman’s Shortform
I will also note what I feel is a somewhat concerning trend. It’s happened many times now that I’ve critiqued something about Anthropic (its RSP, advocating to eliminate pre-harm from SB 1047, the silent reneging on the commitment to not push the frontier), and someone has said something to the effect of: “this wouldn’t seem so bad if you knew what was happening behind the scenes.”
I just wanted to +1 that I am also concerned about this trend, and I view it as one of the things that I think has pushed me (as well as many others in the community) to lose a lot of faith in corporate governance (especially of the “look, we can’t make any tangible commitments but you should just trust us to do what’s right” variety) and instead look to governments to get things under control.
I don’t think Anthropic is solely to blame for this trend, of course, but I think Anthropic has performed less well on comms/policy than I [and IMO many others] would’ve predicted if you had asked me [or us] in 2022.

Akash 23 Aug 2024 0:12 UTC
17 points
13
in reply to: aogara’s comment on: Zach Stein-Perlman’s Shortform
It’s hard for me to reconcile “we take catastrophic risks seriously”, “we believe they could occur within 1-3 years”, and “we don’t believe in pre-harm enforcement or empowering an FMD to give the government more capacity to understand what’s going on.”
It’s also notable that their letter does not mention misalignment risks (and instead only points to dangerous cyber or bio capabilities).
That said, I do like this section a lot:
Catastrophic risks are important to address. AI obviously raises a wide range of issues, but in our assessment catastrophic risks are the most serious and the least likely to be addressed well by the market on its own.As noted earlier in this letter, we believe AI systems are going to develop powerful capabilities in domains like cyber and bio which could be misused– potentially in as little as 1-3 years. In theory, these issues relate to national security and might be best handled at the federal level, but in practice we are concerned that Congressional action simply will not occur in the necessary window of time. It is also possible for California to implement its statutes and regulations in a way that benefits from federal expertise in national security matters: for example the NIST AI Safety Institute will likely develop non-binding guidance on national security risks based on its collaboration with AI companies including Anthropic, which California can then utilize in its own regulations.

Akash 21 Aug 2024 23:01 UTC
20 points
8
in reply to: Zac Hatfield-Dodds’s comment on: Zach Stein-Perlman’s Shortform
@Zac Hatfield-Dodds do you have any thoughts on official comms from Anthropic and Anthropic’s policy team?
For example, I’m curious if you have thoughts on this anecdote– Jack Clark was asked an open-ended question by Senator Cory Booker and he told policymakers that his top policy priority was getting the government to deploy AI successfully. There was no mention of AGI, existential risks, misalignment risks, or anything along those lines, even though it would’ve been (IMO) entirely appropriate for him to bring such concerns up in response to such an open-ended question.
I was left thinking that either Jack does not care much about misalignment risks or he was not being particularly honest/transparent with policymakers. Both of these raise some concerns for me.
(Noting that I hold Anthropic’s comms and policy teams to higher standards than individual employees. I don’t have particularly strong takes on what Anthropic employees should be doing in their personal capacity– like in general I’m pretty in favor of transparency, but I get it, it’s hard and there’s a lot that you have to do. Whereas the comms and policy teams are explicitly hired/paid/empowered to do comms and policy, so I feel like it’s fair to have higher expectations of them.)

Akash 17 Aug 2024 18:50 UTC
7 points
1
in reply to: Akash’s comment on: Akash’s Shortform
One plausible explanation is that industry still thinks it’s likely to kill the bill, and they just didn’t feel like they needed to play their cards sooner.
But this still leaves me surprised– I would’ve expected that it’s in industry’s interest to kill the bill earlier in the process because:
1. It might be easier to kill earlier on because it hasn’t gained much traction/support
2. If you want to appear like you’re open to regulation (which seems to be the policy of major AI companies), you probably want to kill it in a relatively silent/invisible way. If you have to be very loud and public and you get to the point where there are a bunch of media articles about it, you lose some credibility/reputation/alliances (and indeed I do think industry has lost some of this “plausibility of good will” as a result of the SB1047 saga)

Akash 17 Aug 2024 18:47 UTC
17 points
4
on: Akash’s Shortform
Why didn’t industry succeed in killing SB1047 [so far]?
If someone had told me in 2022 that there would be a bill in CA that the major labs opposed and that the tech industry spent a fair amount of effort lobbying against (to the point of getting Congresspeople and Nancy Pelosi to chime in), I would’ve been like “that bill seems like it should get killed pretty early on in the process.”
Like, if the bill has to go through 5+ committees, I would’ve predicted that it would die within the first 3 committees.So what’s going on? Some plausible explanations:
- Industry has less power over AI legislation than I (and maybe some others) thought
- Industry has more influence on the federal government than on the CA legislatures
- Industry underestimated SB1047 early on//didn’t pay much attention to it and the opposition came relatively late in the game
- Scott Weiner is really good at building coalitions and forming alliances
- SB1047 is relatively light-touch and the burden is very high when industry tries to fight light-touch things
What do you think are the most noteworthy explanations for why industry has failed to kill SB1047 so far?

Akash 16 Aug 2024 20:28 UTC
16 points
0
on: The Tech Industry is the Biggest Blocker to Meaningful AI Safety Regulations
In 2022 and early 2023, I remember having some conversations with folks about whether or not industry groups would support regulation. (At the time, the thought of governments becoming concerned enough about risks from advanced AI to consider regulation was one of those things that seemed like it wouldn’t happen for a long time, if ever).
Perhaps I and some others were naive. But I genuinely bought the idea that if governments ended up getting more concerned about AI risks, then some of the industry players would end up supporting common-sense regulations. Especially industry players that had acknowledged AI risks (e.g., Anthropic, OpenAI, DeepMind). I think this was also a fairly commonly-held opinion within AI safety circles (and indeed was part of the story for why people concerned about AI safety should support Anthropic, OpenAI, and DeepMind).
In some ways, it’s very sad to see how wrong these takes were. The “maybe industry will support regulations or maybe even proactively push for regulations” takes have not aged well.
In other ways, I think the SB1047 saga has mobilized a lot of AI governance/policy folks and caused them to realize where industry stands. There is a bit of poetic Litany of Gendlin energy here– if industry is not going to support even light-touch regulation, it is worth swallowing this difficult pill and orienting to the landscape accordingly.
(That said, I am interested in seeing how Anthropic reacts to the amendments. I think Anthropic has done a lot less for comms/policy efforts than many folks had predicted it would, and I think many are uncertain if its comms/policy efforts will differ meaningfully from its competitors.)
(I also think that as evidence of risk increases, some industry players might either change their minds or feel compelled to support regulation out of fear of looking super unreasonable.)

Akash

Ver­ifi­ca­tion meth­ods for in­ter­na­tional AI agreements

Verification methods for international AI agreements