TFD

• TFDApr 18, 2025, 6:17 PM

  3 points

  0

  in reply to: Chris_Leong’s comment on: Chris_Leong’s Shortform

  I just think it’s really bad if people feel that they can’t speak relatively freely with the forecasting organisations because they’ll misuse the information.

  To “misuse” to me implies taking a bad action. Can you explain what misuse occurred here? If we assume that people at OpenAI now feel less able to speak freely after things that ex-OpenAI employees have said/​done would you likewise characterize those people as having “misused” information or experience they gained at OpenAI? I understand you don’t have fully formed solutions and that’s completely understandable, but I think my questions go to a much more fundamentally issue about what the underlying problem actually is. I agree it is worth discussing, but I think it would clarify the discussion to understand what the intent of such a norm would (and if achieving that intent would in fact be desirable).

  (This is distinct from my separate point about it being a mistake to hire folk who do things like this. It is a mistake to have hired folks who act strongly against your interests even if they don’t break any ethical injuctions)

  If Coca-Cola hires someone who later leaves and goes to work for Pepsi because Pepsi offered them higher compensation, I’m not sure it would make sense for Coca-Cola to conclude that they should make big changes to their hiring process, other than perhaps increasing their own compensation if they determine that is a systematic issue. Coca-Cola probably needs to accept that “its not personal” is sometimes going to be the natural of the situation. Obviously details matter, so maybe this case is different, but I think working in an environment where you need to cooperate with other people/​institutions means you also have to sometimes accept that people you work with will make decisions based on their own judgements and interests, and therefore may do things you don’t necessarily agree with.

• TFDApr 18, 2025, 5:54 PM

  2 points

  0

  in reply to: ryan_greenblatt’s comment on: ryan_green­blatt’s Shortform

  (You could say “disempowerment which is gradual” for clarity.)

  I feel like there is a risk of this leading to a never-ending sequence of meta-communication concerns. For instance, what if a reader interprets “gradual” to mean taking more than 10 years, but the writer thought 5 would be sufficient for “gradual” (and see timelines discussions around stuff like continuity for how this keeps going). Or if the reader assumes “disempowerment” means complete disempowerment, but writer only meant some unspecified “significant amount” of disempowerment. Its definitely worthwhile to try to be clear initially, but I think we also have to accept that clarification may need to happen “on the backend” sometimes. This seems like a case where one could simply clarify they have a different understanding compared to the paper. In fact, Its not all that clear to me that people won’t implicitly translate “disempowerment which is gradual” to “gradual disempowerment”. It could be that the paper stands in just as much for the concept as for the literal words in people’s minds.

• TFDApr 18, 2025, 5:28 PM

  5 points

  6

  in reply to: Chris_Leong’s comment on: Chris_Leong’s Shortform

  But this only works if those less worried about AI risks who join such a collaboration don’t use the knowledge they gain to cash in on the AI boom in an acceleratory way.

  Can you state more specifically what the alleged bad actions are here? Based on some of the discussions under your post about professional norms surrounding information disclosure, I think it is worth distinguishing two cases.

  First, consider a norm that limits the disclosure of some relatively specific and circumscribed pieces of information, such as a doctor not being allowed to reveal personal health information of patients outside of what is needed to provide care.

  Second, a general norm that if you cooperate with someone and they provide you some info, you won’t use that info contrary to their interests. Its not 100% clear to me, but your post sounds a lot like this second one.

  I think the second scenario raises a lot of issues. Its seems challenging to enforce, hard to understand and navigate, costly for people to attempt to conform to, and potentially counterproductive for what seems to be your goal. You are considering a specific case at a specific point in time, but I don’t think that gives the full picture of the impact of such a norm. For example, consider ex-OpenAI employees who left due to concerns about AI safety. Should the expectation be that they only use information and experience they gained at OpenAI in a way that OpenAI would approve of?

  Now, if Epoch and/​or specific individuals made commitments that they violated, that might be more like the first case, but its not clear that is what happened here. If it is, more explanation of how this is the case would be helpful, I think.

• TFDApr 15, 2025, 6:09 PM

  13 points

  2

  on: OpenAI #13: Alt­man at TED and OpenAI Cut­ting Corners on Safety Testing

  11 former OpenAI employees filed an amicus brief in the Musk vs. Altman lawsuit

  If I’m reading the docket correctly, first amendment expert Eugene Volokh has entered an appearance on behalf of the ex-OpenAI amici. I don’t want to read too much into that, but it is interesting to me in light of the information about OpenAI employees and NDAs that a first amendment expert is working with them.

• TFDApr 15, 2025, 5:46 PM

  1 point

  on: AISN #51: AI Frontiers

  A group of former OpenAI employees filed a proposed amicus brief in support of Musk’s lawsuit on the future of OpenAI’s for-profit transition. Meanwhile, OpenAI countersued Elon Musk.

  I think this is the first time that the charter has been significantly highlighted in this case. My own personal view is the charter is one of the worst documents for OpenAI (therefore good for Musk), and having their own employees stating that it was emphasized a lot and treated as binding are very bad facts for OpenAI and associated defendants. The timeline for all this stuff isn’t 100% clear to me, so I can imagine there being issues with whether the charter was timed such that it is relevant for Musk’s own reliance, but the vibes of this for OpenAI are horrendous. Also raises an interesting possibility of whether the “merge-and-assist” part of the charter might be enforceable.

  The docket seems to indicate that Eugene Volokh is representing the ex-OpenAI amici (in addition to Lawrence Lessig). To my understanding Volokh is a first amendment expert and has also done work on transparency in courts. The motion for leave to file also indicate that OpenAI isn’t necessarily on board with the brief being filed. I wonder if they are possibly going to argue their ex-employees shouldn’t be allowed to do what their doing (perhaps trying to enforce NDAs?), and Volokh is perhaps planning to weigh in on that issue?

• TFDApr 14, 2025, 7:59 PM

  1 point

  1

  on: The­o­ries of Im­pact for Causal­ity in AI Safety

  Strongly agree that causality is a useful and under-estimated perspective in AI discussions. I’ve found the literature on invariant causal prediction, helpful for understanding distributional generalization, especially some of the results in this paper. One of the papers you reference above puts the core issue well:

  generalization guarantees must build on knowledge or assumptions on the “relatedness” of different training and testing domains

  Based on the way I sometimes see the generalization of LLM or other models being discussed, its not clear to me that this idea has really been internalized by the more ML/​engineering focused segments of the AI community.

  As for lesswrong specifically, I think the prevalence of beliefs about LDT/​FDT/​newcomb-like problems impacts the way causality is sometimes discussed in an interesting way. I think both there would be great benefit in both wider AI circles and lesswrong in particular seeing increased discussion of causality.

The limits of black-box eval­u­a­tions: two hypotheticals

• TFDApr 11, 2025, 8:10 PM

  1 point

  0

  on: AI #111: Giv­ing Us Pause

  Not only did it have nothing to do with aircraft, obviously, my reluctance to endorse it and sign the petition was that it read too much like a Bill of Attainder against OpenAI in particular.

  I believe in the circuit court opinion about the federal tiktok bill they determined it was not a bill of attainder. It seems like similar reasoning would apply here. This seems like it might actually be somewhat relevant in future, if events play out where politicians/​the public are much more concerned then various legislatures might consider bills that specific mention leading labs by name.

A differ­ent take on the Musk v OpenAI pre­limi­nary in­junc­tion order

• TFDMar 6, 2025, 6:22 PM

  1 point

  0

  on: AI #106: Not so Fast

  A judge has ruled that on the merits Musk is probably correct that the conversion is not okay

  The question that stopped Musk from getting one was whether Musk has standing to sue based on his donations. The judge thinks that is a toss-up. But the judge went out of their way to point out that if Musk does have standing, he’s a very strong favorite to win, implicitly 75%+ and maybe 90%.

  Can you explain what parts of the order lead to these conclusions? for several of the counts The Court does find standing issues, but the count relevant to this (breach of charitable trust) is addressed in section III.c. of the order. I’m not a lawyer and definitely might have missed or misunderstood something, but reading that section, it isn’t clear to me that the main issue is about standing. My read is that The Court thinks there is a factual question about whether a trust/​contract exists at all, with that question going to the merits of the count, not to standing.

  I think footnote 11 is consistent with this reading. This footnote occurs in section III.c. in the following context (document link):

  Because the threshold question of whether a charitable trust was created remains a toss-up,
  Musk has not demonstrated likelihood of success on the merits sufficient to obtain an injunction. The request for an injunction barring any steps towards OpenAI’s conversion to a for-profit entity
  is DENIED.11

  And the text of the footnote:

  Defendants also challenge plaintiffs’ standing. As Musk has not been directly affiliated
  with OpenAI for several years, any standing must come from an interest in OpenAI’s assets.
  California Corporations Code Section 5142(a). The Court is aware of the distinction between
  Restatement (Second) of Trusts § 391 and Restatement (Third) of Trusts § 94 and cmt. g, plus the
  California state authorities following the Restatement (Third). Thus, for purposes of this motion,
  the Court finds plaintiffs’ standing sufficient as a settlor given the modern trend in that direction.
  The motion to dismiss on this issue is DENIED. Further briefing on this topic is not necessary.

  Again, I’m not a lawyer, but I don’t see how this footnote can be consistent with standing being the main issue.

  A lot of the comments about whether the conversion would be in the public interest occur in the context of evaluating the preliminary injunction factors, of which the public interest is one factor. The order first address likelihood of success on the merits, finding this to be a toss-up. The Court then says about the other factors (emphasis in original):

  Because of this, the remaining Winter factors are derivative of, and dependent on, the first.
  If a trust was indeed created, then preventing or remedying breach would be in the public interest.

  This finding about the public interest is expressly conditional (“derivative of, and dependent on”) the assumption that a trust was created, but that is “a toss-up”. Likewise, the public interest is in “preventing or remedying breach”.

  I think it is helpful to distinguish two arguments against the conversion:

  1. Musk/​plaintiffs advance a theory that such a conversion would breach a trust/​agreement that OpenAI/​defendants made when Musk donated to OpenAI.

  2. Separately, people have raised issues with whether the conversion would be consistent with the OpenAI non-profit’s charitable purpose (which seems based on various documents to heavily overlap with what is in the “public interest”).

  Only the first of those issues is before The Court that is writing this order. I interpret some of the commentary around this order to be suggestion that The Court is commenting (perhaps implicitly) on the second issue. Its not clear to me if that is the case. I think its entirely possible that The Court is only commenting on the first issue (because it is the one relevant to the order) and isn’t expressing any opinion on the second. It seems to me the court is saying something like “assuming OpenAI/​defendants did make a commitment not to convert OpenAI into a for-profit, it can’t possibly be against the public interest or contrary to the balance of the equities to require them to do so for the relatively short period from now until trial”. But in The Courts view the other preliminary injunction factors essentially collapse into the likelihood of success on the merits factor due to the conditionality of that determination, and thus since The Court doesn’t quite think Musk/​plaintiff’s evidence is quite strong enough to say success on the merits is “likely”, The Court denies the preliminary injunction. I think this is much more narrow then some of your comments and some that you quote imply. I do think some parts of the order I could see an argument that if you “read between the lines” the judge might be putting in some things that kind of cast shade at OpenAI, but I think they are pretty far from definitive.

  and Judge Rogers pointed this out several times to make sure that message got through

  Can you elaborate on what parts of the order you had in mind for this?

  *edited to fix link

• TFDOct 19, 2024, 3:25 AM

  3 points

  2

  on: Sab­o­tage Eval­u­a­tions for Fron­tier Models

  One question about the threat model presented here. If we consider a given sabotage evaluation, does the threat model include the possibility of that sabotage evaluation itself being subject to sabotage (or sandbagging, “deceptive alignment” etc.)? “Underperforming on dangerous-capability evaluations” would arguably include this, but the paper introduces the term “sabotage evaluations”. So depending on whether the authors consider sabotage evaluations a subset vs a distinct set from dangerous-capabilities evaluations I could see this going either way based on the text of the paper.

  To put my cards on the table here, I’m very skeptical of the ability of any evaluation that works strictly on inputs and outputs (not looking “inside” the model) to address the threat model where those same evals are subject to sabotage without some relevant assumptions. In my subjective opinion I believe the conclusion that current models aren’t capable of sophisticated sabotage is correct, but to arrive there I am implicitly relying on the assumption that current models aren’t powerful enough for that level of behavior. Can that idea, “not powerful enough” be demonstrated based on evals in a non-circular way (without relying on evals that could be sabotaged within the treat model)? Its not clear to me whether that is possible.

  *edited to fix spelling errors