unruly abstractions

Per­sona Cor­rup­tion and Role Mis­cast­ing in Emer­gent Misalignment

• unruly abstractions 25 Mar 2026 22:20 UTC

  1 point

  0

  in reply to: Richard_Kennaway’s comment on: Com­par­ing Across Pos­si­ble Worlds

  “gay men” was just my example to illustrate (Recall I said “imagine that...”).
  
  There will always be things in “reality” that are not in the training data.
  Of course, more and more people are getting represented in the corpus of data.
  But, as theorists like Spivak point out, there will always be people left out or misrepresented.
  And as the latest research shows, even with perfect fidelity in the training data, GenAI suffers from mode collapse, leading to undesired homogenization.

  I like to foreground the impact of this all has in the people in the margins.
  But this is a more general problem, one that has been identified as the main issue affecting the reliability of AI.

  
  The point I am trying to make is that AI deals with these implicit counterfactuals in one way or another.
  As you pointed out, we do not want our AI to hallucinate, but we do want it to extrapolate and adapt outside its training if possible.
  Resolving this tension is not trivial.
  

• unruly abstractions 25 Mar 2026 3:48 UTC

  1 point

  0

  in reply to: Richard_Kennaway’s comment on: Com­par­ing Across Pos­si­ble Worlds

  Ooh, thanks, they were vestigial.
  
  I was going to reference The Queer Algorithm to show how counterfactuals, as a concept, appear elsewhere.
  
  Chevillon discusses how AI implicitly needs to form counterfactuals when dealing with missing data. For instance, imagine that the training corpus has no representation of gay men. When generating outputs, the AI behaves as if the training data did contain those representations. Chevillon further argues that the way AI currently performs these implicit counterfactuals relies on interpolating dominant patterns in the data, which is insufficient for minoritized communities.

Com­par­ing Across Pos­si­ble Worlds

Pok­ing and Edit­ing the Circuits

How Far Can Ob­ser­va­tion Take Us?

In­tro: Non-Iden­ti­fi­a­bil­ity of Ex­pla­na­tions

• unruly abstractions 2 Sep 2025 18:05 UTC

  1 point

  0

  in reply to: Trevor Hill-Hand’s comment on: Cat­e­gory-The­o­retic Wan­der­ings into Interpretability

  Good feedback!

  I am still trying to figure out my workflow. I like writing on Typst, but I realized it’s not very easy to go from Typst → Less Wrong. Also, a lot of my writing is sorta experimental. I’m trying to determine which parts of my writing should be directed to which platforms/​audiences.

  I will make this a linkpost

  And yes, Eugenia Chang is amazing :)

Cat­e­gory-The­o­retic Wan­der­ings into Interpretability