Ian Rios
Independent AI Safety Researcher
Website: https://unrulyabstractions.com
unruly abstractions
Karma: 44
Ooh, thanks, they were vestigial.
I was going to reference The Queer Algorithm to show how counterfactuals, as a concept, appear elsewhere.
Chevillon discusses how AI implicitly needs to form counterfactuals when dealing with missing data. For instance, imagine that the training corpus has no representation of gay men. When generating outputs, the AI behaves as if the training data did contain those representations. Chevillon further argues that the way AI currently performs these implicit counterfactuals relies on interpolating dominant patterns in the data, which is insufficient for minoritized communities.
Good feedback!
I am still trying to figure out my workflow. I like writing on Typst, but I realized it’s not very easy to go from Typst → Less Wrong. Also, a lot of my writing is sorta experimental. I’m trying to determine which parts of my writing should be directed to which platforms/audiences.
I will make this a linkpost
And yes, Eugenia Chang is amazing :)
“gay men” was just my example to illustrate (Recall I said “imagine that...”).
There will always be things in “reality” that are not in the training data.
Of course, more and more people are getting represented in the corpus of data.
But, as theorists like Spivak point out, there will always be people left out or misrepresented.
And as the latest research shows, even with perfect fidelity in the training data, GenAI suffers from mode collapse, leading to undesired homogenization.
I like to foreground the impact of this all has in the people in the margins.
But this is a more general problem, one that has been identified as the main issue affecting the reliability of AI.
The point I am trying to make is that AI deals with these implicit counterfactuals in one way or another.
As you pointed out, we do not want our AI to hallucinate, but we do want it to extrapolate and adapt outside its training if possible.
Resolving this tension is not trivial.