Although I think the bigger problem is, what does that even mean and why do you care? Why would you care if it was 20% Hemingway / 40% Steinbeck, rather than vice-versa, or equal, if you do not care about whether it is actually by Hemingway?
In John’s post, I took it as being an interesting and relatively human-interpretable way to characterize unknown authors/users. You could perhaps use it analogously to eigenfaces.
There is hardly anyone who matters who doesn’t have at least thousands of words accessible somewhere.
I see a few different threat models here that seem useful to disentangle:
For an adversary with the resources of, say, an intelligence agency, I could imagine them training or fine-tuning on all the text from everyone’s emails and social media posts, and then yeah, we’re all very deanonymizable (although I’d expect that level of adversary to be using specialized tools rather than a bog-standard LLM).
For an adversary with the resources of a local police agency, I could imagine them acquiring and feeding in emails & posts from someone in particular if that person has already been promoted to their attention, and thereby deanonymizing them.
For an adversary with the resources of a local police agency, I’d expect most of us to be non-identifiable if we haven’t been promoted to particular attention.
And for an adversary with the resources of a typical company or independent researcher, I’d expect must of us to be non-identifiable even if we have been promoted to particular attention.
It’s not something I’ve tried to analyze or research in depth, that’s just my current impressions. Quite open to being shown I’m wrong about one or more of those threat models.
In John’s post, I took it as being an interesting and relatively human-interpretable way to characterize unknown authors/users. You could perhaps use it analogously to eigenfaces.
I see a few different threat models here that seem useful to disentangle:
For an adversary with the resources of, say, an intelligence agency, I could imagine them training or fine-tuning on all the text from everyone’s emails and social media posts, and then yeah, we’re all very deanonymizable (although I’d expect that level of adversary to be using specialized tools rather than a bog-standard LLM).
For an adversary with the resources of a local police agency, I could imagine them acquiring and feeding in emails & posts from someone in particular if that person has already been promoted to their attention, and thereby deanonymizing them.
For an adversary with the resources of a local police agency, I’d expect most of us to be non-identifiable if we haven’t been promoted to particular attention.
And for an adversary with the resources of a typical company or independent researcher, I’d expect must of us to be non-identifiable even if we have been promoted to particular attention.
It’s not something I’ve tried to analyze or research in depth, that’s just my current impressions. Quite open to being shown I’m wrong about one or more of those threat models.