As a sci-fi plot, we could imagine an AI that models people sufficiently well that it can manipulate them by replacing words in a text by their synonyms. Predict the probability of outcome X if the person reads the text with Synonym1, predict the probability of X if the person reads the text with Synonym2, choose the one with higher probability; now do this for all words in the text that have synonyms in a dictionary.
(I do not expect this to work in reality. First, there is too much noise in the environment to predict such microscopic changes. Second, even if you magically could, the effect of the entire text is unlikely to be a linear combination of effects of individual words.)
I assume (but don’t know for sure) that what’s happening in the videos isn’t as they appear (e.g. forging handwriting isn’t that hard), but it’s at least an interesting fictional example of a somewhat-additive attack like this.
As a sci-fi plot, we could imagine an AI that models people sufficiently well that it can manipulate them by replacing words in a text by their synonyms. Predict the probability of outcome X if the person reads the text with Synonym1, predict the probability of X if the person reads the text with Synonym2, choose the one with higher probability; now do this for all words in the text that have synonyms in a dictionary.
(I do not expect this to work in reality. First, there is too much noise in the environment to predict such microscopic changes. Second, even if you magically could, the effect of the entire text is unlikely to be a linear combination of effects of individual words.)
This reminds me of these two Derren Brown videos: https://www.youtube.com/watch?v=43Mw-f6vIbo https://www.youtube.com/watch?v=sEmCQzueyEQ
I assume (but don’t know for sure) that what’s happening in the videos isn’t as they appear (e.g. forging handwriting isn’t that hard), but it’s at least an interesting fictional example of a somewhat-additive attack like this.