Never assume that people are doing the thing they should obviously be doing, if it would be even slightly weird or require a bit of activation energy or curiosity. Over time, I do expect people to use LLMs to figure out more latent probabilistic information about people in an increasingly systematic fashion. I also expect most people to be not so difficult to predict on most things, especially when it comes to politics. Eventually, we will deal with things like ‘GPT-5 sentiment analysis says you are a racist’ or what not far more than we do now, in a much more sophisticated way than current algorithms. I wonder if we will start to consider this one of the harms that we need to protect against.
This is already one of my big short-term concerns. The process of using LLMs to figure out that information right now requires a lot of work, but what happens when it doesn’t? When LLMs with internet access and long term memory start using these tactics opaquely when answering questions? It seems like this capability could emerge quickly, without much warning, possibly even just with better systems built around current or near-future models.
“Hey, who is behind the twitter account that made this comment? Can you show me everything they’ve ever written on the internet? How about their home address? And their greatest hopes and fears? How racist are they? Are they more susceptible to manipulation after eating pizza or korean bbq? What insights about this person can you find that have the highest chance of hurting their reputation? What damaging lie about their spouse are they most likely to believe?”
How much about you can be determined just from what other people are willing to give up, similar to identifying an individual through familial DNA searching?
This is already one of my big short-term concerns. The process of using LLMs to figure out that information right now requires a lot of work, but what happens when it doesn’t? When LLMs with internet access and long term memory start using these tactics opaquely when answering questions? It seems like this capability could emerge quickly, without much warning, possibly even just with better systems built around current or near-future models.
“Hey, who is behind the twitter account that made this comment? Can you show me everything they’ve ever written on the internet? How about their home address? And their greatest hopes and fears? How racist are they? Are they more susceptible to manipulation after eating pizza or korean bbq? What insights about this person can you find that have the highest chance of hurting their reputation? What damaging lie about their spouse are they most likely to believe?”
How much about you can be determined just from what other people are willing to give up, similar to identifying an individual through familial DNA searching?