And, the degree to which people wanted things was even more incoherent than I thought. I thought people wanted things but didn’t know how to pursue them.
How does one figure out goals and what to want? Usually I find myself following the gradient of my impulses. When I can find coherent goals, executing is relatively straightforward. Finding goals in the first place is IMO much harder. If you map it onto the question of how to find meaning in life, it’s more colloquially recognizable as a hard thing!
There has been at least one documented case of attempted social engineering with an AI collecting a dossier and using it to exert reputational pressure, and it seems to have happened accidentally rather than deliberately: An AI Agent Published a Hit Piece on Me and The Operator Came Forward