Intelligence Dissolves Privacy
The future is going to be different from the present. Let’s think about how.
Specifically, our expectations about what’s reasonable are downstream of our past experiences, and those experiences were downstream of our options (and the options other people in our society had). As those options change, so too our experiences, and so too our expectations of what’s reasonable. I once thought it was reasonable to pick up the phone and call someone when I wanted to talk to them, and to pick up my phone when it rang; things have changed, and someone thinking about what’s possible could have seen the dilution of that signal into noise coming. So let’s try to see more things coming, and maybe that will give us the ability to choose what it will actually look like.
I think lots of people’s intuitions and expectations about “privacy” will be violated, as technology develops, and we should try to figure out a good spot to land. This line of thinking was prompted by one of Anthropic’s ‘red lines’ that they declined to cross, which got the Department of War mad at them; the idea of “no domestic bulk surveillance.” I want to investigate that in a roundabout way, first stepping back and asking what is even possible to expect, here.
“any legal use”
Widespread access to intelligence will change privacy expectations dramatically, by allowing for 1) much more recording of information, 2) much more processing of recorded information, and 3) much more sophisticated interpretation of that information.
In American contexts, law enforcement officers have access to a wide range of information about people, but require permission to look at it (a ‘warrant’). If you’re a person of interest in a crime, they can look at your cell phone records to get evidence about whether or not you were involved in the crime, but otherwise you’re protected by the 4th Amendment from unreasonable searches and seizures. But what determines what is reasonable?
Some considerations:[1]
What protects the privacy of innocent individuals. People with access to LEO systems might have their own personal reasons to look up information–like seeing what their ex-girlfriends are up to–which society doesn’t want them to be able to act on.
What is informative. If you have to review the cell phone records of everyone in the LA area whenever a murder happens in LA, you’re probably going to be wasting many hours of investigative effort, because most of those people are irrelevant to the case.
What is cheap. If you need to put a public servant with health care and a pension on reviewing information, it is harming the public (who is paying for this!) to make them review irrelevant information.
This has already been changed by technology becoming cheaper. It would be prohibitively expensive to have police doing stakeouts on every corner; it is not prohibitively expensive for every shopkeeper to have a CCTV system recording the street outside of their shop, and those costs continue to decline. Put together a network of those, and now a city can be under near-complete surveillance.[2]
AI continues these trends. If LLMs can review cell phone records for pennies on the dollar, it might make sense to look at a hundred times as many records. And now rather than having to have a person go camera-to-camera and track the movements of an individual thru the city, you can have a software system using facial recognition and gait analysis and spatial modeling to track whole crowds at a feasible cost.
So as a technical matter, it is already possible or it’s not very far from interested parties being able to track your location at any time, if you have your phone on you or you’re in a car or you’re inside a city, in a ‘bulk’ rather than targeted way. As a social matter, this might seem pretty impolite—or it might be part of a trade most people are willing to make.
In particular, one other way that technology changes the dynamics is by making it easier for attackers to do significant amounts of damage, which raises the value of surveillance, and of predicting and catching crimes before they happen rather than investigating and punishing them after the fact.
Let’s return to the interpretation of information, and look at some subtler ways that increased intelligence will change the dynamics. Many things can be inferred from ‘public records’ in nonobvious ways. For example, if your camera is fast enough and sensitive enough, you can measure someone’s heartbeat just by watching the subtle blush-and-pale cycle of the blood in their face; standard cameras are good enough for this, and changes in heartbeat are informative about thoughts, along with other subtle changes in facial appearance.
But I’m going to talk about gaydar, because it ties back into the broader social questions of where we want society to end up. Sometimes, people can guess the sexual orientation of another person just by observing them, using both deliberate and accidental features. Gay men sometimes benefit from looking gay (“the earring in his right ear suggests—” or “his haircut implies—”), and also they have various developmental differences that can manifest in appearance. In 2018, Wang and Kosinki trained a neural network to do it off of dating photos and it substantially outperformed humans (80% success rate rather than 60% success rate.)[3][4]
So as we we get more widespread intelligence–as software gains capabilities that were formerly available only to human experts and use of that software becomes potentially widespread–we stop being able to hide some things. What do we want to do about that?
This doesn’t include a line for when an expert observer could have guessed that I’d be gay, which is probably a decade earlier.
The world has changed a lot here, over the course of my lifetime! When I was a child, being gay was mostly hidden and navigating being gay required subtlety and discernment. Part of the response to the AIDS crisis, at the insistence of gay rights groups, was to prioritize patient confidentiality over stopping the spread.[5] But now, as an adult, being gay is mostly not hidden; you don’t have to use a profiling algorithm on my face, up until Facebook removed it in 2022 you could just go to my profile and see that I’ve checked the box for “interested in men”. (Or you could search my LessWrong comments, or–)
So from the perspective of the 2020s in America, it feels actually pretty benign. (But that’s not universal; the situation is both worse in other countries, and the prospect of ‘transvestigating models’ feels much less benign.)
More broadly, it seems to me like there are three options for how we can react to widespread knowledge of things that were previously hidden:
Acceptance. Knowing who’s gay and who’s straight is fine, because both options are fine, and not being confused or ignorant is useful. (I wasted a lot of hope pining after straight guys, and a few straight women wasted hope pining after me, which all could have been avoided.)
Purging. Knowing who’s gay and who’s straight allows you to remove all of the gay boys from your all-boys high school, because you actually wanted an all-straight-boys high school.
Pretend ignorance. Even tho you could know who’s gay and who’s straight, you have a policy like Don’t Ask, Don’t Tell where everyone tries to act like they don’t know it. (After all, only a stalker would know something like that.)
I think there’s situations where each of the three is the most appropriate option. In particular, I think the situation for acceptance of sexual minorities has been on this positive trendline in part because of increased knowledge and decreased privacy. The understanding that lots of gay people were ‘normal’ did a lot of normalize being gay!
I also think it’s easy to find situations where purging or filtering are quite sympathetic. I in fact would strongly support my local subway system tracking who the most anti-social riders are and banning them, so that the system is cleaner and safer, and if it is cheaper and better to do so with facial recognition technology or similar ‘totalitarian surveillance measures’ , that seems probably worthwhile. (Similarly, it’s very nice to not be murdered in a terrorist attack.) But it’s also easy to see how such technologies can be deployed for undesirable ends; if border agents look thru someone’s phone to try and determine if they’re a member of a terrorist network, they can also determine whether they’ve made social media comments that are critical of Trump. A current controversy is how much local cities should sign on to surveillance networks like Flock; when many jurisdictions have committed to not cooperate with federal law enforcement on immigration enforcement, signing on with a contractor which does cooperate with federal law enforcement runs counter to those commitments.
As a fan of the truth and a believer in its efficacy, I am most biased against the third option. Yet many widely and strongly cherished parts of our society rest on it! Anti-discrimination laws that bar decision-making based on race are an example that it seems unwise to recklessly drop, and yet race is often quite easy for people or systems to infer.[6] European regulations about privacy seem to me to mostly be insisting that technology develop in this direction.
Yet my overall sense is that we cannot stick our heads in the sand for long. The highest priority uses will drive adoption of the mass surveillance technology, and I strongly suspect that concerns about terrorism, great power conflict, and small-scale bad actors will be serious enough that these highest priority uses will not be foregone. Then once the camel’s nose is in the tent the rest of the camel will follow. The best way out is to fix our goals and preferences:
Rather than hoping that local entities can prevent the enforcement of immigration rules that are clearly not in their interest, develop new immigration rules that cities across America would be comfortable cooperating with, and then use the advanced technology to do so cheaply.[7]
Develop watchmen that watch the watchmen, such that people with access to bulk surveillance systems and are using them in corrupt ways are themselves found and punished.
Most controversially, become comfortable with benign deviancy (of the sort which will turn out to be much more prevalent than it seems) and align criminal standards with actual behavior (a world where the true speed limit is 20mph higher than the posted one will not be well-served by ubiquitous vehicle tracking).
Perhaps most importantly, it might help you to start behaving as if you’re being watched and things about you are more obvious than they once were.
- ^
I should note that I’m thinking like an economist or systems designer, not a lawyer. There must be extensive case law on what people currently think is reasonable or unreasonable, but that’s only relevant for reasons of continuity. We’re imagining the future, of what things will look like after people have adapted to their new situation, which plausibly involves major changes to the underlying laws.
- ^
Consider also the situation with cashless toll systems like EZ Pass, which have long worried privacy advocates, as they can (and sometimes are!) used to track where people travel. There’s nothing fundamentally rights-violating here, tho; this could be replicated by anyone with enough eyes in enough places. (We already consent to each car having a unique identifier to make tracking easier!)
- ^
“Wait,” you might say, “60% success rate for a trait with a baseline prevalence of less than 40%? How did the human guessers do worse than always guessing ‘straight’?” In their dataset, they equalized the number of homosexual and heterosexual faces, so pure-chance guessing would have scored 50%. This is still an artificial situation (they’re dating site photos, not street photos) which doesn’t take into account base rates.
- ^
A friend points out that Wang & Kosinski generated a bunch of critical responses, like this one which claims that the effect is primarily due to styling choices, rather than underlying biological features. How much of the effect is deliberate vs. accidental determines how much choice one has in whether to hide the feature, but in the current equilibrium people can “hide in plain sight”, where a member of the public might just think “oh, a shark plushie”, whereas a fellow transgirl will see a Blåhaj and view it as an identity marker. With more widespread intelligence, more people can more cheaply become ‘in the know’, and so this channel will become costlier (and, to the extent the predictive features aren’t styling choices, may be impossible to hide).
- ^
As someone interested in public health, this horrifies me, but I acknowledge the ways I am a sweet summer child who grew up with the internet and the dramatic upswing in acceptance and downswing in intolerance, and decision-makers in the 80s had very different life experiences and expectations.
- ^
When I was a data scientist, I ended up looking into the contours of compliance here. Many things that aren’t race are nevertheless informative about race, and so you can construct a composite out of information which individually is legal or ethical to use, which is just a proxy for race, and thus the composite is illegal or unethical to use, and so people developed statistical techniques to try to make sure this isn’t happening, and that the composite is composed just of ‘legitimate’ influence. This involves being deliberately and willfully blind to facts about the world to achieve some social end, but that’s what polite ignorance is!
- ^
There are too many horror stories of ICE misbehavior, detaining American citizens and racially profiling people on the street. Would the situation be improved or made worse by a national facial recognition database? On the one hand, Flock and similar systems would allow ICE to notice whenever someone without legal residency went out in public; on the other hand, there would be no excuse for not immediately checking the database and releasing legal residents. I think immigration reform means we can get the benefit of the latter without having to pay the costs of the former; of course, immigration reform is its own problem that deserves its own post.
See Kelsey Piper’s discovery that Opus 4.7 can reliably identify her as the author of unpublished text.
If this is already the case for more public figures, it could be for everyone else very very soon.
Realizing “it might help you to start behaving as if you’re being watched and things about you are more obvious than they once were” is a pretty major update that will shock the world when it becomes necessary.
Interestingly, in the book The Truth Machine by James Halperin (which I’d recommend and made me much more like this than I’d otherwise be after reading it at age 8), they have a period of amnesty for all past crimes to try to handle the disruption to society of a perfect truth machine.
This is the one of the reasons why I try to add phrases and sentence structures which are far from my usual way of talking in my online comments. Bonus points if I split these text personalities by platform to avoid meta pattern spotting.
It leads to some frankly poor spelling and grammar choices but it adds noise for any patternmatching AI system to filter.
Claude Sonnet 3.something could do it for me, and I am a much much less public figure than Kelsey. (On the other hand, I am also older and so there is more of my writing out there on the interwebs. Also, I only tried it once and didn’t run any sort of carefully controlled experiment on it, so maybe it just got lucky.)
LLMs also provide an obvious way to prevent this kind of de-anonymization: ask them to paraphrase your text in their own style. Then your text will have the markers of “written by Opus 4.7” instead of “written by Kelsey Piper.”
Story from my past: at university, I once partook of a game called the “assassins’ guild”. It was a kind of Battle Royale. Fifty-odd participants would each be circularly assigned two “targets” from the other forty-nine, and instructed to “kill” them (for example, by writing “knife” on a stick and poking them with it, or by shooting them with a nerf gun). You’d be told their halls of residence, so you could find them there, if needed.
Your targets were revealed at 09:00 on the first day. I found my target’s Facebook page, found a post announcing her going to uni, and saw she was studying a subject which shared a module with my subject. From there I was able to pull up the timetable of her subject, guess which lectures were mandatory, and notice that she had a mandatory one right now. I “killed” her at 10:00 as she left the lecture hall.
I didn’t even get the first kill! Someone else pulled off the same trick even quicker than me!
This is with a smart uni student’s level of skill: the only thing it took was effort. If AI is good enough to do this, then privacy removal will be very easy to perform at scale.
We had basically the same game (until it was banned) many years (I presume) before your experience. I’m not even sure the game made sense in your day, with the sort of information you describe, let alone in the modern day. We have made a profound transition from a world where you didn’t know most (theoretically knowable) things to one where at near zero cost you can know anything.
It was even easier when I played this game in 2004, when people put their class schedules on Facebook.
There will be no privacy anymore, because of a simple exploit (or molochian mechanic, if you prefer that framing).
1: The government says “We need more access to information to stop [taboo]. If you disagree then you’re a [taboo] apologist or somebody who supports [taboo]”
2: The public doesn’t dare to call out the government, because anyone who argues the pro-privacy position will be accused of defending [taboo], and merely being accused of immoral behaviour can ruin your life (fired from your job, debanked, blacklisted from flying). It’s socially costly to try to defend privacy, but you can easily virtue signal by looking for those who are brave enough to do so, and accusing them of being evil.
Life is split into two halves, our public lives, and our private lives. The former is fake and performative, the latter is real, and filled with illegal behaviour (almost always benign, acceptable in the context in which it occurs, or occuring around people who understand and accept the risks. Laws apply globally and generally, but context only exists locally and specifically, and if you act appropriately locally you act inappropriately globally, and vice versa)
The only safety which has ever existed is security through obscurity. Every system can be hacked, the attack vector just isn’t known yet. What’s a password except hidden information? What is a public persona except selective hiding of information? What is privacy except the right to hiding information?
Every intelligent people like to say that they’re on the side of truth, but if you think about what that means, and what must result from it, it’s simply a terrible idea. Most parameters become destructive for society when they get too close to either 0 or 1, and the (ignorance)-(total information) gradient is no different. If every action you had ever taken and every thought you had ever thought was exposed for the world to see, your life would be ruined. But this is true for everyone except maybe for extremely boring people. Also, “acting like you’re being watched” is not very healthy, psychologically speaking. The whole virtue signaling tendency might even be a consequence of this psychological pressure to look good at any time, meaning that we could be entering a feedback loop.
Finally, all your moral examples are about things you agree with, and which the majority of people agree to. Discrimination is bad, terrorism is bad, ICE is bad. You’re making it too easy for yourself here. The publicly acceptable opinions that you’re proudly sharing online now might be taboo 15 years from now. By that time, you will either need to change what you believe, or act like your values changed. If you change, then your histoy will be used against you. If you don’t change, then you will be attacked for your “outdated” worldview.
Well said. The unfortunate fact of the matter is that I really really don’t trust most governments, and especially not the US government to implement this tech sanely. There are too many short sighted, or flat out unjust laws on the books.
To me, a canary in the coal mine is drug laws. A free society does not outlaw them, at most it limits availability and creates strong incentives to stop using them (incentives like free rehab and a support network, mandatory risk education, etc, not incentives like “we’ll imprison you for having this drug”).
I don’t either. Safety is merely the excuse for grapping more power.
I have to disagree with the drug law example. All kinds of hedonistic behaviour will likely be legal in the future, why wouldn’t it be? The old pressure against things like drugs was downstream of religious morality. That doesn’t exist anymore. The modern morality is that you’re not allowed to discriminate against any groups which hold power. What the media tells you is that you’re not allowed to discriminate against the powerless, but this cannot possibly be true. If a group had no power, then trivially, it would be socially acceptable to harm them. If it’s a social taboo to harm you, then that’s a type of power.
All super-stimuli and wire-heading you can think of will become legal, and there will only be a pushback if it gets so bad that it starts literally giving people brain damage (we have yet to see just how bad the consequences of giving babies ipads are). If these things are not legalized, people won’t be able to vent their frustrations, and they will instead start fighting back.
The set of things which will become, and are becoming illegal, includes anything which makes you unpreditable, illegible, hard to control, or able to influence people around you in a way that the public consensus does not approve of. I find that freedom of speech is a much stronger canary. Of course, they will argue that it still exists, by changing the definition of it over time “it only applies to the government”, “hate speech is excluded”, “misinformation is excluded”, “freedom of speech does not mean freedom from consequences”, and other poor arguments.
A thing to keep an eye out for is your ownership over things. It will all turn into subscription services, so that your access can be revoked at any time (for instance, if you oppose somebody in power or engage in behaviour which a company doesn’t like)
“If it’s a social taboo to harm you, then that’s a type of power.”—that thrid to last word is crucial. Throughout that whole paragraph you’re treating power as a binary which a group either has or doesn’t have and then using that simplification to argue that the power (of any kind) held by one group is equivalent to the power (of any kind) held by another group. Just because a group has some non-zero power afforded to them by a social taboo doesn’t mean that other groups don’t have far more power over them through any number of other means.
Power probably ought to be considered multi-dimensional, but don’t let that be an excuse against updating your intuition. Some people think that feminism is the most popular in countries where women are oppressed, but the relationship is the exact opposite. In countries where women are the most oppressed, feminism doesn’t succeed as a movement, and in countries where women hold a lot of power, it ends up being very popular. There’s largely no victims of injustice in society who are recognized as such, for those who are recognized as victims already enjoy protection through that recognition. The fact that this is not well known is also a testament to how power allows you to decide what’s true.
In any case, the least misleading definition I can give you is “the most powerful are those who win”, and a good proxy is “those who hold power over you are those you’re not allowed to insult”.
Do you not think this sort of social power is the most important? I find that physical power isn’t worth much in the modern world, and that intellectual power can only protect you from individuals. Scott Alexander is smart, but that didn’t save Slate Star Codex. He had to publicly defend his innocence and assure people that he held the same political views as the general public. Despite the right-wing being about as popular as the left (necessarily true since Trump won the election), the social power between being associated with one over the other was enough to terrify Scott. I think it would have gone even worse for him if he didn’t have such standard values and opinions, and if he didn’t have social power himself (famous people speaking out for him and a large count of followers).
I kind of think the power to actually kill people—legally via state imposition, or illegally, both made far easier with money, etc. matters far more. We’re just not used to worrying about being subject to that power. Correctly, for most of us.
But the implication of “we don’t need to worry about a type of power others won’t / can’t apply” is that excess use of social power is also largely irrelevant for those who don’t fear it. And it’s largely relevant to more socially powerful people—most humans don’t ever need to worry about the NY Times “exposing” them.
What would you categorize such power as? I think it’s in-between social power and good old crime. But the former will be much more terrifying in the future (it’s only slightly scary now), especially for intellectuals who do more than merely read popular research and regurgitate it to less intelligent people. If a truth seeking person never goes outside the overton window, then they’re a fraud.
Many called for me to be kicked out of society a few years ago because I refused to take the Covid vaccine, as it wouldn’t prevent me from getting infected (despite the media claiming otherwise), because Fauci kept moving the goalpost on the herd immunity threshold (never getting it right), and because I knew it wasn’t 100% safe.
Kicking people out of society (which effectively kills them) will be possible soon. Maybe “bad people” (non-conformists and those who still use ‘the r-word’) like myself are the first to go, but once it’s in place it will be easy for whoever is in power to use it on whoever is not.
You might have noticed that some people can get fired for what they do or say in their private life, and that you often don’t need to break the rules on a platform to get banned from it, breaking rules off-platform is sufficient. You can even be punished if the people you associate with are unpopular. My point is that norms are changing towards everything getting tied together. Imagine criticizing the government, and then finding that your car won’t start. And that you can’t access your bank account. And that you can’t even leave for another country because you’re on the no-fly list. Even worse, you’re also banned from your local supermarket so you can’t buy food. Where you previously had X things that you could lose access to, with one misstep costing you access to exactly one place, you now have X places in which you can lose access to everything all at once. The merging of different contexts is causing problems like context collapse. What’s acceptable to say traditionally depends on the local environment, but if what you’ve said leaks to another environment, you might be attacked for it (especially if it’s taken out of context maliciously). Everyone breaks the law, but we can still be flexible about it locally, being judged only in that local context at that time. This is changing, and it will get worse with the death of hidden information and the forced legibilization of society.
I wouldn’t say social power irrelevant to most. If you Google “Young people don’t dance” you’ll find that cringe culture (fear of being judged) is deemed to be a main cause. Irony and ironic humor is also on the rise (together with Bathos) and that’s actually a defense mechanism against judgement.
I would like to at least say that, although the vaccine was less than perfectly safe, not taking it was (and is) bad risk management: given the likelihood of eventual infection, it’s safer overall to get a vaccine than to risk a COVID infection in an unvaccinated state. And being vaccinated probably at least reduced the amount that an infected person would spread the virus. (Fauci’s “less than honest” communication was still both terrible and morally wrong.)
If you did catch the virus while unvaccinated, though, you’re in roughly the same position as people who got one dose of vaccine anyway.
Your profile says “My writing is likely provocative because I want my ideas to be challenged.”
I’m sure there are places that would work for you, and you should probably go to those places, instead of here.
One also needs to “move fast and break things” when discussing ideas if they want to learn with any efficiency. It’s a waste of both mine and others time to write “I think” and “in my opinion” in front of every statement. Besides, this place claims to be truth-seeking.
I present too many ideas at once, but I’m not exactly wrong. I think people dislike me because I seem to have different values and didn’t use examples which resonated with them (like death of privacy making it easier for ICE to deport people), but I’m getting the disagree downvotes.
To put it more mathematically and thus neutrally, the attack/defence asymmetry is growing, changing the payoff matrix in a undesirable way. To explain how social tyranny is tied to government tyranny, China’s social credit system serves for a good example and aligns with my arguments.
Can you provide more information as to what the problem is? This response only seems to prove my point, but I may actually be overlooking something important.
I’m not talking about a lack of hedging. Being too busy to think through and clearly present your thoughts wastes the time of others. And not following community norms isn’t bravery, nor is your lack of tact.
Frontpage comment guidelines:
Aim to explain, not persuade
Try to offer concrete models and predictions
If you disagree, try getting curious about what your partner is thinking
Don’t be afraid to say ‘oops’ and change your mind
It’s quite clear to me, I even added links to be transparent about the “background knowledge” I was refering to. Of course, it will be unclear to somebody unfamiliar, but that’s how it’s meant to be. There’s conversations on LW that I can’t engage because I lack familiarity, and explaining something in a way that even children can understand requires about 10-15 times more text.
I’m following those community norms. I explain myself, make predictions about the future, offer concrete models (physical, intellectual and social power as categories), and asked what category he’d put murder in. The overton window statement is an unpleasant truth at worst, but this is a website where we frequently discuss the end of humanity.
It wasn’t snark to call myself a non-conformist, I was essentially saying “even if my examples are all about people getting punished in ways that you agree with, the mechanics behind are neutral and may be used against you all the same”. I could just have quoted that poem “First they came for the X, but I did not speak up, for I was not an X” but that has been done to death. I have to disagree both that I lack tact, and that the topics warrants it. And a topic being taboo only results in public opinion being stuck in the stone age in regards to their understanding of it anyway (like mental health in the early 1900s).
When I get a response which is entirely incompatible with my own message, like the one by David above, I tend to guess at what they mean, and to present every conflict between our models, and this is probably a bad habit. Even here, I have to guess which parts of my messages people might have a problem with
Just FYI, virtually no one who genuinely lacks tact believes that they lack tact. That’s the nature of the problem: there are societal norms which, if you’re aware of them, implicitly guide your behavior and, if you’re not, you will violate without even knowing it (pretty much by definition).
Well, can you point out an instance in which I’ve lacked tact? And why would this not be a virtue on LW? Truth doesn’t care about our feelings nor our morals, it’s brutal and it’s also ugly (with the exception of the beauty in mathematics).
One can talk with the goal of socializing, or with the goal of learning the truth, the two approaches are in conflict. If you attempt to do both at once, your efficiency will be terrible.
You’re only communicating if people are listening to you. If you talk in a way that causes fewer people to listen to you you’re being anything but more efficient. The goal isn’t to issue forth your great truths into the empty void—it is to be heard and understood by the greatest number of people. Tact is the avoidance of needless offense to other people. It is a cop-out to say that the only way something brutal or ugly can be communicated is to offend people in the process. What that statement actually means is that you can’t be bothered to find a tactful way to communicate.
Some posts and comments are not for all people, and I think that’s alright. There’s a somewhat controversial quote which goes “no meaningful communication is possible among people not sharing a common window of 30 IQ points”, and I think it’s more true than false in intellectual contexts. Also, I just remembered—I got into the habit of writing in a way that stupid people cannot understand doing my time on Reddit. Precisely by filtering stupid people, do I avoid offending them, and entering into discussions with people who weren’t meant for the information at hand.
I also think it’s enought that my comments hold no malice or hostility. Information-seeking is inherently perverted, so it’s hard to do in a tasteful way. And navigating around all peoples insecurities is frankly not possible. I could claim that your comment is offensive to me, and that would be totally unfalsifiable, you’d have no defense other than asserting that I’m being unreasonable.
And in a sense, I think it’s rude for people who cannot handle reality to enter into intellectual discussions, since they cannot provide any value, and because they have a tendency to attack the people who can. Your own perception of reality is your own responsibility, this was the cultural norm around 40 years ago, and things steadly get worse as this norm fades further.
I think there’s value in phrasing some statements as “I think X” instead of just “X” because “I think X”, in practice, is a less confident assertion that doesn’t imply “and I expect you to also believe X” nearly as strongly. 🤷♂️
Just curious: do you limit all of your choices and activities to those you are certain are 100% safe?
No, but I don’t like to be in a system which has exploits without defenses. At the very least, it should be a costly action to screw people over. That way, it will only happen rarely, and when there’s a good reason for it (and one can thus avoid creating that reason).
I do not want to be in a situation in which somebody could kill me on a whim without suffering consequences for it, nor do I want to be in a situation which allows me to easily kill another person and get away with it (because I become aware that other people could exploit this as well).
I might rate above average in neuroticism, but I feel less safe as a result of more knowledge. Most people feel safe because they’re ignorant, not because they’re safe. Experts tend to be cynical because naivety is the inverse of understanding
Since we established right up front that power should be thought of a multi-dimensional (and I agree with that structure), it isn’t really meaningful to ask whether one kind (or linear combination of kinds) of power is more or less important than another.
”In countries where women are the most oppressed, feminism doesn’t succeed as a movement, and in countries where women hold a lot of power, it ends up being very popular.”—I find it misleading to compare “most oppressed” with “hold a lot of power”. At the very least we need to modify the latter to “hold a lot more power”, but far better would simply be to place these both along a power spectrum—more power vs. less power. The implication of “hold a lot of power” is that in these societies women hold more power than men, and that certainly wasn’t true in societies such as America in the 1960s when feminism gained significant popularity.
In general, I suspect that ideas such as feminism only become widely popular when they stop seeming outlandishly far-fetched, regardless of how appealing their theoretical objectives may be.
I think the main reason for the decrease of personal privacy is technological progress. There have been more back-steps (wiretapping, movement of communications onto the internet, ability for private messages to be screenshotted, normalization of video recording in public places) than forward steps (the Fourth Amendment in the United States, secrecy of voting). The main thing is that corporations and the government both want to enforce more surveillance policies now. I think it’s important to think about what that means and what consequences (if any) that would have on the general populace.
I agree entirely with that reason.
I think private messages should be able to be screenshot (because the user should control their phones and its apps. Any inversion of control seems wrong from a software development perspective)
I agree that corporations and the government are on the same side, so to speak. I think the media is more aligned with these than against. But I also think public opinion is being manipulated with tactics which make people distrust on another. Snowden was labeled a traitor following the leak of NSA papers, and this label made people reluctant to defend him.
If we assume that 40% of America thought of him as a traitor, then the media could have made it seem like 70% of people thought of him as a traitor. This could, through conformity effects, make people update their stance towards the perceived majority opinion, so that the rate of people who thought of him as a traitor actually became 70%. I believe there’s a lot of exploits similar to this, and that many people overlook them (or consider the change to be organic)
Curated. This is a fairly simple point that I hadn’t seen expressed before.
I’d previously thought a bunch about how privacy would change due to having more-of-people’s-lives available to be read online, and by having the scale to process large amounts of that data and look for patterns, etc.
I’d thought about intelligence being generally powerful. I’d thought somewhat about what it meant, for most people to have access to more intelligence. But not about how the raw intelligence-at-scale applied to privacy. The “gaydar at scale” example makes the point evocatively.
I have to point out here that author David Brin has been writing about this for years. His nonfiction book on the issue, which also goes into detail about his solution to how to organize a world in which the technology for mass surveillance is possible, is called The Transparent Society.
I realize that I don’t know what exactly did drive this change? I’ve mostly seen it attributed to a generational thing—“millennials and people younger than that don’t like phone calls”—and while I am in fact a millennial who doesn’t like phone calls, I don’t know what made it different in the past. Just that instant messaging etc. becoming available meant that calls were no longer the only way of quickly contacting someone else?
For context, I’m 65. “Back in the day” for most people, most of the time, phone calls were rare enough that no one worried about being interrupted by them coming in randomly. I suspect the frequency with which people send text messages, combined with the effectively zero cost of phone calls (hint: they weren’t always that way!), raised the frequency (or potential frequency) of phone calls to the level where people started worrying about the interruptions. Just a theory.
I think it might have to do with a millennial discomfort with intruding on people (relative to older generations), and the perception of phone calls as such.
Hard to speak for others here but the unending amounts of spam/scam texts and calls from random numbers probably does not help that situation.
I discussed this a bit with others after posting my comment and learned that the spam thing is a serious problem in the US. I don’t know what causes the difference but it’s less of a problem here. Looking at my call history I have on average one unanswered call from an unfamiliar number (presumed spam) every eleven days this year, which is relatively minor. Spam texts even less so.
I agree. I’ve had the same number in Sweden for over twenty years, and only get a handful of spam calls per year. And I’m not particularly careful about giving out my number either.
Thanks for writing this post! I agree this is important.
I’ve been banging this drum for a while:
2023-02 Preparing for Less Privacy
2011-04 Giving up on Privacy
“develop new immigration rules that cities across America would be comfortable cooperating with”—this is the “make government super-efficient because, of course, we can be certain it is always doing what we want” argument. The alternative is “always make sure there’s some sand in the gears of government because when it is not doing what we want things can go really badly, so let’s limit the velocity with which that can happen”.
Yep; I think we are entering a technological regime where slow government immediately leads to human extinction, or the replacement of that government with a fast government. While it lasted, slow government had some upsides, but I think it makes sense to change with the times.
(For example, in a lot of classic cyberpunk literature, the old governments of the world are still around, just irrelevant compared to the corporations running important parts of the important cities.)
I’m reminded of something Noah Smith said on a podcast, which I’m going to paraphrase.
The advantage of dictatorship, or other systems with extremely concentrated political power, is that dictators get things done, because nobody can tell them no.
The disadvantage of dicatorship is that dictators get stupid things done, because nobody can tell them no.
What is your theory of how fast government can prevent extinction?
I think this is easiest to see in biorisk contexts. A fast government is more able to process new information, identify new threats and respond appropriately, and develop new capacities as required. Like, we’re still ~2 years in to discussion of mirror life with no ban / official government designation; with the current rate of technological progress, that’s ok but I think it wouldn’t be if technological progress were running ten times faster. (And that’s just saying “this isn’t ok”, not even developing new monitoring systems or countermeasures.)
“A fast government is more able to process new information, identify new threats and respond appropriately...”—I was with you right up to that last part. In what way is a fast government more able to respond appropriately?
The main answer is “responding at all is better than not responding”. Yes, I’m aware that governments have made lots of terrible decisions over the years—it’s not obvious to me that if we had the Bush-created pandemic preparedness office at the beginning of COVID (rather than it being dissolved earlier by Trump), they would have made things better instead of worse—but from my vantage point it is obvious that Operation Warp Speed was good and the FDA being slow (both on testing earlier and on approving the vaccine) was bad. In many contexts, velocity is a virtue on its own.
If we accept your premise that slow governments immediately lead to human extinction that in no way implies that fast governments are any better. Yes, if a fast government made the right decisions it could, in theory, implement those decisions more quickly, but I see no logical reason to believe that a fast government is any more likely than a slow government to make correct decisions and quite possibly the opposite. I’m afraid I don’t find the contents of cyberpunk literature, classic or otherwise, to be a basis for large-scale political decisions or structures.
I actually wonder whether information asymmetries will even be sustainable in the long run, or whether we’ll eventually approach a world where access to information is relatively equalized (that will all depend on how centralized things become). If things do become fairly equalized, the issue may shift from information asymmetries and how to control them to attention asymmetries. I argued something like that here.
I think you hit on this some by shifting the argument from “what can be known?” to “what should be done with what is known?” But I think the next problem may be more than just good attention-allocation decisions. Rather, it may be more so establishing proper defenses against AI-powered attention manipulation, since that seems a likely next locus of control.
I wonder if there are technological shields that can be developed—using intelligence—to protect / shield privacy. Similar to VPNs. Like suggest parts of my face I could cover to hide my heart rate or insert noise into my call or browsing history.
There are, but none will be allowed to exist. All tools which help privacy also makes it easier to get away with breaking the law or engaging in immoral behaviour. Online anonymity is under attack through ID verification, and they’re trying to ban encryption. VPNs are also increasingly illegal.
The system is trying to make the world more legible so that it can automate its judgement of you, the processing of your information, and the collection of your private information (since it’s valuable to companies and attractive to governments). They will claim that you must be doing something illegal if you want privacy, and since most people don’t want to look bad, few will come to your defense.
Where (outside of widely-recognized-as-totalitarian societies) are VPNs illegal?
Those bans are a work-in-progress. I expect it to take a few more years personally.
Senate Bill 73 was signed on March 19, making websites accountable for users use of VPNs in the state of Utah. Denmark tried to ban VPNs recently (perhaps not a full ban, just a move to “combat piracy”), but due to pushback it was dropped. Officials in France have talked about VPN bans in connection to a planned social media ban for teens under 15, and there’s a proposed EU regulation called “Child Sexual Abuse Regulation” (it’s also known as Chat Control”) and VPNs seem like a likely way to get past it, so I figure VPNs will be targeted next. VPNs also have legitimate uses which are hard to ban, so they might simply restrict their use.
When laws like this don’t go through at first, they simply try again and again until they manage, so it’s just a question of time. The slippery slope fallacy is rarely much of a fallacy when it comes to things like this
My impression was that the FBI and CIA etc has always been trying to ban encryption an similar, but has so far mostly failed. Strong encryption was illegal (from the Wikipedia page on PGP):
”in February 1993 Zimmermann [inventor of PGP encryption] became the formal target of a criminal investigation by the US Government for “munitions export without a license”. At the time, cryptosystems using keys larger than 40 bits were considered munitions within the definition of the US export regulations; PGP has never used keys smaller than 128 bits, so it qualified at that time.”
But this was later changed to allow strong encryption for the public. So while I share your worry, I take hope that the slippery slope is not inevitable.
I’m reminded of https://xkcd.com/504/ (which was written before “crypto” meant “cryptocurrency”).
It sounds like you’re arguing that every restrictive law that’s ever attempted is eventually enacted. Is that really your claim?
Either the restrictive law itself, or a close equivalent, yes. But the reason is that tyranny is an attractor state in many systems.
Revolutions, civil wars and other strong disruption such can reset a tyrannical state into a more free state, but that seems to happen less recently due to globalism, and there’s a sort of meta-progress of away from freedom which is not reset, so the baseline of privacy and freedom is approximately the inverse of technological achievement.
Now, you might want proof of this, or stronger arguments than what I have. In the hard sciences, which society regards as the most difficult intellectual work, proofs are possible. But every day life is actually much more complicated than the hard sciences are, and too many things depend on eachother for us to be able to isolate anything as a cause. Some people, myself included, are good with more complex systems, and we’re no worse predictors than those who specialize in just one thing and get renown for that. Academia is rather hostile to those who don’t commit the McNamara fallacy, as them commiting the fallacy makes them unsympathetic towards people who make claims which are difficult to verify. Nassim Taleb has pretty good takes on complex systems which I recommend
I think if you actually dig into the facts here you’ll see that while there may be a gradual increase in restrictive laws over long periods of time the majority of restrictive measures introduced—just like the majority of nearly all measures introduced—fail. This is particularly true at the Federal level, while at the state and local level there’s a lot of variation from place to place.
Deflecting demands for proof, particularly as that term is understood in the “hard” sciences, is not unreasonable. Deflecting demands for stronger arguments is just intellectual laziness.
The period is not so long. It’s enough that if you were to implement 10 years of change in a single day, you’d have mass protests or a civil war on your hand. Culturally, the world is currently changing very fast, since we can only compare to the past, which changed much more slowly.
A gradual decrease in freedom over time is a really bad sign, for the same reason that a gradual increase in temperature, and a gradual decrease in birth rates are really bad signs. The media attacks the social status of those who don’t panic enough about global warming, and it attacks the social status of those who panic too much about privacy, and then people compete in showing how good and proper they are (by signaling that they do not belong to the group which is being criticized). It’s a waste of time to argue against common opinion, even when one is correct and in possession of good arguments. Disagreements about facts play out as if they’re power and value conflicts
This read to me as a bit inevitabilist. Putting aside for a moment that it might be hard to avoid: do we actually want to live in a world in which anyone can easily find out anything about anyone else?
See also: large language models will be great for censorship, large scale de-anonymization with LLMs.
Appreciate the concise but insightful post. This issue strikes me as one of the most important mundane impacts to get ahead of.
Beyond law enforcement implications, it feels like public data dumps of information (biometric, geolocation, text comms) will take google-stalking to new heights.
Expansion of panopticon capabilities seems inevitable… do you see any other plausible levers (along the lines of Overseeing the Overseers you mentioned) that keep implementation within limits acceptable to today’s public?
It’s retroactively true too: anything you’ve ever been caught doing on camera, and any post you’ve ever made to a forum (under any pseudonym), will soon be linked to your identity by any entity with access and capacity. Behaving more circumspectly from now on won’t protect against that, unfortunately.
Incidentally, I’ve been making this point since at least 2021.
Can you say more about why you strongly suspect this?
This post is deeply discomforting for me, but I am worried that it will be the future regardless.
I must say I disagree on your issue with the European approach. I do think that the current ignorance approach to preventing racial discrimination is the best one. Unless that sentence is completely unrelated to the previous one, and I completely missed your point. (this is highly probable, so if it is the case, please do explain what part of EU privacy law you are concerned about, and ignore the rest of this comment).
Doing option 1 relies on trusting people to make unbiased decisions – people that, even if honest, cannot guarantee they will not be influenced by subconscious biases. One solution to that which is common in some places is to enforce certain quotas, but that makes the competition itself less fair, among other issues.
Option 2 seems to be “just let people discriminate”, which I think we agree on is not optimal. Although feel free to tell me if I’m missing something here.
Option 3 makes it easier for people to actually focus on what does matter, without being distracted by things that only bring up prejudice. This isn’t being ignorant about important things, it’s using a screen to make it easier to not be distracted by things that aren’t relevant that would have caused one to make worse decisions.
Of course I’d prefer to live in a world where we don’t need to hide information that shouldn’t influence people, but that doesn’t seem realistic to me in the short and medium-term.
Isn’t what information shouldn’t (and should) influence people constantly changing?
What we as society think is moral is something that changes all the time, and laws and norms follow it, that’s just the way society works. That doesn’t mean we should give up on enforcing morals because they change anyway. We make laws that try to stay up to date with what we as society think is fair.
Are you equating access to information and morality?
Been thinking about writing an article like this for a decade or so, especially linking it to the sharing of information between institutions such that access to (usual) public goods that people often use to function (Gmail Accounts, Apple Accounts, X accounts, any account or sign-in where the company can decide to ban you for any reason) ever since I heard about people potentially not being let into a bar because they behaved poorly at a different bar. This combined with cancel-like culture is going to make the future a very unfun place to live until the norms change and people realize what it’s like for their privacy and things they don’t consider all that bad about themselves to be judged by groups who find those things bad.
In some cases, there will be stalking.
Blackmail and other kind of fraud are also easy when gathering more knowledge about the target becomes easier.
In the Chinese system, there’s also Social Credit Scores. The surveillance information gets used to grade your behavior and you get benefits accordingly, without really knowing what specific behavior caused the benefits.
I don’t get this part.
Why would the acceleration of true information accessibility in one domain, eliminate legal/cultural barriers to it in another?
It’s definitely true that AI can physically enable both of the following,
Improved stylometric analysis of civilians, for people who are more likely to have ties to Hezbollah
Maximally accurate reconstruction of unclothed 3d models of conventionally attractive public figures
Yet, these two examples feel worlds apart, to me, in their likelihood of ever reaching mass adoption. The former might already be happening, yet the latter seems all but guaranteed to go down your 3rd path (though ‘pretend ignorance’ may be the wrong term for it)
We spend enormous energy protecting bank statements at some level, treating them as surveillance risks. Yet people voluntarily share with AI systems something far more revealing: how they actually think, what they believe, what they are uncertain about, etc. A bank statement can tell you someone eats at the same place twice a week. But a long AI conversation tells you how their thinking drifts, what they are afraid of, what kinds of arguments move them, and thoughts they would never say out loud. I think the most sensitive data is no longer behavioural.
Using expectation as a limiting factor cannot end well.
Duncan Sabien’s recent post on “social dark matter” is relevant, and a good read. A decrease in privacy will, by Sabien’s principles, most likely lead to a societal reckoning with quite a lot of social dark matter. This paints a rather hopeful picture of growing intelligence ultimately leading to growing acceptance, but the road there will probably be rather rocky.
I would correct this to “they can be (and sometimes are!) used.” I imagine this is a typo.
Related video:
“start behaving as if you’re being watched”—I’m not sure what to make of the fact that the chillingness of these words isn’t overwhelmingly self-evident. It is much easier for a totalitarian government to pick off a few isolated dissenters than a group that was able to gather and organize “in secret” at least up until they reached a viable size.
This post persuaded me to unshare most of my Submissions to this platform.
The worst case scenario, of course, is a system enforcing an absolute lack of privacy, when that system is owned by an enforcement agency which is not dedicated to any true prosocial goals, but to the continuation of its own power structure. And that’s the end state of all surveillance and power structures because that’s where the motivation and feedback reward loops trend. Anything which does NOT enforce its own monopoly on power is overtaken by things which DO enforce their own monopoly on power. The has nothing to do with the advancement of humanity or science or anything like that—we could easily end up in a very stable, very stagnant worldwide society of dictatorships which do not promote growth in any meaningful fashion because the mechanisms of enforcing control have been perfected and the need to enforce control remains the top priority and survival strategy, well over any other goals.
The more power you have, the more you can enforce your monopoly, the more incentive you have to enforce your monopoly. Consolidation of power occurs and at the top levels, and then high level echo chambers drive antisocial, anti-humanistic behavior with no consequences.
Congratulations, you’ve invented something worse than “I have no mouth and I must scream”: the perfect private torture chamber for an all powerful cadre of the worst humanity has to offer.
Most people will be mostly fine, of course. Even massively corrupt abusive slave owning states usually manage to ensure some sort of reproduction and survival threshold, and people can adapt to almost anything—read some North Korean escapee stories with the understanding that most people in North Korea are, in fact, happy most of the time, even if they have zero effective freedom or control over their destiny. The people who escape were people who were unhappy enough to try. Not everyone wants out.
Also, understand that you—the reader here—will likely not be part of the owning class in any such state. Everyone imagines some way in which they will hack the system, in much the same way that in the heights of the memory regression fad everyone was telling stories about being royalty in a past life. You will at best be a valued worker for a little while, in so far as you are willing to utterly subsume your ethics, morals, code of conduct, and personal identity to that of whatever owns you.
Studying the worst of historic fascist dictatorships and slave states with an eye towards the behaviour of the owned folks is a nice way of seeing the survival strategies which might work for you in the future, so you can extrapolate a bit for ways to retain scraps of your personal independence and dignity. For example, if you’ve the resources for it, it might someday be useful to consider botox for facial paralysis to prevent cameras from inferring anything about your thoughts. :)
Okay, the scenario you’re presenting is clear. Let’s imagine a future where privacy isn’t a real human concern.
Children with continuous brain monitoring systems being virtually cloned to create a brain map under the guise of being able to teach them any knowledge quickly. They might say: “We have to be the country with the best researchers; it’s necessary.”
Interconnected relational maps of private human knowledge, relationships, biases, etc., could be used under the pretext of preventing terrorist attacks.
A system assisted in redirecting the behavior of people under surveillance. In this future, the goal wouldn’t be to eliminate people but to redirect their thoughts. An AI seeing what your contact lenses see, correcting your negative impulses (negative according to the norms of that state) in real time.
Systems for uploading private information to the cloud. The problem of memory would disappear, for better or for worse. Dopamine from the past will flood your mind when you need it.
Are there early signs that indicate we are heading towards that future?