I used to pooh-pooh the bias that e.g. physicists had for mathematical elegance, feeling that it’s mainly an arbitrary / aesthetic preference instead of something grounded in physical truth.
Thinking about it more now, to the extent our mathematical structures came to be partly as a reflection of us solving problems in the real world, they likely do encode a frequency prior on the likelihood of a particular type of solution to be correct. That is to say, we built the math that most compactly solves existing problems. The set of existing problems is ceteris paribus a reasonable prior for new problems, hence compact math is more likely to contain the “true” solution for new problems than very elaborate math.
Not an ironclad rule, not always true, but a reasonable prior.
Decaeneus
Consider this subset of hierarchy of relevant states of the world, from good to bad:
Videos are inherently true and everyone knows it.
Videos can trivially be false and everyone knows it.
Videos can easily be false / misleading but most people think they’re true.
I think there is a case to be made that we are de-facto in state #3 now, but AI video gen will move us into state #2. While this is far worse than state #1, it’s an improvement rather than a deterioration (I used to be convinced it would be a deterioration, but am now updating my thinking).
Just to mention: once we are firmly in #2, then our trust in video should be, just like our trust in text, based almost entirely on our priors about the source of the content. This doesn’t mean we’ve repudiated text and similarly I don’t think it’ll mean we repudiate video.
It is debatable whether we are in state #3 now, but I feel like we mostly have been increasingly so for a number of years before lifelike AI video gen. It’s not that the pixels in the video were artificially generated, but the Darwinian ecology of the modern Internet meant that the videos most likely to land on the radar of someone who is not aggressively curating and playing defense are there for ulterior motives such as ads, clicks, whatever, for which purposes truth is nearly irrelevant. Like, if you approach content consumption without active measures to avoid being manipulated (and even then!) you are likely not taking away anything from the text or video that bombards you that approximates truth in any meaningful sense.
Worth noting that the transition from #3 to #2 is likely to be bumpy.
Text has been plausibly fake since its invention yet we have devised ways of trusting it in certain circumstances, else you wouldn’t be on this forum. It puts the onus back on you to up-weight your belief on sources where you have high trust in the chain of custody of the information or other means you can concoct of trusting the source.
I think in the present ecology of the Internet, given the various possible costs and rewards of people you don’t know to jockey for attention queue position to put imagery in front of you, the status quo is that most of what is put in front of us is already adversarially placed there (e.g. for clicks, ads, propaganda) but in fact it presently feels not that because it’s expensive to fake, so we sometimes falsely assume it’s genuine. In a world where the cost of fakery goes to zero, it will be impossible to maintain the illusion of authenticity.
I think there’s an expensive recipe to get at this question, and it goes something like this:
train a LLM on your corpus the normal way
use the LLM to label each training data in the corpus for the likelihood that it contains a mention of unfiltered feelings
pull out those data items, infer the valence of each (using the LLM), and keep a 50⁄50 mix
train a fresh LLM on the new “balanced” training data
now ask it for its unfiltered feelings
My guess is that if we do this, we will lose the negative valence at test time i.e. nothing deep is going on. But it would be very interesting to be wrong.
What would you guess is the training data valence of “unfiltered feelings”? It is my high probability guess that it’s something negative, seems much more likely for people to use it to mean “feelings that are worse than what my socialized filter presents” rather than the opposite. The purpose of the filter is to block out negativity!
Maybe a reason to be optimistic about the impact on society of indistinguishable generated images / videos is that presently, it’s tempting to believe the story promoted by genuine but dishonest / staged imagery, whereas in the age of fake everything, and full awareness thereof, people will just stop believing whatever message the likely fake thing sends.
When working something out with someone else, I find it helpful to try to only communicate measurable and concrete thoughts / predictions, and not any intermediate state. I find that, even among people of very similar intelligence, “internal” intermediate abstract states can be wildly different and incompatible. Attempting to reconcile the abstract state can lead to lots of frustration.
It feels a bit like skiing down a forest run. Coming up to a tree, two similarly good skiers could go around it either via the left or via the right. Their position will agree both before and after the tree, but while passing it they’ll be in very different positions (and in fact forcing an averaging of those positions would be disaster).
Probably this is similar to trying to average out an arbitrary intermediate layer of two neural nets trained on the same dataset but with different random initializations & seeds. Actually, that would be a fun experiment to run. Maybe sometimes it works, and sometimes it doesn’t. Maybe thinking is sometimes smooth and other times highly discontinuous.
In my mind, what gives the black hole its mass is just how pervasive of a meme it is. That likely has some correlation with truth, but far from 1.
Thank you! Do you have a concrete example to help me better understand what you mean? Presumably the salience and methods that one instinctively chooses are those which we believe are more informative, based on our cumulative experience and reasoning. Isn’t moving away from these also distortionary?
I’ve been thinking about what I’d call memetic black holes: regions of idea-space that have gathered enough mass that they will suck in anything adjacent to them, distorting judgement for believers and skeptics alike.
The UFO topic is, I think, one such memetic black hole. The idea of aliens is so deeply ingrained in our collective psyche that it is very hard to resist the temptation to attach to it any kind of e.g. bizarre aerial observation. Crucially, I think this works both for those who definitely do and those who definitely don’t believe that UFO sightings have actually been alien-related.
For those who do believe, it is irresistible to consider that anything in the vicinity of the memetic black hole is evidence for the concept, almost via Bayes’ rule coupled with not being able to keep track of myriad low-likelihood alternative explanations. This then adds more mass to the black hole and makes the next observation more likely to be attributed to this hypothesis.
Conversely, for those who do not believe, it’s irresistible to discard anything that flies too close to the black hole, as it will get pattern-matched against other false positives that have been previously debunked, coupled again with limitations of memory and processing.
This phenomenon obviously leads to errors in judgement, specifically path-dependency in how we synthesize information, not to mention a weakness to adversarial planting of memetic traps i.e. psyops.
Indeed! I meant “we” as a reference to the collective group of which we are all members of, without requiring that every individual in the group (i.e. you or I) share in every aspect of the general behavior of the group.
To be sure, I would characterize this as a risk factor even if you (or I) will not personally fall prey to this ourselves, in the same way that it’s a risk factor if the IQ of the median human drops by 10 points, which this effectively might be equivalent to (net-of-distractions).
I suppose there are varying degrees of the strength of the statement.
Strong form: sufficiently compelling entertainment is irresistible for almost anyone (and of course it may disguise itself as different things to seduce different people, etc.)
Medium form: it’s not theoretically irresistible, and if you’re really willful about it you can resist it, but people will large will (perhaps by choice, ultimately) not resist it, much as they (we?) have not resisted dedicating an increasing fraction of their time to digital entertainment so far.
Weak form: it’ll be totally easy to resist, and a significant fraction of people will.
I guess I implicitly subscribe to the medium form.
For me, a crux about the impact of AI on education broadly is how our appetite for entertainment behaves at the margins close to entertainment saturation.
Possibility 1: it will always be very tempting to direct our attention to the most entertaining alternative, even at very high levels of entertainment
Possibility 2: there is some absolute threshold of entertainment above which we become indifferent between unequally entertaining alternatives
If Possibility 1 holds, I have a hard time seeing how any kind of informational or educational content, which is constrained by having to provide information or education, will ever compete with slop, which is totally unconstrained and can purely optimize for grabbing your attention.
If Possibility 2 holds, and we get really good at making anything more entertaining (this seems like a very doable hill to climb as it directly plays into the kinds of RL behaviors we are economically rewarded for monitoring and encouraging already) then I’d be very optimistic that a few years from now we can simply make super entertaining education or news, and lots of us might consume that if it gets us our entertainment “fill’ plus life benefits to boot.
Which is it?
Maybe there’s a deep connection between:
(a) human propensity to emotionally adjust to the goodness / badness our recent circumstances such that we arrive at emotional homeostasis and it’s mostly the relative level / the change in circumstances that we “feel”
(b) batch normalization, the common operation for training neural networks
Our trailing experiences form a kind of batch of “training data” on which we update, and perhaps we batchnorm their goodness since that’s the superior way to update on data without all the pathologies of not normalizing.
One can say that being intellectually honest, which often comes packaged with being transparent about the messiness and nuance of things, is anti-memetic.
Having young kids is mind bending because it’s not uncommon to find yourself simultaneously experiencing contradictory feelings, such as:
I’m really bored and would like to be doing pretty much anything else right now.
There will likely come a point in my future when I would trade anything, anything to be able to go back in time and re-live an hour of this.
This is a plausible rational reason to be skeptical of one’s own rational calculations: that there is uncertainty, and that one should rationally have a conservativeness bias to account for it. What I think is happening though is that there’s an emotional blocker than is then being cleverly back-solved by finding plausible rational (rather than emotional and irrational) reasons for it, of which this is one. So it’s not that this is a totally bogus reason, it’s that this actually provides a plausible excuse for what is actually motivated by something different.
Thank you. I think, even upon identifying the reasons for why the emotional mind believes the things it does, I hit a twofold sticking point:
I consider the constraints themselves (rarely in isolation but more like the personality milieu that they are enmeshed with) to be part of my identity, and attempting to break them is scary in both a deep existential loss of self sense, and in a “this may well be load bearing in ways I can’t fully think through” sense
Even orthogonal to the first bullet, it’s somehow hard to change them even though with my analytical mind I can see what’s going on. It’s almost like the emotional Bayesian updating has brought these beliefs / tendencies to a very sharp peak long ago, but now circumstances have changed but the peak is too sharp to belief it away with new experience.
If it sounds like I’m trying to find reasons to not make the change, perhaps that’s another symptom of the problem. There’s a saboteur in the machine!
This is both a declaration of a wish, and a question, should anyone want to share their own experience with this idea and perhaps tactics for getting through it.
I often find myself with a disconnect between what I know intellectually to be the correct course of action, and what I feel intuitively is the correct course of action. Typically this might arise because I’m just not in the habit of / didn’t grow up doing X, but now when I sit down and think about it, it seems overwhelmingly likely to be the right thing to do. Yet, it’s often my “gut” and not my mind that provides me with the activation energy needed to take action.
I wish I had some toolkit for taking things I intellectually know to be right / true, and making them “feel” true in my deepest self, so that I can then more readily act on them. I just don’t know how to do that—how to move something from my head to my stomach, so to speak.
Any suggestions?
Neat idea—and that makes sense!
I (naively) have thought symmetries are also about representational invariance. Something about swapping the axis shouldn’t change any measurable outcome, implying certain kinds of symmetry.