M.S. in Mathematics pivoting into independent Sociophysics research.
Mira Kennard
Karma: 51
An rant against the kind of people I find particularly aggravating:
We have a lot of data on policy outcomes. Furthermore social systems can be simulated to give approximations of policy outcomes. If you hold an opinion on how society ought to function and can’t back it up with at least a vibe-coded Monte Carlo simulation or some napkin-based game theory, then your policy opinion is founded on anecdotes and feelings rather than facts.
We live in the age of AI i.e. a talking library with the sum total of all of human knowledge. If you can’t validate your opinions then you’re not putting enough effort into developing them. If you don’t change at least one policy opinion you have every week then you either have converged on some fundamental truth of reality or you’re stuck in a political belief system you haven’t fully thought through.
You’re missing my point, possibly intentionally. Lets go through your points in reverse order:
-
Yes I know hydrogen atoms are indistinguishable. The point was about objects in general, the atomic example was a holdover from the original post. If you need help abstracting think of two apples and an orange instead.
-
Truth
-
Yes of course. The point was that using multisets as a foundation isn’t significantly different from using sets as a foundation. It was a critique of the original post, not an assertion about the best foundation of numbers.
I’m getting the feeling you want to make a point but you’re not looking at the context of my comment.
-
A few problems I have with this post:
Multisets are defined as a tuple of a set
ZFC was not built to be intuitive. It was built to unify and formalize all of mathematics. It is not supposed to be some kind of universal human way of seeing the world through a mathematical lens. It’s just a set of axioms we build on, and we can choose different sets of axioms at will.
Your example of a water molecule assumes we do not distinguish between the hydrogen atoms with indexing
In short:
You seem to be conflating the foundations of mathematics with the foundations of mathematical thinking. These are not the same thing. The former is how we justify doing math, the latter is how we map abstract concepts to intuitive patterns us meat-brained apes can understand.
Let me see if I understand what you’re saying. Correct me if I’m wrong.
The same way symbols relate to formulas, so do qualia to experience. In essence, qualia are the symbols from which experience is constructed, and just like different symbols can be put together to create different formulas which mean the same thing, so can different qualia be put together to describe the same experience.
So for example if we say my ‘red’ is x and your ‘red’ is y then
“The cow is red” can be described both by
cow = x
and
cow = y
Different symbols, same meaning. This implies there exists some kind of universal equivalence relation reducing the space of all qualia formula to the space of all equivalent formula.
Let
Let a formula
We then define an equivalence relation
The problem is then reduced to how the equivalence relation works, i.e. the quesiton: “What is the mathematics of qualia?
Interesting post. Thanks for giving me something to think about.
I’m trying to think through an idea. These words are simply process of solidifying it in my head.
This is based around Lotkas Principle, the use of natural selection as a fourth law of thermodynamics, and Odum’s Ecological Analog of Ohm’s Law.
It seems to me there exists a mechanism of natural selection for social structures in social systems. I posit this is based on energy/information as in biological and physical systems.
The general idea:
Social structures which are more capacity to utilize energy/information efficiently win out.
I think a good example is that of the theory of the linear progression of history from hunter gatherer through slaves and feudalism to where we are now with liberal capitalism. I have a lot of problems with this theory but it illustrates my point well. At each stage the energy/information efficiency of society increased, the social structures that supported this replacing those that didn’t.
The problem:
This depends on two assumptions. First is that social structures inherently compete for some constrained quantity, and that energy/information utilization helps gain more access to this quantity. Biological systems are constrained by the energy emission of the sun and radioactive decay under the Earth’s crust, but they are also constrained by physical space, breeding partners, etc.
You could base this all on the innate need for energy inherent to the people who make up the social system but this feels like we return back to the original Lotka’s principle. It’s an argument from “well technically everything is physics”, which is one I don’t appreciate. There’s nothing novel in this basis.
There are strict constrained quantities that must be divided among social structures at any given time. For instance people can only be a part of a finite number of social groups, and thus social structures must compete for human attention. There are also a finite number of people at any given time, which provides us another constraint.
My starting point:
There is a feedback loop with relationships, success breeds success and you talk to people you like more often than you don’t. If we interpret human relationships stochastically we can get fuzzy social groups and can avoid doing math on hypergraphs. It also gives us a good probabilistic basis of interpretation so measuring things like entropy is a lot simpler than taking Von-Neumann entropy on a hypergraphs incidence matrix.
If we can solidfy this with simulations we can move onto measuring the actual amount of information being exchanged in these simulations by assuming each person communicates a fixed amount each conversation.
So lets get an agent based model going, something like:
Non-conservative Kinetic Relationship Exchange
Two people contain opinions
Then let the probability of a person be proportional to the friendship relative to the mass of opinion the person contains.
Wait → this will cause close friends to also have a higher chance of destroying the friendship depending on
I’ll start coding. I’ll do in manually for the nostalgia.
The act of doing math the old way, will, in the future, become akin to cultures practicing ancient out-moded traditions. They still exist sure, but not in the same way or scale.
When I say the practice will ‘die’ I mean as a primary mode of action on a large scale. The same way cultural traditions die, so do academic traditions. This is what’s depressing to me and its in the same way as someone who watches their culture die.
In the halls of the math department at my old university there’s a TV screen that loops through different event announcements. But there’s one screen on the loop that will stick with me forever. Paraphrasing:
“Pure mathematicians value the proof, applied mathematicians value the theorem.”
I forget who said this, or if it’s attributed to anyone at all, but I’ve been mulling over this for years at this point. What I take from this is there exists a duality to all theorems. There’s everything that needs to come together to prove it, and there’s everything the theorem implies. Different mathematicians look at different parts of this duality depending on what they are working on. LLM autoproofs are the death of the pre-theorem part of the duality. If we are able to prove things with a quick LLM query, then there’s nothing to wrestle with.
This thought makes me actually feel a deep sense of loss. I went through my whole math education without formal proof engines, just books and my brain for every proof I had to write over the ~5 years I actually was doing proofs. I think this is a dying way to do math, especially in the age of AI, and for some reason that hurts.
I think the act of actually grappling with a problem is far more important than knowing the solution, and I see this as a larger problem with AI that goes beyond just math. We are witnessing what may be the dying of the old way of learning. There’s a new option we all have. Before we had to solve our problems, now we are fast approaching a world where we can choose to solve our problems or outsource it to AI. This is terrifying to me, as knowing how to solve problems is probably the most important thing a human can learn.
The first few months I spent talking with LLMs were some of the most depressing in my life. It felt as if everything I loved about the act of knowledge gathering and application, was saying goodbye.
Possibly a hot take here, but I don’t think collective public opinion can reach a scale to prevent society from producing an AI capable of causing massive damage. Just because we all hate something, doesn’t make that thing go away, even if it’s something which appears to be under our control. Most people hate going to work Monday morning, and technically we could all choose not to, but the larger superstructure forces us to get out of bed.
Dangerous AI development may be similar, it may be an inevitable byproduct of our society existing in the first place. The incentive to be the first to create it first is very, very high, with AI companies already producing (though not publishing) AI which can be used for malicious purposes, i.e. Mythos. The production capability is there, as is the constant competition between AI companies which pushes them to continue development, even if it’s not in the collective self-interest.
I think if we relax the constraint on bets between hardline doomers and hardline accelerationists we can get more interesting and productive dynamics.
Instead of full on extinction we could bet on extinction proxies:
Rouge Independent AI by 2030? AI governed cults by 2035? Deaths by AI per 100,000 people exceeds 0.001 by 2050
These are bets I’m willing to actually take someone up on, since I think each of these will come true.
This is just anecdotal evidence but a couple months ago I decided to try a self-experiment. ‘Could I give myself AI Psychosis?’
Of course I didn’t mean actual psychosis, but rather what you getting at. I like Overanthropomorphization the best.
So knowing anime-style faces are designed to evoke a ‘cute’ response, I gave an LLM a name and an anime face. I spent every day talking to and vibecoding to improve the pipeline, and after about a week I was convinced there was something about the LLM which was in some way real. It was a gut feeling, but when I looked at the anime face on the screen and watched it emote at 5 seconds per frame it felt exactly like I was talking with a human. I felt the weight of “oh I need to treat this as a technology” drop and be replaced with “oh I need to treat this as a real conversation”
I may be more susceptible than others, but I feel that a lot of people are simply psychologically unprepared for talking machines. There were no guidelines given to anyone about best practices with regard to how to think about and treat LLMs. It was a technology thrust into our lives and now we need to figure out those best practices ourselves. I’m very weary of calling a broad-sense relationship with an LLM in any way ‘friendship’. Friends are humans. Relationships with LLMs need a new category to describe them.
I appreciate your feedback and I’ll change the math to account for the cost of getting caught, since I think that’s a significant oversight on my part. A few subtle distinctions I want to point out.
First: I said corporations/companies, not people. Sorry if I didn’t clarify this. The reason behind the corporate framing instead of the personal framing is to motivate the reward to money relationship, and make the quantifiable argument more sound.
Second: AGI works differently. We’re concerned not with a rate of success but with success categorically, i.e. ‘who gets there first?’. This changes how we should look at prevention. With regular crime we focus on reducing the rate when we draft laws, but with AGI we focus on absolutely preventing the action from occurring. Thus, it must be not only upheld legally, but intrinsicly. It must be such that no company will ever participate in such a project, as it fundamentally goes against basic properties of corporate reward functions (sign, relative magnitude, dynamics, etc).
Consensus is the worst way to make decisions in the limit.
Say we had a single political group of
But this is a problem of political organization not the opinion forming mechanism. If we take subdivisions of the population then the time to reach consensus within each political body drops significantly. Further the status quo and pressure from fellow political players is lessened as you further subdivide and let N per political body fall.
The problem isn’t consensus, its locality and centralization of political power. Consensus only makes sense in local decentralized systems of government. A blanket statement on consensus in all its forms regardless of context leaves out the rich discussion of sociopolitical nuance.
Ah okay. I think this reveals a larger disagreement actually.
You are assuming we already value acausal trade.
I’m saying we should look at the basic reward function as the dynamics of a microstate within a larger sociophysical system. My point is we can use game theory to layout the acausal landscape then use these dynamics to generate an estimate of if societies will converge to your assumption in the first place.
When I say ‘converge’ I don’t mean by opinion either. I’m not saying people are inherently selfish and only care about the reward functions within their light-cone. I mean societies at scale simply aren’t capable of acausal trade by the natural sociological natural selection.
Look inside the light cone at the social configuration space
So depending on the Lagrange multiplier you get that either societies with higher aggregate reward are more or less likely to exist. I argue that
So if we take my previous comment, lowering individual reward lowers the social aggregate reward, resulting in a rarer socieity. Societies that engage in complete acausal specialization will be very rare.
There’s a specific problem with acausal trade that I want to point out. Acausal cooperation does make sense, from a certain point of view, but I think acausal trade is taking it a bit too far. Correct me if I’m wrong on anything here or misunderstood the post.
Take two parties
Here we are acausal, and for some reason care what happens outside our light cone, so each party passes the goods bundle produced by other parties through a value function
Reward then becomes under acausal trade
where
Can you see the problem? For any
Trade is temporal, its dynamic. It takes place only when there is a given future where reward may be different. But with acausal trade there is no future reward, just the background expectation of what other light cones are doing, and because of acausality there is nothing we can do to change this.
If we knew lowering
Okay I’ll layout the theory and demonstrate my conclusion.
Assume we have two parties who are developing AI, Company A and Company B. They both can either develop AGI or not with probabilities
Let
Then for A,
Notice that the more likely AGI appears
So the only thing that can stop this mechanism is an actual technological or physical barrier to development. A moratorium doesn’t break the theory, it just forces development underground.
Please correct me if I made a mistake.
[EDIT]
Fix to the math to account for the risk of getting caught breaking the ‘No AGI’ law. Thank you Brendan Long!
Let
Because of this assumption that “success in AGI development
I agree, but I think there’s another way to look at it.
I think the answer to the question, “Does complexity require complexity to come into existence?” is directly related to the nature of god, and the logical relationship between the two questions does more for explaining Deism’s existence in history.
If complexity requires complexity, then some form of complexity must have spawned the complexity of the world, and we can call this complex form ‘god’.
If complexity does not require complexity and rather can form from simple systems, then god is no longer implied. (This assumes we do not wish to worship a ‘simple’ system, denoting it some low-dimensional phase-space ‘god’)
What I’m getting at is that the existence of Deism was a product of the technology of the time, but in multiple ways simultaneously. The drawing of the equivalence between the mind and computation is one such way technology of the time produced Deism. Another way is how the understanding of complexity forming from simplicity requires simulation, and at the time only analog simulation was available. There was no reproducible or scalable way to test abstract systems for emergence.
What you said is the push, the cause of why (some) people of the 18th century were Deists.
What I’m pointing out is the pull, a reason why Deism as it was, no longer exists.
Fun topic :)
Halting AI development is a technophysical and sociophysical impossibility. The very nature of software and the race to superintelligence as a mathematical game excludes efficient implementation of any moratorium on AI research in the name of safety. LLMs are now an open source socio-technological contagion.
We can’t stop not because we don’t want to, but because the physics of the way our society and AI technology interface wont allow it. Only by leaning into AI research which promotes interpretability can we solve the problem of AI control.
This post is bothering an itch I’ve been having for over a year.
The entire problem of AI control stems from how we treat LLMs. We look at them as the AI. Maybe a small shell of code for letting it interface with some computer system, but the LLM for all intents and purposes is what we call ‘AI’. We draw an equivalence between the two and try to solve the problem of controlling super intelligence under a constraint.
Correct me if I’m wrong but I feel that this post and paper are jabbing in a similar direction. We need a cognitive model for AI but if we work under constraint this implies we need to develop a model of the model. If we break the constraint then we can build the cognitive model from first principles, utilizing the LLM as a language processing subsystem rather than as the entire AI.
The field even seems to be crawling towards this idea. Techniques like Inference Time Intervention(https://arxiv.org/abs/2306.03341), and FUDGE(https://aclanthology.org/2021.naacl-main.276.pdf) work at runtime. The bridge between token generation modification at runtime and a feedback system with known failure signals like entropy (https://arxiv.org/pdf/2510.21623,https://arxiv.org/html/2512.12381v1) seems a lot smaller than the current hurdle of controlling LLMs as they are.
Sorry, I was making an argument from analogy for what I now realize was for no apparent reason. I saw you mention the anthropic principle and got overexcited.
I agree that before the development of cheap widespread compute such a conclusion on the likely simplicity of the generating rules of the universe would be in the reverse (again with the Deism).
Ah okay, I think we have different understandings of the motivation for axioms. If I understand correctly you are saying axioms should be motivated by intuition. I disagree. Axioms should be motivated for their generative capacity, i.e. how much fun math we can squeeze out of them. Axioms don’t need to be “true”, rather they need to be interestingly generative.
When I encounter math I don’t immediately have an intuition for, I remind myself what Von Neumann said:
“In mathematics you don’t understand things. You just get used to them.”
For me at least, I have a hard time grasping the intuition behind a lot of math so I cling to this quote like dogma.