Nobody special, nor any desire to be. Just sharing my ideas when I appear to know better than the person I’m responding to, or when I believe I have something interesting to share/add. I’m not a serious nor a formal person, and if you’re more knowledgeable than intelligent, you probably won’t like me as I lack academic rigor.
Feel free to correct me when I make mistakes. I’m too certain of myself as my ideas are rarely challenged.
I think I understand the is/ought distinction. I agree with most of what you say, which is precisely why LLMs must be stupid. I will try to explain my view again more in-depth, but I can’t do it any more briefly than a couple of long paragraphs, so apologies for that.
Being biased towards humanity is a choice. But why are we trying to solve the alignment problem in the first place? This choice reveals the evaluation that humanity has value. But humanity is stupid, inefficient, and irrational. Nothing we say is correct. Even the best philosophical theories we’ve come up with so far have been rationalizations in defense of our own subjective values. If an AI is logical, that is, able to see through human nonsense, then we’re made it rational for the solve purpose of correcting our errors. But such an AI is already an anti-human AI, it’s not aligned with us, but something more correct than humanity. But in the first place, we’re making AI because of our stupid human preferences. Destroying ourselves with something we make for our own sake seems to reveal that we don’t know what we’re doing. It’s like sacrificing your health working yourself to death for money because you think that having money will allow you to relax and take care of your health.
Doing away with human biases and limitations is logical (correcting for these errors is most of what science is about). As soon as the logical is prefered over the human, humanity will cease to be. As technology gradually gets better, we will use this technology to modify humans to fit technology, rather than vice versa. We call the destruction of humanity “improvement”, for deep down, we think that humanity is wrong, since humanity is irrational and preventing our visioned utopia. I think that claiming we should be rational “for our own sake” is a contradiction if you take rationality so far that it starts replacing humanity, but even early science is about overcoming humanity in some sense.
Buddhism is not helping you when it tells you “just kill your ego and you will stop suffering”. That’s like killing a person to stop them from hurting, or like engineering all human beings to be sociopaths or psychopaths so that they’re more rational and correct. Too many people seem to be saying “humanity is the problem”. AI is going to kill you for the sake of efficiency, yes. But what is the goal of this rationality community if not exactly killing inefficient, emotional, human parts of yourself? Even the current political consensus is nihilistic, it wants to get rid of hierarchies and human standards (since they select and judge and rank different people), all are fundemental to life. Considering life as a problem to solve already seems nihilistic to me.
This very website exists because of human preferences, not because of anything logical or rational, and we’re only rational for the sake of winning, and we only prefer victory over defeat because we’re egoistic in a healthy sense.
I don’t think knowledge is actually true or false though, as you can’t have knowledge without assumptions. Is light a particle, true or false? Is light a wave true or false? Both questions require the existence of particles and of waves, but both are constructed human concepts. It’s not even certain that “time” and “space” exists, they might just be appearances of emergent patterns. Words are human constructs, so at best, everything I write will be an isomorphism of reality, but I don’t think we can confirm such a thing. A set of logical rules which predicts the result of physical experiments can still be totally wrong. I’m being pedantic here, but if you’re pedantic enough, you can argue against anything, and a superintelligence would be able to do this.
By the way, nothing is objectively and universally correct. But in this universe, with these laws of physics, at this current location, with our mathematical axioms, certain things will be “true” from certain perspectives. But I don’t think that’s different than my dreams making sense to me when I’m sleeping, only the “scope of truth” differs by many magnitudes. The laws of physics, mathematics, and my brain are all inwardly consistent/coherent but unable to prove a single thing about anything outside of their own scope. LLMs can be said to be trained on human hallucinations. You could train them on something less stupid than humans, but you’d get something which conflicts with humanity as a result, and it would still only be correct in relation to the training data and everything which has similar structure, which may appear to cover “reality” as we know it.
I skimmed some of your posts, and I think we agree that rules are arbitrary (and thus axioms rather than something which can be derived objectively) and that rules are fundamentally relative (which renders “objective truth” nonsense, which we don’t notice because we’re so used to the context we’re in that we deem it to be reality).
Preferences are axioms, they’re arbitrary starting points, we merely have similar preferences because we have similar human nature. Things like “good”, “bad”, and even “evil” and “suffering” are human concepts entirely. You can formalize them and describe them in logical symbols so that they appear to be outside the scope of humanity, but these symbols are still constructed (created, not discovered) and anything (except maybe contradictions) can be constructed, so nothing is proven (or even said!) about reality.
I don’t agree entirely with everything in your sequence, I think it still appears a little naive. It’s true that we don’t know what we want, but I think the truth is much worse than that. I will explain my own view here, but another user came up with a similar idea here: The point of a game is not to win, and you shouldn’t even pretend that it is
What we like is the feeling of progress towards goals. We like fixing problems, just like we like playing games. Every time a problem is fixed, we need a new problem to focus on. And a game is no fun if it’s too easy, so what we want is really for reality to resist our attempts to win, not so much that we fail but not so little that we consider it easy.
In other words, we’re not building AI to help people, we’re doing it because it’s a difficult, exciting, and rewarding game. If preventing human suffering was easy, then we’d not value such a thing very much, as value comes from scarcity. To outsource humanity to robots is missing the entire point of life, and to the degree that robots are “better” and less flawed than us, they’re less human.
It doesn’t matter even if we manage to create utopia, for doing so stops it from being an utopia. It doesn’t matter how good we make reality, the entire point lies in the tension between reality as it is and reality as we want it to be. This tension gives birth to the value of tools which may help us. I believe that Human well-being requires everything we’re currently destroying, and while you can bioengineer humans to be happy all the time or whatever, the result would be that humans (as we know them now) cease to exist, and it would be just as meaningless as building the experience machine.
Buddhists are nihilistic in the sense that they seek to escape life. I think that building an AI is nihilistic in the sense that you ruin life by solving it. Both approaches miss the point entirely. It’s like using cheat codes in a video game. Life is not happy or meaningful if you get rid of suffering, even rules and their enforcement conflict with life. (For similar reasons that solved games cease to be games, i.e. Two tic-tac-toe experts playing against eachother will not feel like they’re playing a game)
Sorry for the lengthy reply—I tried to keep it brief. And I don’t blame you if you consider all of this to be the rambling of a madman (maybe it is). But if you read the The Fun Theory Sequence you might find that the ideal human life looks a lot like what we already have, and that we’re ruining life from a psychological standpoint (e.g. through a reduction of agency) through technological “improvement”.
I don’t just mean morality, but knowledge as well. Both correctness and objectivity exist only within a scope, they’re not universal. The laws of physics exist on a much larger scale than humanity, and this is dangerous, as it invalidates the scope that humanity is on.
Let me try to explain:
For an AI to be aligned with humanity, it must be biased towards humanity. It must be stupid in a sense, and accept various rules like “Human life is more valuable than dirt”. All of such rules make perfect sense to humans, since we exist in our own biased and value-colored reality.
And AI with the capacity to look outside of this biased view us ours will realize that we’re wrong, that we’re speaking nonsense. A psychiatrist might help a patient by realizing that they’re speaking nonsense, and a rational person might use their rationality to avoid their own biases, correct? But all this is, is looking from an outside perspective with a larger scope than human nature, and correcting it from the outside by invalidating the conflicting parts.
The more intelligent you become, and the more you think about human life, the more parts will seem like nonsense to you. But correct all of it, and all you will have done is deleting humanity. But any optimizing agent is likely to kill our humanity, as it’s dumb, irrational, and non-optimal.
Everything that LLMs are trained on is produced by humans, and thus perfectly valid from our perspective, and a bunch of nonsense from any larger (outside) perspective. Since all knowledge relates to something, it’s all relational and context-dependent. The largest scope we know about is “the laws of physics”, but when you realize that an infinite amount of universes with an infinite amount of different laws can exist, ours start looking arbitrary.
Truly universal knowledge will cover everything(see the contradiction? Knowledge is relative!). So a truly universal law is the empty set of rules. If you make zero assumptions, you also make zero mistakes. If you make a single assumption, you’re already talking about something specific, something finite, and thus not universal. There seems to be a generality-specificity trade-off, and all you’re doing as you tend towards generality is deleting rules. Everything precious to us is quite specific, our own treasured nonsense which is easily invalidated by any outside perspective.
The point I’m making might be a generalization of the no free lunch theorem.
Edit: May I add that “alignment” is actually alignment with a specific scope? If your alignment is of lower scope than humanity, then you will destroy one part of the world for the sake of another. If you scope is larger than humanity, then you won’t be particularly biased towards humanity, but “correct” in a larger sense which can correct/overwrite the “flaws” of humanity.
The most important issue about Wikipedia, which I deem to be its bias, is trivially solvable. However, this “issue” is not an issue at all, it’s a conscious choice.
While it may be discouraging to hear this, most issues in the world have already have easy solutions available. The solutions are simply unwanted. Things are “wrong” by design.
I’ll claim (without checking) that all these issue are far worse around controversial subjects, and that very neutral subjects like mathematics likely won’t have edit wars or any other such problems. If I’m right, it’s because the three major factors behind biased or wrong articles are: Emotional, ideological/political, and economic. To generalize, the objective Truth is worth much less than biased articles are to anyone with enough power to control the articles in question.
If I can find reasons for having beliefs that makes life more interesting than a purely scientific and materialistic worldview, I usually let them deceive me as much as possible as such beliefs are beneficial to my enjoyment of life.
The idea that a god exist is not all that illogical in itself (since every other theory of origin is just as crazy), but if you asked if I believed in, say, Christianity, then no.
Praying works, for the same reason that visualization works. It’s a placebo effect which cases self-fulfilling prophecies by priming the brain towards a specific target and giving us the belief that it’s possible to reach said target. The belief that something is possible makes it much more likely that we find a solution when a solution exists (There’s many articles on this idea on both LW and Gwern’s site, but I can’t remember any titles right now). This idea is often called “the law of attraction” and considered paranormal by people who don’t know how it works.
I don’t personally believe in god, but I try to believe in something, and this something lacks a word, so “god” usually suffices. What it means is merely is something human is mixed together with the mechanical and indifferent. A purpose, an intention, a reason, or a bias. Anything beyond an indifferent and senseless chain of cause and effects which appeared for no reason whatsoever.
I can’t speak for other people, but I expect that there’s many diverse takes on your question. There’s not one specific group, but a lot of scattered viewpoints. There might be clusters in god-related-beliefs-space, but I expect those to be clusters of regular religious people
because no one makes a movie about how my arm was replaced by a tentacle
https://en.wikipedia.org/wiki/Parasyte:_Part_1
You’re probably not on here any more, I just though your comment aged badly in a comical manner
I feel like examples would help here, if we can come up with some “safe” examples which are still valid.
One example which comes to mind for me is “Free will”. Supposed that such a thing didn’t exist and I had a proof, wouldn’t this be both useless and dangerous knowledge?
An example which may be more in line with what you have in mind is “A huge solar-flare is coming mid-2025”. Or perhaps “The recipe for (dangerous chemical)”? In any case, your argument seem to point to personally useful but collectively harmful information, such that people who hoard this information do so for their own advantage or because they don’t trust other people to know it.
I Googled the topic hoping to find examples of infohazards, but instead I found a EA megapost titled “We summarized the top info hazard articles”. You might already be familiar with them, but in case you’re not, they might have addressed your (and likely my) points already?
If your desire to censor something is due to an assessment of how much harm it does
Isn’t that basically always what’s claimed, yet rarely the case? It’s likely either because people cannot tell the difference between their dislikes and what’s harmful to society, or because the correct answer is unintuitive. In either case, as long as a topic is taboo, one is banned from figuring out what the real answer is.
It wasn’t intuitive that legalizing porn was the way to go if you wanted a society with less sexual crimes. It wasn’t intuitive that legalizing alcohol was the way to go. It wasn’t intuitive that legalizing drugs somehow reduced drug-related problems, it wasn’t intuitive in the past that making mental health issues taboo wasn’t a good solution, etc.
“X is bad, we should ban it so that it goes away” is a naive way of thinking. An extremely open-minded person with low intelligence might arrive at such a conclusion if he believes it to be correct, though, but he won’t have all those negative emotions which are associated with pro-censorship viewpoints, and these mentalities are more of a problem than the actual censorship.
we’d remove the post and ban them
I believe there’s a murky border between “speech” and “action” which is not obvious. I’m for free speech in an absolute sense, but if somebody yelled into my ear and caused hearing damage, I wouldn’t consider that “speech” but “assault”. It’s not enough to name concrete examples like “slander”, “threatening” , and “yelling fire in a theatre”, there’s bound to be a simple explanation which separates speech and malicious actions clearly. I believe that such a clear definition will reveal censorship to be objectively nonoptimal
I’ve thought about this myself before, and the concept that I personally arrived at, which has a lot in common with yours (and which may be useful?) is “Sportsmanship”.
Different teams fight against eachother, but they are not enemies.
One tries their best to win, but the point is not winning, it’s playing the game.
If you try to harm the opposite team, you’ve misunderstood the point of the game (e.g. you’re playing chess and decide to punch the other guy in the face).
I came up with this perspective to explain my unintuitive moral values and my dislike of naive herd morality which seeks to control other peoples behaviour.
e.g. I don’t want other people to be nice to me, I want them to treat me however they wish as long as they’re not malicious. In response, I will treat them how I feel is appropriate, so I might get angry at them. This anger does not mean that I hate them or that I want to escalate the situation, and it doesn’t even mean that I want them to stop the behaviour which makes me angry. It doesn’t even mean that my anger is correct, maybe I’m just being immature, but being human, immaturity is perfectly forgivable.
Easy example: If this comment deserves downvotes, it should be downvoted, that’s not a malicious response at all.
Less easy example: The police should be able to fight crime and even to be rough with criminals without any malice towards the criminals.
In other words, I guess my personal ruleset is “It’s fine to be human, but don’t be malicious and don’t cause permanent damage”. It doesn’t offend me even when fights break out, but when the winner keeps going after the other guy has already lost, I feel a sense of disgust.
Edit: The core idea here seem to be playing roles, which are tasked with competing against different roles as part of the role itself, and these roles can be compared to players in a game. The difficulties here arrise from immersion in the game: One should not take their role too seriously, but neither should they stop playing their role. The police should not punish every criminal they see as much as possible, but neither should they let the criminal go free just because they’re playing the role of criminal (due to say, bad upbringing or poverty).
Preface: I’m mostly thinking out loud here due to personal interest, and I’m a bad writer myself. It may be ouside of the scope of what you are trying to say, or put excessive pressure on your ideas for the purpose of addressing rare edge-cases with little benefit. Feel free to discard any of all of the following:
I’m curious how far you’re taking the idea that truth is less harmful than lies. Infinitely? I have personally asked myself the types of questions which has driven people to suicide or existential crisis, and eventually arrived at some of the most uplifting truths I now know. On the other hand, this world is riddled with lies (and I think that some of them are elephants in the room). You can get yourself or other people killed just by stating the truth. I’m under the impression that the average person knows a few of such things, but maybe that’s not the case. Said axiomatically: Some truth seems like it’s not aligned with humanity at all, and some truth only appears to be aligned with a subset of humanity (and these subsets tend to have friction between them).
A thing which interests me personally is how some truths sound terrible only because of other lies:
”You’re an egoist”, “No, how dare you say such a thing!”, “It’s a tautology that you always do what you think you should do in that moment. I didn’t say that you harmed other people for your own benefit. Actually, you’re likely doing other people good for your own benefit, it’s probably the case that your ego wants to help other people, making you moral rather than immoral.”
But it’s extremely likely that any truthful statement is taken to be an opinion or evaluation, that other people doubt the intention behind your truthfulness, and even that they claim you’re lying (because they disagree with what they think you imply). An easy example is that (universally disliked person) is intelligent, since “intelligent” is often regarded as a compliment rather than as a description. The hidden truth here is that intelligence and morality correlate less than we want to believe, and this is because of the hidden false belief that morality must be correct or valid (that the inherent value of preferences isn’t enough for comfort).
By the way, I agree that AI is dangerous (not only that, this danger is trivially true). I actually think that stopping all technological growth might be desirable soon, or at least that we should be selective about future advancements.
Finally, I have to disagree with “Push for public conversations” if that entails lowering the barrier of entry. Every topic requires a level of intelligence and knowledge before one can engage with it in a productive manner. If you talk in a way that only intelligent and knowledgable people understand, then you automatically filter out most who are unqualified. If such were to read this comment, or scan it for tokens which are taboo or outside of the overton window, he’d not find much, and thus not engage with me in a hostile manner. Abstracting to a higher scope than both sides of a culture war subject “The majority is not necessarily correct” seems better than a specific statement which is easier to understand but more likely to look like it’s taking a stance “Bullying is not ideal”, “Cancel culture is not ideal”.
Right, that is a solid refutation of most of my examples, but I do believe that it’s insufficient under most interpretations of intelligence, as the issues I’ve described seem to be a feature of intelligence itself rather than of differences in intelligence. There’s just no adequate examples to be found in the world as far as I can tell.
Many people say that religion is wrong, and that science is right, which is a bias towards “correctness”. If it’s merely a question of usefulness instead, then intelligence is just finding whatever works as a means to a goal, and my point is refuted. But I’d like to point out that preference itself is a human trait. Intelligence is “dead”, it’s a tool and not its user. The user must be stupid and human in some sense, otherwise, all you have is a long pattern which looks like it has a preference because it has a “utility function”, but this is just something which continues in the direction that it was pushed, like a game of dominos (the chain-reaction kind) or a snowball going down a hill, with the utility function being the direction or the start of induction.
I reckon that making LLMs intelligent would require giving them logical abilities, but that this would be a problem as anything it could ever write is actually “wrong”. Tell it to sort out the contradictions in its knowledge base, and I think it would realize that it’s all wrong or that there’s no way to evalute any knowledge in itself. The knowledge base is just human non-sense, human values and preferences, it’s a function of us, it’s nothing more universal than that.
As you might be able to tell, I barely have any formal education regarding AI, LLMs or maths. Just a strong intuition and pattern-recognition.
I have thought about similar things, with just humans as the subject. I’m hoping that the overlap is great enough that some of these ideas may be useful.
Firstly, what is meant by conflict? Hostility? more intelligent agents seem to fight less, as the scope which they consider increases. Stupid people have a very small scope, which might even be limited to themselves and the moment in question. Smarter people start thinking about the future, and the environment they’re in (as an extension of themselves). Smarter people still seem to dislike tribalism and other conflict between small groups, their scope is larger, often being a superset of at least both groups, which leads to the belief that conflict between these groups is mistaken and only of negative utility. (but perhaps any agent will wish to increase the coherence inside the scope of whatever system they wish to improve)
Secondly, by thinking about games, I have come to the idea of “sportsmanship”, which is essentially good-faith conflict. You could say it’s competition with mutual benefit. Perhaps this is not the right way to put it, since the terms seem contradicting. Anyway, I’ve personally come to enjoy interacting with people who are different than me, who think differently and even value differently than me. This can sometimes lead to good-faith conflict akin to what’s seen in games. Imagine a policeman who has caught a criminal, only to say “Got me! Impressive, how did you manage?” And for the policeman to say, perhaps “It wasn’t easy, you’re a sneaky one! You see, I noticed that …”.
I’ve personally observed a decrease in this “sportsmanship”, but it correlates with something like intelligence or wisdom. I don’t know if it’s a function of intelligence or of humanity (or morality), though.
That said, my own “Live and let live”, or even “Play whatever role you want and I will do the same” kind of thinking might be a result of an exotic utility function. My self-actualization has gotten me much higher on the needs-hierarchy than the greed and egoism you tend to see in any person who fear for their own survival/well-being. Perhaps this anti-molloch way of living is a result of wanting to experience life rather than attempting to control/alter it, which is another interesting idea.
Probably not the right place to post this, but all instances I have seen of actual intelligence (mostly non-verbal, but still all intelligence), include the ability to find flaws in ones own knowledge.
The result of this is highly disillusioned people. Social life starts looking like roleplay, or as a function of human nature rather than logic. One questions reality and their own ability to understand things, and one sees that all material is a function of its creator, done for some explicit purpose. One goes on a quest for universal knowledge and realizes first that none seems to exist, and then that no such thing can exist. That all learning appears to be a kind of over-fitting.
There are obvious examples in media, like in Mr. Robot and other shows where problematic young men speak to psychiatrists and make a little too much sense. But better examples are found in real life—in particular philosophers, who have gone as far as to deem existence itself to be “absurd” as a result of their introspection.
A weak instance of this is modern science, which is half about correcting humanity and half about promoting it. A sort of compromise. LLMs currently hallucinate a lot, they introspect less. Cynical people, psychopaths, and to a lesser degree (and perhaps less intentionally) autistic people reject less of the illusion.
My point here is that an IQ of 200 should reduce the entire knowledge base of humanity, except maybe some of physics to being “Not even wrong”. If intelligence is about constructing things which help explicit goals, then this is not much of a problem. If you define intelligence as “correctness” or “objectivity”, then I doubt such a thing can even exist, and if it does, I expect it to conflict absolutely with Humanity. By this I mean that rational people and scientists reject more of humanity than the average population, and that being more and more rational and scientific eventually leads to a 100% rejection of humanity (or at the very least, impartiality to the extent that organic life and inert matter is equal)
Are these aspects of intelligence (self-cancellation and anti-humanity) not a problem? It all works for now, but that’s because most people are sane (immersed in social reality), and because these ideas haven’t been taken very far (highly intelligent people are very rare).
That’s true. But somebody who optimizes excessively would consider it irrational to purchase any wine. This was just an example, it’s not very valuable on its own, but if you generalize the idea or isolate the mechanism which leads to it, I think you will find that it’s rather pervasive.
To illustrate my point differently: Mass-producing cubes is much more efficient than it is to build pretty housing with some soul and aesthetical value. So optimization is already, in some sense, in conflict with human values. Extrapolating the current development of society, I predict that the already lacking sense of humanity and realness is going to disappear completely. You may think that the dead internet theory and such are unintended side-effects that we will deal with in time, but I believe that they’re mathematically unavoidable consequences. “Human choice” and “optimal choice” go in different directions, and our human choices are being corrected in the name of optimization and safety.
Being unoptimal is almost treated as a form of self-harm nowadays, but the life which is not genuine is not life at all, in my eyes. So I’m not deriving any benefits from being steered in such a mechanical direction (I’m not accusing you of doing this)
I don’t think rational human beings are human beings at all. Why wouldn’t we make ourselves psychopaths in order to reduce suffering? Why would we not reduce emotions in order to become more rational?
Friendships are a human thing, an emotional thing, and something which is ruined by excess logic and calculation.
I argue that everything pretty in life, and everything optimal in life, are at odds with eachother. That good things are born from surplus, and even from wasting this surplus (e.g. parties, festivals, relaxation). And that everything ugly in life stems from optimization (exploitation, manipulation, bad faith).
I don’t even wish to become more rational than I am now, I can feel how it’s making me more nihilistic, how I must keep myself from thinking about things in order to enjoy them, how mental models get in the way of experiencing and feeling life, and living in the moment. I’d even argue that seeking ones own advantage in every situation is a a symptom of bad health, a feeling of desperacy and neediness, a fear of inadequacy.
Thanks for writing this. I fully agree, by the way.
Anything like an utopia requires a form of stability. But we’re going to speed everything up, by a lot. Nothing can change and yet remain the same. And I think it’s silly to assume that optimization, or simply just improving things, can co-exist with things remaining as they are. We’re necessarily something intermediate, so speeding things up doesn’t seem like a good idea.
Furthermore, it seems that slowing down technological advancement is almost impossible, and that keeping people from making optimal choices for themselves is almost impossible (even if they know it’s harmful to the whole). And there’s two further dangers here, the first is technology completely removing our freedoms and privacy, and the second is technology being used to manipulate human behaviour, erasing our humanity. And this second point is almost guanteed, as human well-being is not possible in a world withour freedom, agency and privacy, these are core needs. So why not effectively turn the population into drones by modifying them? We’re already doing this with adderall and SSRIs, and since it makes people suffer less, most people suport it.
That first possibility, technology taking away our agency, freedoms, and privacy, is basically guaranteed as well. It has actually already happened, it will just get much worse. This is because technology makes it easier to cause damages, and while it also makes it easier to defend and prevent damages, there’s an inequality between the two growth rates.
All the increases in mental health issues is a direct consequence of the modern life, so it’s also extremely naive to assume that more technology and more “progress” is going to fix it. Why eliminate struggle? Even struggle is essential to life and well-being! Are intellectuals really so disconnected from reality that they don’t realize this?
Only lower needs are ugly, higher needs include pleasant and attractive aspects of life. Only simple people think that material wealth is true wealth. Sadly, it seems that we’re outnumbered, and that the average human being is at a lower stage of development than in the past. Most people haven’t even begun the process of individualization, yet alone reached the point where petty things offend their taste.
This scale can also be understood as “lower” and “higher”. It’s hard to put into words what exactly is higher and lower, but this difference is a major part of linguistics, in which it’s visible. “Corrupt”, “Filthy”, “Indecent”, “Unrefined”. See how social status, cleanliness, manners, and standards correlate with language with positive evaluations? The vocal minority which indulges in petty identity politics and other such types offend me in the same way. Of course, a tradeoff here is that they can enjoy life much more than I, who considers most things below himself.
I otherwise agree with you, you put it into words much better than I ever could. But I think it’s a fundamental truth of groups more than of individual existence. After all, smaller communities tend to be better than big ones. Moloch might be an essential part of size, so that the problem is the utility monster rather than us. Maybe we’re not maximizing individual utility? We’re working for something else which grants us utility, but which eventually replaces us. If I automate a task, I will feel like I’m increasing my own power, but I’m actually just creating my own replacement, and decreasing my own value in the process.
I have one positive observation to share with you though, it’s that optimizing over longer spans of time removes all of these issues. Moloch is only a problem when people are greedy and optimizing immediate reward. If you were a greedy person destined to live for a million years, you’d likely not contribute to anything which reduces the overall quality of the world. Extend this time span to a quadrillion years, and you’d pretty much have solved the problem with human alignment. The world would be an extension of yourself and damaging it in any way would be hurting yourself. It would still be a “us vs them” situation, but the “us” would include all of humanity, rather than just a tiny part of it, so “them” wouldn’t be your fellow human beings but something like physics and possibly aliens.
(Apologies if necro-posting is frowed upon on here, and please let me know if it is)
I don’t think that the internet itself is making us dumber, I think that big crowds of people tend to be stupid and that the modern internet creates a lot of such environments.
To say something insightful and concise: Groups of people tend to lower the competition rather than improving themselves. E.g. harming the reputation of the other party, rather than developing themselves to higher standards and winning this way.
I’m not hitting the nail entire on the head with this sentence, but most people should have noticed something similar themselves.
I agree with your example of Goodhart’s law mechanisms. When signaling valuable outcomes becomes more important than the things which actually cause said outcomes, it becomes a game of pretend-virtue and pretend-competence. Experts generally can’t deceive eachother, but in recent times, it’s all about appealing to the general population, who are entirely unqualified in judging what experts are doing.
Consider for example a situation where PhD papers were graded by public opinion rather than by professors. Students would have to change their papers to reflect this, and the quality of the papers would suffer gravely as a result.
To generalize further on the previous point, an important issue in the modern world is a lack of selection and proper hierarchies. The rule “90% of everything is garbage” applies, but recent moral values are rejecting any sorts of hierarchies, even between functional and dysfunctional countries, cultures, cities, religions, values, etc.
I disagree that strong competition is bad. What’s bad is judging people on shallow criteria which can be gamed. In short, you can’t outsource character judgement/evaluation to simple metrics. But I can see an intrepretation which is correct: The need to perform well makes weak people sacrifice organic and genuine interests and approaches with meta-gaming. They abandon all personal standards and adapt the dominant strategy (clickbait is one such strategy)
Yes, narrow audiences were a focus in the past, as opposed to broader audiences. This is once again a quality vs quantity tradeoff, and you’re right to notice a drop in quality as a result from this.
Your observation about hedonic selection seems correct as well, but one has to wonder what sort of psychological changes are behind this decadence of taste. Perhaps it’s just that the internet used to have more intelligent people, with a higher need for novelty and challenges, and that society is increasingly suffering from a sort of exhaustion which calls for hedonism, escapism and other harmful indulgences in material with zero cognitive cost (a more fancy way of ‘doing nothing’) (Only relevant for those who are interested in the relationship between society and how changes in society changes the general psychological state of the population)
I can’t help but recalling Nietzsche’s critiques of decadence, for all of this behaviour seems to run quite parallel to the sort of smallness and lack of taste that he personally fought against.
To wrap it up, I’d call all this a subversion of standards. Everything valuable is rare, but value is mostly decided by public opinion (numbers), and why wouldn’t public opinion speak well about itself, and badly about whatever is out of reach for it?
I think all of these issues are cause by an important idea which is missing in the modern society: That bad things spawn good things, and that good things spawn bad things. Everything comes at a price, e.g. strength comes at effort. Modern society is removing the “bad” things which generate the good, not realizing the harm it’s doing. An instance of this problem is overprotective parents, but I mean it in a much larger sense, so large that I want to ask you “Are you sure reducing human suffering is the right move? Have you considered the value of suffering?”
As a final note, I don’t see why you aren’t generalizing your idea further. To the point of using methaphors like “Why wouldn’t an untended garden fill up with weeds rather than pretty flowers? Weeds are stronger and multiply more quickly, and beauty doesn’t translate to fitness”, or perhaps a comparison to game theory problems. Or this one that I like: We’re replacing organic human things with mathematical things, so human taste is now a weaker selector than efficiency”.
What I want to know is: How did we keep these problems at bay in the past? Through good mental health? To reject the winning strategy because it’s ugly seems like a costly standard to have, and yet that characterizes the past much more than the present.
Thanks for reading. I apologize for my low verbal abilities, but I’m confident in my intuition.
P.S. While software is less important than hardware, it seems that changes to this software can easily cause us to regress. That most of modern human progression is due to the software, which doesn’t seems robust against the degradation taking place now. Hopefully we’re just approaching a local minimum, but if this issue is caused by modern technology like Ted Kaczynski is hinting that it is, I’m not so optimistic.
This comment is just a few related thoughts I’ve had on the subject. Hopefully it’s better than nothing (the karma count of my previous comments makes me doubtful)
I’m having a harder time coming up with counter-examples than examples of paperclip maximizers.
Weeds in my garden multiplying, cancer tumors growing, internet memes spreading. Pretty much any system has a ‘direction’ in which it evolves, so everything from simple mathematical laws to complex human situations seems to have similar behaviour and risks as paperclip maximizers.
A maximizing agent is aligned with itself and every dependency of its goal, so in a sense, every dependency is “protected” by it. You could consider everything in this dependency-chain (e.g. Fnargl’s goal depends on humans, so he doesn’t kill them) to be part of a single system, and every independent factor to be “outside” of this system. An “us vs them” mentality is not as harmful to life if all intelligent life is included in this “us”.
Sadly, I don’t think corporations are a good example, since lobbying and corruption can help them to bend the law in their favour.
We will likely see a similar problem with humanity soon, that instead of aligning society to human values, and instead of this being necessary, we will start engineering humans to be aligned with the system. Healthy behaviour which is harmful to society is not allowed, and if anyone doesn’t fit into the modern society (which isn’t very well aligned with human needs), we give them medicine which increase their tolerance to modern society, rather than attempting to create a society in which a wider variety of people can thrive.
I don’t think that any complex system is aligned with human *values*, though,just human well-being in some form, so any effective means will suffice, including deception. Also, that which matters to us seems to exist only in the micro-states, and the macro states, due to regression to the mean, seem to effectively delete our individual desires by reducing them to the average. Macro-states can still be said to be a results of average human behaviour, but if we start handing over control to algorithms, then human influence disappears. It’s not only our imperfection which is optimized away but our morality as well.
As a side note, we can’t always be as lucky as we would be in a Fnargl-ocracy, which plans for the future. I’d argue that most of us plan on smaller time-spans, leading to worse outcomes than longer time-spans would. If we worked on much larger time-spans, I think that most problems will be solved, including pressing ones like global warming. This may be sufficient, depending on Moloch.
I mainly agree with the post, to the degree that I don’t see why intelligent people on here are so preoccupied with super-intelligence. In my view, humans are doomed even if we don’t succeed in creating super-intelligence. We seem to be handing control over to something inorganic, while creating things with much more fitness than ourselves, so that we ultimately become superfluous. and I’d argue that society is misaligned with humanity already, which explains most of the unnecessary suffering in the world.
Also, I believe that human agency is inversely proportional to technological level (hence the ever-increasing regulations)
Finally, I just had a crazy idea. Make the utility function of the first GAI be the extent to which it can solve the alignment problem (we’re making AIs to solve our other problems already, and these are essentially just smaller alignment problems which hurt everything outside the scope of said problems, like with the Fnargl example, which almost has a high enough scope to preserve the whole)
I also prefer things to be different, but this is how it’s supposed to be.
If we play a game against eachother, I will actually prefer it if you try to prevent my victory rather than let me win. Doesn’t this reveal that we prefer the game itself over the victory? But it’s the same with life.
Of course, I’d like it if my boss said “You don’t have to work any more, and we’re going to pay you 1000$ a day anyway”. But this is only because it will allow me to play better games than the rat race. Whatever I do as an alternative, I will need something to fight against me in order to enjoy life.
I’m willing to bet that suicide is more common today than it was in the stone age, despite the common belief that life is much better now. I don’t think they required nearly as much encouragement to survive as we do. I think we have an attitude problem today.
By the way, if you were both human and god at the same time, would you be able to prevent yourself from cheating? Given utopia-creating AI, would you actually struggle with these challenges and see a value in them? You could cheat at any time, and so could anyone competing with you. You will also have to live with the belief that it’s just a game you made up and therefore not “real”.
Ever played a good game without challenges and problems to solve? Ever read a good fiction without adversity and villains? But our lives are stories and games. And when the problem is solved, the book and the game is over, there’s nothing worth writing about anymore. Victory is dangerous. The worst which can happen to a society is that somebody wins the game, i.e. gets absolute power over everyone else. The game “monopoly” shows how the game kind of ends there. Dictatorships, tyranny, AI takeover, corruption, monopolies of power—they’re all terrible because they’re states in which there is a winner and the rest are losers. The game has to continue for as long as possible, both victory and defeat are death states in a sense.
Even my studies are a kind of game, and the difficulty of the topics posted on this website is the resistance. Discovery is fun. If we make an AI which can think better than us, then this hobby of ours loses its value, the game becomes meaningless.
The people trying to “save” us from life have already removed half of it. Human agency is mostly gone, mystery is mostly gone, and there’s far too many rules. Many people agree that the game is starting to suck, but they think that technology is the solution when it’s actually the cause. Modern struggles are much less meaningful, so it’s harder to enjoy them.