If we go by analogy with Godel, Turing, and basically anything else involving self-reflection, one has to be pessimistic and place a higher probability on a proper fully consistent RDT simply being impossible, which could actually have some rather dramatic consequences downstream for e.g. AI alignment.
dr_s
Karma: 1,232
However, such a person would still have to admit that the mutual information between input and output bits is high. There is something inherently “computery” about that circuit even if you refuse its specific interpretation. It just computes a different function.
there is a correct non-power-maximizing solution
Questionable—turning the universe into paperclips really is the optimal solution to the “make as many paperclips as possible” problem. But yeah, obviously in Civ IV taking over the world isn’t even an instrumental goal—it’s just the actual goal.
Ok, sorry, I phrased that wrong—I know the scenario you described isn’t a utility monster one, but it can be turned into one simply by running up the knob of how much the king enjoys himself, all while being just as unfair, so it’s not really like total utility captures the thing you feel is actually wrong here, is my point. I actually did write something more on this (though in a humorous tone) in this post.
I don’t mean that “it’s subjective” fixes everything. I just mean that it’s also the reason why it’s not entirely right IMO to write off an entire universe based on total utility. Like, my intuition is that if we had a universe with net total negative utility it still wouldn’t be right to just snap our fingers with the Infinity Gauntlet and make it disappears if it wasn’t the case that every single sentient in it, individually, was genuinely miserable to the point of wanting to die but being unable to.
Regarding the second part, I’m not against rules or limits or even against suffering.
The reason why I bring up rules and limits is more to stress how much our morality—the same by which we judge the wrongness of the universe—is borne of that universe’s own internal logic. For example, if we see someone drowning, we think it’s right to help them because we know that drowning is a thing that can happen to you without your consent (and because we estimate that ultimately on expectation it’s more likely that you wish to live than to die). I don’t mean that we can’t judge the flaws of the universe, but that our moral instincts are probably a better guide to what it takes to improve this universe, from inside (since they were shaped by it) than to what it would take to create a better one from scratch.
Video games are a good example; they have rules and limitations and loss conditions, but they are engineered with the player in mind and for his benefit, while in life, conscious beings are not promised interesting or fair experiences and might be just randomly tortured.
True, but also, with the same power as a programmer over a game, you could as well engineer a game to be positively torturous to its players. Purposefully frustrating and unfair. As things are, I think our universe is just not intelligently designed—neither to make us happy nor to make us miserable. Absent intent, this is what indifference looks like; and I think the asymmetry (where even indifference seems to result more in suffering than pleasure) is just a natural statistical outcome of how enjoyable states are just rarer and more specific than neutral or painful ones.
I don’t like total sum utility because it’s vulnerable to lots of hacks—your “reverse Omelas” is essentially a utility monster scenario, and in fact is exactly vulnerable to this because if you make the powerful king happy enough it says that the situation is good and should not be changed.
But also, I think morals make more sense as a guide towards how should we strive to change the world we’re in—within the allowances of its own rules of self-consistency—than how to judge the world itself. We don’t know precisely why the world works the way it does. Maybe it really couldn’t work any other way. But even if there was a happier universe possible, none of us can just teleport themselves to it and exist in it, as its laws would probably be incompatible with our life. If there were no rules or limits, ethics would be easy: you could always achieve that everyone be happy all the time. It’s because there are rules and limits that asking questions like “should I do X or Y? Which is better?” make sense and are necessary. As things are, since a net positive life seems possible in this universe, we don’t really have a reason to think that such a thing can’t be simply made available to more people, and ideally to everyone.
I think the point was that if you assume that the space of possible universes is vast and completely random, it’s already an infinitesimal fraction of them that will allow life to exist at all, never mind intelligent life. And depending on how you define that space, you could imagine universes more “intelligently designed” than ours towards being good, but also universes more designed towards being bad: actively malevolent and hell-bent on keeping alive sentient beings and still inflicting suffering, not just being indifferent. So in that sense, in expectation, our universe might actually be pretty middling. It’s honestly hard to say without a bigger frame of reference.
I mean, it’s one thing to say that it’s a shit hand to be dealt (fair), it’s another to say that we should expect it of the universe. We should expect it of a God if there was one—but absent that, blind unintelligent chaos can hardly have any expectations placed on it at all. It doesn’t do things for reasons.
more importantly, hurt the interest of those who prefer to pretend that life is good and the world is just for their own selfish reasons.
Isn’t that sort of contradictory? If there are people who have selfish reasons to act like life is good in general, obviously their life at least must be good enough for them to be satisfied. That makes the whole thing subjective, unless you take a very naive total sum utility approach.
Not like any of us has a “destroy universe and end all suffering” button ready to press and just refuses to anyway.
I think.
Most universes are hostile to life, and at most would develop something like prokaryotes.
I mean, by anthropic principle, we couldn’t ever be born in such universes. And I think this only applies if we consider various tunings of our known laws of physics. At the two extremes of the possibility spectrum though there is:
-
the constants aren’t really constants but emergent phenomena themselves, and thus this is actually the only universe possible
-
the laws aren’t in any way constant, the general symmetries and principles aren’t constant, there’s a whole Tegmark Type IV multiverse out there to explore
In the former case, then I guess there’s not much to complain about. In the latter, there surely are an infinity of universes better than this one (though also many much worse).
-
I think from this perspective there are two fundamental things wrong with the Universe:
consciousness is an emergent, not fundamental, property (as far as we know; panpsychists would object to this), and can only exist within subsystems with specific order qualities;
the second law of thermodynamics means that order-wise, most interactions in the universe are negative sum.
So 1 means that death is a thing, and 2 means that death is in fact sort of the default, and that scarcity and all sorts of reasons to fight are a powerful driver of anything alive, which needs to preserve its negentropy to stay so. A universe that didn’t have these things would not be necessarily free from bad things, but you could at least say that said bad things wouldn’t be life-ending, and that they would generally be the fault of someone; that choices could be made to mitigate them, but if they happen it’s because someone chooses so. That said, I’m not sure what such a universe would look like—probably merely a pure network of consciousness nodes (“individuals”) interacting with each other freely with no space or time. Way too alien to imagine properly.
This kind of writing makes me uncomfortable in a way I can’t put into words, like the feeling one gets when they look at a liminal photograph.
I think it’s a fair feeling. There’s a certain very famous (at least in our country) Italian 19th century novel in which at a point a priest sets out to bamboozle a peasant boy to get out of doing something he doesn’t want to. His solution is to begin piling up complexity and adding a healthy dose of Latin on top, so that the illiterate farmer remains obviously confused and frustrated, but can’t even quite put the finger on where he was cheated.
To put it bluntly: talking all difficult is a good way to get away with making stupid stuff sound smart and simple stuff sound complex. You don’t even necessarily do it on purpose, sometimes entire groups simply drift into doing it as a result of trying to up each other in trying to sound legitimate and serious (hello, academic writing). Jargon is useful to summarize complex concepts in simple expressions but you often don’t need that much and the more you overload your speech with it the less people will be able to get the entire thing. Even for people who do know the jargon, recalling what each term means isn’t always immediate. So, given how easy it is to overuse jargon to fool people or to position themselves above them, it’s not that strange that we sometimes develop a heuristic that makes us suspicious of what looks like too much of it.
With the two extracts you posted, the first one sounds to me like just another declination on the theme of “to stop [bad thing] we should all become GOOD” which is always a very “no shit Sherlock” thing. The second extract honestly I can’t quite tell what is precisely saying either, which is worrying in its own way.
So, yeah, +1 for just talking as simple as possible. Not any simpler, hopefully, but there’s rarely a risk of that.
By having a MD in Engineering and a Physics PhD, following the same exact courses you recommend as potentially containing the answer and in fact finding no direct answer to these specific questions in them.
You could argue “the answer can be derived from that knowledge” and sure, if it exists it probably can, but that’s why I’m asking. Lots of theories can be derived from other knowledge. Most of machine learning can be derived from a basic knowledge of Bayes’ theorem and multivariate calculus, but that doesn’t make any math undergrad a ML expert. I was asking so that I could read any previous work on the topic. I might actually spend some more time thinking about approaches myself later, but wouldn’t do it without first knowing if I’m just reinventing the wheel, so I was probing for answers. I don’t think this is particularly weird or controversial.
I don’t? I wondered if there might be one, and asked if anyone else knew any better.
I’m asking if there is a name and a specific theory of these things. I strongly disagree that just studying thermodynamics or statistical mechanics answers these questions, at least directly—though sure, if there is a theory of it, those are the tools you need to derive it. There are obvious thermodynamic limits of course, but they are usually ridiculously permissive. I’m asking if there’s a theory that tries to study things at a lower level of generality, is all, and sets more narrow bounds than just “any nanomachine could not go above Carnot efficiency” or “any nanomachine would be subject to Brownian motion” or such.
Well, as I said, there might be some general insight. For example biological cells are effectively nanomachines far beyond our ability to build, yet they are not all-powerful; no individual bacterium has single-handedly grown to grey goo its way through the entire Earth, despite there being no particular reasons why it wouldn’t be able to. This likely comes from a mixture of limits of the specific substrate (carbon, DNA for information storage), the result of competition between multiple species (which can be seen as inevitable result of imprecise copying and following divergence, even though mostly cells have mechanisms to try and prevent those sort of mistakes) and perhaps intrinsic thermodynamic limits of Von Neumann machines as a whole. So understanding which is which would be interesting and useful.
I wouldn’t say they do. This is not about known science, it’s about science we don’t know, and what can we guess about it. Some considerations on gravity and quantum mechanics do indeed put a lower bound on the energy scale at which we expect new physics to manifest, but that doesn’t mean that even lower energy new physics aren’t theoretically possible—if they weren’t, there would be no point doing anything at the LHC past the discovery of the Higgs Boson, since the Standard Model doesn’t predict anything else. Thought to be fair, the lack of success in finding literally anything predicted either by string theory or by supersymmetry isn’t terribly encouraging in this respect.
Fair, I wasn’t aware of this! Though I’m going to guess that American Football still is uniquely bad in the rates of such injuries? This paper suggests that:
https://doi.org/10.4088/PCC.20br02653
I think there’s going to be some repeated trauma involved in lots of sports (e.g. skiers stress their knees a lot and have a lot of risk of fractures, lots of team sports involve risks of collisions with other players, deep divers often black out on resurfacing, boxing literally requires one to cause some kind of head trauma to the opponent as one condition to end the game, and so on so forth).
I would guess football, hockey, rugby, boxing, kick-boxing and MMA to be amongst the worst sports for this stuff. Wrestling too possibly though obviously in that case it’s more like the result of performances gone wrong.
picking actions it thinks humans would take
Honestly this kinda feels like what LLM agents do… with the exception that LLM agents have been trained on a vast corpus including lots of fiction, so their definition of “actions humans would take” tend to be fairly skewed sometimes on some particular topics.
Yes, there is certainly a kind of altruistic motivation too, but it doesn’t really explain why individuals seem to be eager to defend their country.
It’s basic Golden Rule stuff. Sure, the army maybe would win without me. But if everyone thought that way and was a freerider, no one would go fight in the army, and the war would be lost. People feel responsible, they feel guilty, they feel ashamed, they have a sense of duty and of what’s right.
You know who else was a young hot-blooded male? All those Russian men who booked it as soon as the invasion started just so they could escape conscription. Russia mostly ended up needing to conscript and press-gang people into fighting its stupid pointless war, because unlike Ukrainians, these men could see clear as day that they had absolutely no good reason to risk their life, and they didn’t give a shit if their country lost.
You realize over a million people in the U.S. practice competitive MMA, right? Say ~0.25% of those people are interested in mortal combat. There’s your arena.
0.25% of 1 million is 2500 people. Hardly a massive social problem that needs addressing with an escape valve. Having a ban on death matches is necessary to prevent people who do NOT crave the battlefield from being variously socially or economically pressured in risking (and losing) their life, as it used to be. If this means that 2500 MMA fighters are a bit frustrated by the impossibility to engage openly and legally in deadly combat on live TV as their berserker hearts would wish, well, tough luck. It’s not even that they can’t do that anyway privately—they just need to be willing to risk jail besides their life.
Well, obviously so. But that sounds more like a PhD program than a LW comment. My point was, there seems to be a trend, and the trend is “self reflection allows for self contradiction and inconsistency”. I imagine the general thrust of a more formal argument could be imagining that there is a “correct” decision theory, imagining a Turing machine implementing such a theory, proving that this machine is in itself Turing-complete (as in, it is always possible to input a decision problem that maps to any arbitrary algorithm) and then it would follow that it being reflexive would require it to be able to solve the halting problem.