TAG

• TAG 8 Dec 2025 12:38 UTC

  2 points

  0

  in reply to: Vanessa Kosoy’s comment on: In­ter­pret­ing Quan­tum Me­chan­ics in In­fra-Bayesian Physicalism

  >The interpretation of quantum mechanics is a philosophical puzzle that was baffling physicists and philosophers for about a century. In my view, this confusion is a symptom of us lacking a rigorous theory of epistemology and metaphysics

  They are symptoms of each other. Epistemology depends on ontology, ontology depends on the right interpretation of physics...which depends on epistemology.

  >What is so confusing about quantum mechanics?

  Recovering the appearance of a classical reality from a profoundly unclassical substructure.

  >and CI will prescribe contradictory beliefs to different agents (as in the Wigner’s friend thought experiment).

  It’s important to notice that Wigners Friend only implies an inconsistency about when collapse occurred: there is no inconsistency between what Wigner and the Friend actually measure. So CI is still consistent as an instrumentalist theory. Indeed, you can cast it as an instrumentalist theory, by refusing to speak about collapse as a real process, and only requiring that observers make sharp real-valued observations for some unknown reason—substituting an irrealist notion of measurement for collapse.

  >In CI, the wavefunction is merely a book-keeping device with no deep meaning of its own. In contrast

  If you are going to be realist about collapse, you need to be realist about WF’s—otherwise what is collapsing? Realism about both is consistent, irrealism about both is consistent.

  >However, the MWI has no mathematical rule for computing probabilities of observation sequences. If all “worlds” exist at the same time, there’s no obvious reason to expect to see one of them rather than another. MWI proponents address this by handwaving into existence some “degree of reality” that some worlds posses more than others. However, the fundamental fact remains that there is no well-defined prescription for probabilities of observation sequences, unless we copy the prescription of CI: however the latter is inconsistent with the intent of MWI in cases when decoherence fails, such as Wigner’s friend.

  “The” MWI is more than one thing. If you are talking about coherent superposition, then there is no need to invent a probabilistic weighting , since standard quantum mechemical measure (WF amplitude) gives you that via the Born rule. But we can tell.we are not in a macroscopic coherent superposition, because it would look weird. If you are talking about decoherent branching, it’s not clear how or whether branches are weighted. That might matter for the purposes of physics, since the experimenter is only operating in one branch, and any other can have no effect. But for some other purposes , such as large world ethics, or anthropics, it might be nice to have weights for branches.

  >and fundamentally this information is all there is

  If you want to do decision theory, and not just probability theory , you need ontology, not just epistemology. But why would the process stop there? Is decision theory the most ambitious goal you can have?

  Most of the current debate about metaphysics centres on consciousness. Physicalism, of a substantial kind, materialism, seems unable to explain phenomenal consciousness—why should an insubstantial, information-only ontology fare any better? Rather than having anything extra to offer, it is more minimal. The same information can appear in different forms .. drinking the wine is not reading the label .. which is a strong hint that information is not there is.

• TAG 4 Dec 2025 12:51 UTC

  2 points

  0

  in reply to: Ape in the coat’s comment on: The Joke

  All the “contentiousness” evaporates as soon as we’ve fixed the definitions and got rid of the semantic confusion.

  Of course not. Having clear semantics is a necessary condition for understanding the world, not a sufficient one. You have to look. Among other things.

  You gather evidence about interpreting evidence

  You can only gather theories about interpreting evidence. You can’t see how well such theories work by direct inspection. It isn’t looking.

  This would work much better if you thought about it concretely. Alice says evidence includes introspections, subjective seemings; Bob says it is only ever objective. What do you do next?

  When you are a result of successful replication of imperfect replicators in a competitive environment with limited resources there is quite a lot of evidence for some kind of “optimality”.

  I doing see why a slug or wallaby is optimising anything, so why should I be? What makes humans the pinnacle of creation?

  If the existed in some kind of separate magisterium where our common knowledge wouldn’t be applicable than yes

  They exist in a separate magisterium where direct, sensory evidence isn’t applicable, because they are about the causes and meaning of whatever sensory evidence you happen to have. The interpretation of evidence.is a separate magisterium from.You gathering evidence , and not in a spooky way.

  So you use indirect feedback

  Which is what? Give concrete examples.

  I can keep applying it to the “tricky cases

  Applying what? You can’t keep applying evidence-gathering to solve the problem of interpreting evidence. It’s unclear whether you are talking about pure empiricism, or.some.kind of vaguely defined solution everything.

  And this way I can aggregate more and more evidence

  Which is not self interpreting, so you are just creating a bigger and bigger problem.

  We can try multiple of them and see how these models predict new data

  But they dont, in the trickiest. I’ve already addressed that point: the Ptolemaic model can by adjusted to fit any data.

  Being already selected for intuitions related to surviving in the world

  I’ve already addressed that point too: you don’t need ontological understanding to survive. You don’t get direct feedback about ontological understanding. So it’s a separate magisterium.

  Okay I think I understand what is going on here. Are you under impression that I’m trying to bring back the old empiricism vs rationalism debate, arguing on the side of empiricism?

  What’s “looking” if not empiricism?

  I’m not arguing for rationalism over empiricism, or against never using empirucism. I’m arguing against pure empiricism as being able to solve all problems. Which is not to say there is something else that does. It’s a mixture of pluralism—there’s more than one kind of epistemic problem and solution—and scepticism—theres no guarantee of solving anything even using more tools than “looking”.

  I already said that here :-

  And Its also not the case that you have to make positive claims about apriori reasoning to point out the limitations of empiricism. And Its also not the case that noticing the limitations of empiricism is the same as refusing to use it at all.

  Yes, it’s all probabilities all the way down, without perfect certainty

  No, it’s worse than that. Probabilities require quantification of how true or likely something is. But there is no way of objectively quantifying that for ontological interpretation. And subjective probability leads to perennial disagreement , not convergence.

  We can come up with adversarial examples where it means that we were completely duped, and our views are completely disentangled from “true reality” and were simply describing an “illusion”, but

  But, that only allows us to reject N false theories , not home in on a single true one. Convergence is a problem as well as certainty.

  Renaming “reality” to “illusion” doesn’t actually change anything of substance

  If your beliefs are illusory, they are false. That might not make an difference instrumentally, to what you can predict, but you are not assuming instrumentalism and neither is Yudkowsky.

  But generally, consider the fact that philosophy reasons in all direction and normality is only a relatively small space of all possible destinations

  What’s normality? If you just mean “saving appearances”, rather than predicting something that is empirically disproveable, then most philosophy does that. What doesn’t? Illusionism? But that’s quite popular around here!

  ---

  I also thought the robot:s answer missed the point quite badly …because it reduced the ought all the way down to an is—or rather a bunch of isses.

  If you dismiss any reduction of ought to is,

  I don’t. As I said:-

  Reducing ethical normativity isn’t bad

  Not to what one would. Your ethical module may not be directly connected to the behavioral one and so your decisions are based on other considerations, like desires unrelated to ethics.

  Are you saying that’s the only problem? That the action you would have taken absent those issues is the right action, in an ultimate sense?

  This doesn’t change the fact that what you ought to do is the output (or a certain generalization of multiple outputs) of the ethical module,

  It’s not a fact. There are any number of ethical theories where what you should do is not necessarily what you would do. e.g. Utilitarianism, which is quite popular round here. When you think about maths , that’s neural activity, but it doesn’t follow that it defines mathematical correctness. Errors are neural activity as.well. The normative question is quite separate. Even if want to reduce it, it doesn’t follow that they only way to do so is to have eight billion correct answers.

  which is a computation taking place in the real world, which can be observed.

  That’s quite irrelevant. The fact that it takes neural activity to output an action tells you nothing about the ethics of the action. “Ought” and “ethical” aren’t just vacuous labels for anything you do or want to do.

  there are potentially eight billion answers to what one ought to do.

  Potentially but not actually.

  Nothing hinges on having exactly right billion right answers. More than one right answer is enough of a problem.

  Once again, when you look, turns out individual ethical views of people are not *that* different

  Yes they are. Political divisions reflect profound ethical divisions.

  There’s a consistent theme in rationalist writing on ethics, where the idea that everyone has basically the same values , or “brain algorithms”, is just assumed … but it needs to be based on evidence as much as anything else.

  Not basically the same, but somewhat similar. And it’s not just assumed, it’s quite observable.

  The differences are observable. Fraught debates are people disagreeing about the value of freedom.versus equality, etc.

  I’m any case, the problem of subjectivism is that there are potentially multiple right answers.

  Human ethical disagreements are mostly about edge cases. Like what is your objective claim here, that human values are not correlated at all?

  No. I don’t accept that ethics “is” whether values you happen to have, or whatever decision you happen to make.

  It’s social constructivism of morality. Which is rooted in our other knowledge about game theory and evolution.

  If morality is socially constructed , the robot is wrong about metaethics. What the robot should do.is follow the social rules, and if its programming is something different, then it’s actions are object level wrong. .

  Yes, this is exactly my point. A lot of things, which are treated as “applied missing the point answers” are in fact legitimately philosophically potent. At the very least, we should be paying much more attention to them.

  Is the robot missing the point or not?

  Therefore it’s not just “by looking” but “pretty much by looking”. I completely agree about the necessity to abandon the notion of certainty

  That’s just the start. The tricky question is now much else we need to abandon. In particular , it’s not clear whether convergence on a single most likely theory of everything is possible, even if you have abandoned certainty.

• TAG 3 Dec 2025 22:54 UTC

  9 points

  0

  in reply to: habryka’s comment on: Reliti­gat­ing the Race to Build Friendly AI

  MIRI’s plan, to build a Friendly AI to take over the world in service of reducing x-risks, was a good one.

  How much was this MIRI’s primary plan?

  It was Yudkowsky’s plan before MIRI was MIRI

  http://​​sl4.org/​​archive/​​0107/​​1820.html

  “Creating Friendly AI”

  https://​​intelligence.org/​​files/​​CFAI.pdf

  Both from 2001.

• TAG 2 Dec 2025 11:20 UTC

  2 points

  0

  in reply to: Ape in the coat’s comment on: The Joke

  I also thought the robot:s answer missed the point quite badly …because it reduced the ought all the way down to an is—or rather a bunch of isses.

  If what one ought to do reduces to what one would do, there are potentially eight billion answers to what one ought to do. That works for a decision theory, because decision theory only needs to guide individual actions—but not for an ethical theory, because ethical theories need to identify bad people who need punishment, and no one is bad by their own values. Predictive accuracy is irrelevant to both, it’s a different ball game.

  There’s a consistent theme in rationalist writing on ethics, where the idea that everyone has basically the same values , or “brain algorithms”, is just assumed … but it needs to be based on evidence as much as anything else. The Rationalist approach isn’t some deep insight, it is quite flawed

  Reducing ethical normativity isn’t bad, but doing it in a way that leads to sweeping subjetivism is bad. If you accept subjectivism, you miss better answers.

  ---

  In the first comic, the engineers answer is a also a plausible philosophical answer. The “philosopher” is assuming that identity has to be based on intrinsic properties, the “engineer” isnt. But there is actually no philosophical fact that identity should be intrinisc. In Bensinger’s When is a Mind Me he takes the “philosopher’s” perspective.

  ---

  In the third comic, the philosopher is technically correct. You can’t achieve certainty from a finite.chain of observations, and you can only make a finite chain. Modern empiricists have admitted this giving up on certainty.

• TAG 1 Dec 2025 12:24 UTC

  11 points

  0

  on: The Joke

  As a warming up exercise, try the tree story with something actually contentious:-

  Alice: rainbows exist, I can see one over there.

  Bob: rainbows don’t exist, there is no arch in the sky, they are an illusion.

  Charlie: A physical phenomenon produces rainbows, but they are not as they appear.

  Etc, etc.

  ---

  Alice: Qualia exist, I’m.having several right now.

  Bob: there is no empirical evidence for qualia, because no instrument can detect them.

  Keith: Qualia can’t exist, because they are non physical by definition. They must be illusions.

  Alice: illusions of qualia are qualia.

  Etc, etc.

  ---

  Alice: I have free will, because it seems to me that I can raise my arm or not as I please.

  Bob: Thou Art physics, so thou hast no free will.

  Charlie: You are still a cause.

  Bob: I am a caused cause, so I am not free.

  Etc, etc.

  ---

  Alice: Ethics is objective because it seems to me that torturing kittens a wrong , and that’s a basic fact about the universe.

  Bob: Everything is physics , so there are no n in natural facts, so moral realism is false. You intuitions are irrelevant.

  Mike: Science is based on intuition just as much as moral realism, it’s intuitions all the way down, baby.

  Etc, etc.

  ---

  Alice: Quarks exist, we have scientific and therefore empirical evidence.

  Bob: Actually, we have never made a direct observation of an isolated quark … Quarks are kind of inferred from the nature and behaviour of nucleons … they are in the nature of explanatory posits.

  David: well, of course, science is all conjecture, there is no evidence of anything and empiricism and induction don’t work!

  Bob: that’s a bit extreme, but we can agree that science uses explanatory conjectures as well as empiricism and induction.

  ---

  Note the lack of agreement about what empirical evidence even is, and about how science works. That’s the very beginnings of the problems with “Lol, just use science”.

  Except… how good are you, really, if you are stuck asking the same questions as a 12 years old child?

  Probably not very, but philosophy isn’t stuck there, since it has plenty of highly technical debates.

  But the most ironic is that if one actually goes on a long philosophical journey in search for the answer to the question “How can we know things about the external world at all?”, then, in the end of this heroic quest, after the three headed dragon is slain, kingdom of reason is saved and skeptics are befriended along the way, on the diamond mural written in golden letter will be found the answer.

  And this answer will be: “Pretty much by looking”.

  Here’s why not:-

  The evolutionary argument guarantees that you can know just enough for your survival, and only that, things like which berries to eat and animals to avoid being eaten by. ( There’s no evidence at all that it’s “optimal”).Typical philosophical problems, about ontology and epistemology, the real nature of things, have no relevance to survival, so the evolutionary argument doesn’t tell you they are soluble. Not everything is on the same footing. Some things are visible, and subject to direct feedback, other things arent.

  To shown that empiricism works, you need to show that empiricism*alone” works, and it works for everything , including the tricky edge cases. And you can’t infer all that from the fact that it works in one , simple case.

  The limitation doesn’t just apply to philosophy, it also applied to science , because science does try to answer questions about the nature of things … scientific realism … as well as making reliable predictions, instrumentalism. Realism depends on the interpretation of observation, which in turn depends on conjecturing explanatory models …models don’t just drop out of the data. Conjecture is a conscious, creative and cognitive process, not something that happens automatically as part of perception. Scientific epistemology is complex, combining conjecture , deduction of predicted observations from models, confirmation/​disinformation of models, incremental changes to models, and wholesale abandonment of models.

  There are direct quantifiable tests for predictive accuracy , but no way of directly comparing a map to the territory. Scientific realists hope and assume that empirical adequacy adds up to ontological correctness .. but it isn’t necessarily so.

  One way of making this point is that ontologically wrong theories can be very accurate.

  For instance, the Ptolemaic system can be made as accurate as you want for generating predictions, by adding extra epicycles … although it is false, in the sense of lacking ontological accuracy, of failing to correspond to reality, since epicycles don’t exist.

  A further way to see this point to notice that ontological revolutions can make merely modest changes to predictive abilities. Relativity inverted the absolute space and time of Newtonian physics, but its predictions were so close that subtle experiments were required to distinguish the two, and so close that Newtonian physics is acceptable for many purposes. Moreover, we can’t rule out a further revolution, replacing current scientific ontology.

  Our inability to make direct comparisons between map and territory extends to an inability to tell how close we are to the ultimately accurate ontology,. Even probablistic reasoning can’t tell us how likely our theories are in absolute terms. We only know that better theories are more probably correct than worse ones, but we don’t really know whether current theories are 90% correct or 10% correct, from a God’s eye point of view.

  Moreover, we can’t safely assume we are making steady, incremental progress towards the ultimately accurate ontological picture for just the reason already given—sligh changes in predictive accuracy going from one theory to another can be accompanied by major changes in ontology.

  and your brain that interpret and aggregate those signals, forming a mental model of the world, to be created by evolution through natural selection, optimizing for inclusive genetic fitness in this world, far from the case that philosophers are all Hegel style rationalists.

  And Its also not the case that you have to make positive claims about apriori reasoning to point out the limitations of empiricism. And Its also not the case that noticing the limitations of empiricism is the same as refusing to use it at all.

  Most of philosophy is going in the opposite direction, doing anything but adding up to normality

  Examples, please.

  Did we really expect that solving philosophy would invalidate the applied answers of the sciences

  Have you solved philosophy? Has anybody?

  There’s not much evidence that the looking based approach is solving problems in practice. Rationalists don’t have clear answers to consciousness or ethics—and how would “looking” help in those areas, anyway? Some things are visible, and subject to direct feedback, other things arent.

• TAG 25 Nov 2025 3:37 UTC

  2 points

  0

  on: Thou art rain­bow: Con­scious­ness as a Self-Refer­en­tial Phys­i­cal Process

  Illusionists, like Dennett and Frankish, argue that this datum is itself a product of cognitive architecture: consciousness only seems irreducible because our introspective systems misrepresent their own operations.3

  Also...there is no reduction..in the sense in the sense of a theory that can predict qualia .. and predicting them as illusions isn’t much different. The

  Irreducubility and ineffability aren’t just immediately given seemings.

  The “hard problem” dissolves once we recognize that the inner glow is how self-modeling appears from the inside

  Some generic “inner glow” is different to qualia, what the Hard Problem is about.

  The demand for an inner witness (“something must be experiencing something”) leads inexorably to regress

  Maybe, but an inner witness isn’t the same thing as, qualia, either.

• TAG 24 Nov 2025 18:42 UTC

  2 points

  0

  in reply to: Chara#2736’s comment on: Suicide Preven­tion Ought To Be Illegal

  Involuntary prevention is what I take issue with

  It’s not that straightforward. Some people might ,while mentally capable, give general consent to being committed if suicidal, but later withdraw it , while incapable ,at least in a professionals judgement.

  My argument is that we all have subjective preferences, and since none of these preferences are inherently superior, we therefore ought to allow each mentally competent individual to decide and act upon their own preferences

  That’s not well defined in the case above.

  perfect ought not to be the enemy of good

  But you’re not calling for less involuntary commitment.

• TAG 23 Nov 2025 13:04 UTC

  2 points

  0

  in reply to: Chara#2736’s comment on: Suicide Preven­tion Ought To Be Illegal

  The notion that one’s moral philosophy, personal values, and religion do not give one the right to infringe upon the freedoms of others is cruxy

  That isn’t the point. The point is that the argument behind Never Intervene is self defeating: you are imposing a lack of intervention on people who might want intervention, and you are doing so because it’s your own belief …you haven’t shown it is a principle that transcends merely subjective preferences.

  , I will note that a society in which everyone can impose their values upon eachothe

  I was not calling for the maximum amount of intervention or imposition: I was noting that there is no neutral ground.

  I intended to say that suicide ideation does not inherently render one mentally incompetent. If an individual is mentally incompetent, they ought to be prevented from committing suicide; if an individual is mentally competent, they ought to be allowed to exercise their autonomy.

  Is there a completely reliable way of determining that?

  Using force on a mentally competent individual cannot be justified by your belief that they will regret their actions later

  I suppose that would be a problem if you had a reliable method of determining competence. Otherwise it’s a judgment call.

• TAG 21 Nov 2025 18:08 UTC

  5 points

  −1

  on: Suicide Preven­tion Ought To Be Illegal

  The fourth objection is not difficult to refute; your moral philosophy, personal values, and religion do not give you the right to infringe upon the freedoms of others

  Says who? That’s as much of a moral claim as anything. You may or may not have the legal right .. but in this cases you do, or you would have nothing to object to.

  The second objection may be rejected on the basis that suicidal individuals are not incompetent nor beyond reason

  Obviously, that’s a continuum.

  The first objection relies upon the notion that suicide is an irreversible decision which many come to regret, as evidenced by suicide attempt survivors who do not reattempt. Unfortunately, this argument suffers from the most literal form of survivorship bias; those who are less committed to dying, and thus more likely to regret a suicide attempt, will use less lethal methods, while those who are more committed to dying, and thus less likely to regret a suicide attempt, will use more lethal methods

  That doesn’t imply “never intervene”, it implies “make a judgment”.

  By prohibiting suicide prevention, the stigma surrounding suicide ideation is reduced. The afflicted can speak to their loved ones and be honest with mental health professionals without risk of imprisonment, and are empowered to seek treatment on their own terms

  That will help some and harm others. There are people regret their suicide attempts, and many of them were saved by interventions.

• TAG 18 Nov 2025 3:33 UTC

  4 points

  1

  on: Va­ri­eties Of Doom

  .

  The second kind of paperclip maximizer is the one Yudkowsky claims to have hadin mind(I don’t believe him) when he coined the phrase

  He didn’t, Bostrom did.

  I phrase the question this way because the way Yudkowsky asks it in Value Is Fragile iskind of incoherent. Our experience of the universe is by and large deterministic,the free will question is largely resolved by chaos theory, which describes theproperties of objects which are deterministic but not predictable

  The only thing determinism could look like is predictability, and very little is naively predictable. So things don’t look deterministic, so determinism is a theory.

  ETA

  ---

  Yudkowsky objects to this viewon the basis that we’re in a block universe,

  We don’t know that. BU is an assumption beyond determinism, which is also unknown.

  that predetermination is a “timeful”concept which ignores that our choices have causal structure on which other structures depend.

  Which does make you part of the cause of the future,but doesn’t give you any freedom to choose between possible alternative futures. It’s a weaksauce notion of free will. It offers no more than the traditional compatiblism we have had for centuries. It is not an advance in the field.

  he even notices that by the time he is consciouslythinking about a premise [the vast majority of the entropic force pushing him towardsit is in the past]

  If determine is true, which we don’t know, there is no need for hedging about “vast majorities”—it’s everything.

  If determinism isnt true, then the butterfly effect can amplify a tiny amount of wiggle room as much as you like.

  But he doesn’t seem toever feel the way in which the entire universe is pressing him into place at anygiven moment to squeeze the last few bits of the hypothesis space out of him

  Determinism is a theory, so you should not expect to feel it.

  What’s the relevance anyway? Being determined doesn’t mean you are doomed.

  ---

  Do me a favor. Reread Meditations On Moloch and pay close attention to that transition on the last section about “Elua”.If you do you might notice that at exactly the moment where Scott Alexander has all but conceded the argument to Nick Land he suddenly appeals to an almost literal Deus Ex Machina to get out of it. This is because Elua does not exist and Landis to a first approximation correct, at least about the parts that are is rather than ought.

  Elua does not exist as a literal deity. Nor does Moloch. Molochian dynamics, the real world thing that Moloch is a metaphor for, do exist. So do Eluan dynamics (democracy, liberalism, free universal education, the rule of law...) If Molochian dynamics were the only game on town, it would have won out in the distant past

  The West into freefall, the only reason I’m writing thissection at all: Humanism is dead guys.

  So I think if you take away the only leverage that humans naturally have—the ability to be useful to others through work—then the leverage that many people have to secure fair treatment for themselves and their communities will drop to literally zero.

  Work is not the only lever. People can provide each other with status, entertainment , attention, etc. Look at what people in workplaces do when they are not working.

  The basic fact is that Democratic Republics dominate because RepublicanFrance was able to deploy more motivated guys with guns than other European states.

  Britain was already a democracy at the time. France didn’t preserves democracy behaving guns, its neighbours did as well, it preserved intuitive being through being the largest country in Europe. It spread Democracy some some.of its former. colonies, but the the British empire did far more.

  Democracy is compatible with monarchy, as in the UK. Democracy doesn’t depend on advanced technology, since the Athenians had it. Democracy doesn’t require revolution: the Canadians have it without fighting a war of independence.

  What’s the point anyway? Perhaps you feel that the spread of democracy is a prima.facie example of Eluan dynamics, and therefore needs to be reinterpreted as actually Molochian?

• TAG 13 Nov 2025 21:34 UTC

  6 points

  1

  in reply to: Vanessa Kosoy’s comment on: Hu­man Values ≠ Goodness

  Humans already follow their actual Values

  What people do is a compromise between what society wants them to do, and what they would otherwise do. Theres a sense in which that’s doing what they value,since they disvalue societal punishments. So there is a sense in which they are always doing what they already want, but it misses an important point .. it’s a misleading bit of cleverness.

  Without getting into moral realism, following all your own values is likely to get you into trouble with society.

• TAG 9 Nov 2025 20:49 UTC

  2 points

  0

  in reply to: kihara.sofia’s comment on: Time, Panpsy­chism, and Sub­strate In­de­pen­dence

  My explanation was a bit confusing, sorry about that! I wasn’t intending for there to be an “original” Mary; she and everyone else only ever existed as a simulation. If we were to assume substrate independence, we’d be fine with saying that the denizens of Sim#1 are conscious.

  I think the assumption the argument works from is that the Consciousness Is Computation. The substrate independence of computation , which I don’t doubt, doesn’t prove anything about consciousness without that.

  And while Sim#2 Mary is not a P-zombie to the alien, she very much is one to the people in Sim#2.

  I guess you’re correct that the right terminology would be that she’s C-zombie, but the people in the simulation can’t know that.

  And since we can’t know for sure whether we ourselves are “really” physical, for all intents and purposes we can’t be sure that there is a distinction between P- and C- zombies.

  It’s about explanation. Dualism has more resources to explain consciousness than physicalism, which has more resources than computationalism, etc. That doesn’t mean you should jump straight to the richest ontology , because that would be against Occam’s Razor. What should you do? No one knows! But there is no fact that you can explain consciousness with algorithms alone.

  Personally, I find physicalist theories of consciousness that don’t include substrate independence quite silly, but that’s a matter of taste, not a refutation.

  Computationalism is a particular form of multiple realisability. Physicalism doesn’t exclude it, or necessitate it. Other forms of multiple realisability are available.

  My vague gesturing at an argument would be something like this: a brain in a vat is halfway between a physical person and a simulation of one

  Err..why? A physical brain that happens to be in a vat is a physical brain, surely?

  first by replacing each neuron with a chip, and then by replacing networks of chips with bigger chips running a network, and so on until the whole thing is a chip. Is it really the case that we’re losing the physics?

  You are losing the specific physics. Computational substrate independence is a special case of substrate independence , but substrate independence in no case implies immateriality.

  ETA

  We used to think that the prediction of dark stars meant that Newton’s fact of gravity broke down when it came to light (which was true); but then again, we thought the same about the prediction of black holes and General Relativity. Nowadays, most physicists (probably) don’t believe that white holes exist, despite the fact that they’re just as predicted by GR as black holes, because they find the prospect absurd (in the absence of evidence).

  You can be forced into a belief in counterintuitive conclusions by strong evidence or arguments … and you should only believe it on the basis of strong evidence and arguments. The rule is not “never believe in counter intuitive conclusions” .

• TAG 5 Nov 2025 15:28 UTC

  3 points

  0

  on: Time, Panpsy­chism, and Sub­strate In­de­pen­dence

  If substrate independence is true, we have no problem saying that Sim#1 Mary was conscious, and that everyone else is conscious in both Sim#1 and Sim#2. But, if we say that Sim#2 Mary is not conscious… then we have to grapple with the fact that she is a P-zombie.[6]

  She is not exactly a p zombie. The Mary in sim #1 is not a p-zombie version of the original Mary, because she is only a functional duplicate, not a physical duplicate; and the Mary in Sim #2 is only a behavioural duplicate. So the question of “what difference explains the loss of consciousness” is easily answered—all three are differerent.

  And I don’t need to reinvent the wheel here, so I’ll just claim that belief in P-zombies is incoherent, and we don’t really have a good reason to say that she isn’t conscious. So Sim#2 Mary, a mere recording of Mary, must be cons– wait, what?and

  P zombies aren’t incoherent , they just contradict physicalism. And you are talking about c zombies, anyway.

  Physicalism has it that an exact atom-by-atom duplicate of a person will be a person and not a zombie, because there is no nonphysical element to go missing. That’s the argument against p-zombies. But if actually takes an atom-by-atom duplication to achieve human functioning, then the computational theory of mind will be false, because CTM implies that the same algorithm running on different hardware, will be sufficient. Physicalism doesn’t imply computationalism, and arguments against p-zombies don’t imply the non existence of c-zombies—unconscious duplicates that are identical computationally, but not physically.

  I think this is folly. I think we’re engaging in a category error if we’re thinking of things this way — we’re not fully grappling with the consequences of substrate independence. Are the people in Sim#1 and Sim#2 conscious twice, like some kind of deja-vu they can’t experience? I really don’t think so.

  There’s no strong reason to think they are conscious once.

  We say X is conscious if and only if there is such a thing as ⟨what it’s like to be X⟩. If when we run the automaton, we have reason to think that there is such a thing as what it’s like to be the simulated brain, but we also conclude that it shouldn’t matter whether or not you run the automaton

  Something gets lost at each stage. Going from a physical embodiment to a computational simulation loses the physics; going from a computational simulation to a behavioural simulation loses the counterfactual possibilities of the computational simulation; going from a behavioural simulation that actually runs to a notional one loses actual occurrence. Any of those losses could affect consciousness.

  I’ve come to a nearly delusional form of belief (it’s not like I’m exactly convinced), that isn’t even fully articulated here; I’ve come to really think that this whole thing is quite bogus, that there really is no difference between realism and solipsism and nihilism and a strange kind of theism

  That should be taken as a reductio as absurdum of the GAZP

  But you see the importance of the question, “How far can you generalize the Anti-Zombie Argument and have it still be valid?”

  Clearly , the answer isn’t “indefinitely” .

  ---

  @JBlack The problem with Dust theory is that it assumes that conscious states supervene on brain states instantaneously. There is no evidence for that. We should not be fooled by the “specious present”. We seem to be conscious moment-by-moment, but the “moments” in question are rather coarse-grained, corresponding to the specious present of 0.025-0.25 second or so. It’s quite compatible with the phenomenology that it requires thousands or millions of neural events or processing steps to achieve a subjective “instant” of consciousness. Which would mean you can’t salami-slice someone’s stream-of-consciousness too much without it vanishing: and also mean that spontaneously occurring Boltzman states are conscious; and also preserves the intuition that computation is a process —that a computational state is defined as being a stage of a computation.

• TAG 5 Nov 2025 12:22 UTC

  6 points

  0

  in reply to: Hide’s comment on: Tomás B.’s Shortform

  https://​​en.wikipedia.org/​​wiki/​​Helen_(unit)

• TAG 5 Nov 2025 11:47 UTC

  3 points

  0

  in reply to: Vanessa Kosoy’s comment on: Wei Dai’s Shortform

  philosophy historically did not have the right tools to solve the problems. Theoretical computer science, and AI theory in particular, is a revolutionary method to reframe philosophical problems in a way that finally makes them tractable.

  Theoretical computer science can tell you are not implementing some kind of perfect algorithm, because they tend not to be computable. It can’t tell you what you should be implementing instead.

  Naturalised ethics has been around for ages. It tends to tell you that de facto human ethics is an evolutionary kludge, not something mathematically clean.

  The open question, https://​​en.wikipedia.org/​​wiki/​​Open-question_argument the question of what is the true ethics would be, is still open. Examining the de facto operation of the brain isn’t going to tell answer it.

  About “metaethics” vs “decision theory”, that strikes me as a wrong way of decomposing the problem. We need to create a theory of agents. Such a theory naturally speaks both about values and decision making, and it’s not really possible to cleanly separate the two. It’s not very meaningful to talk about “values” without looking at what function the values do inside the mind of an agent.

  Even if you need to at least address values and decision theory , it doesn’t follow that that’s all you need. Something can be a truth without being the whole truth.

  If you only look within the minds of agents, you are missing interactions between agents. Looking inwards excludes loom my outwards.

  Just as you can’t understand money by microscopically examining coins and banknotes, you can’t understand ethics just by honing in on internal psychological processes.

  If you only look within the minds of agents, and only consider values and decision theory, you are likely to end up with something like ethical egoism … not because it is true, but you haven’t even considered alternatives.

  Humans already follow their actual Values, and will always do because their Values are the reason they do anything at all.

  But I don’t see how that says anything about ethics. Merely wanting to do something doesn’t make it ethical; and being ethical need not make something intrinsically motivating. Extrinsic motivation, rewards and punishments ,are ubiquitous .. unless you’re on a desert island. So it’s not a case of everyone always following their intrinsic motivations, and if it were, that’s still on the “is” side of the is-ought divide.

  It’s not very meaningful to talk about “decisions” without looking at the purpose of decisions.

  It’s not very meaningful to talk about ethics without looking at the purpose of ethics. Is ethics really just values, and nothing else? Is it really just decision making , like any other kind? Does it actually have no distinguishing characteristics?

  First, “ethics” is a confusing term because, on my view, the colloquial meaning of “ethics” is inescapably intertwined with how human societies negotiate of over norms. On the other hand, I want to talk purely about individual preferences, since I view it as more fundamental

  Fundamental to what? Ethics? Even if ethical behaviour is made of individual decisions, that doesn’t mean it reduces it to individual decisions, made atomistically , without regard to social mores or other people’s concerns.

  ---

  The three word theory is that “Ethics is Values” That leaves a number of unanswered questions, such as: why it’s all about me;? are all values relevant? do I have the right to put someone in jail merely for going against my values?

  It’s prima facie unlikely that such a simple theory solves all the age old problems (at least it would requires the supplementary assumption that values are hard to understand in themselves, in order to explain the persistence of ethical and metaethical puzzles) And it is easy to see the flaws.

  The one thing that the three word theory is supremely good at it is explaining, is motivation. Your values are what motivate you, so if your values are also your morals you can’t fail to be motivated.by morality.

  Is it all about me? Rationalists typically argue the case for for the three word theory by asking the rhetorical question whether you would support an ethical system that had nothing to do with your wishes. That’s a none/​some/​all confusion. I want ethics to have something to do with me, but that does not make it all about me, or mean all values are equally ethical.

  For one thing, people can have preferences that are intuitively immoral. If a psychopath wants to murder, that does not make murder moral.

  For another, values can conflict. Not all values conflict. Where they do, the three words theory doesn’t tell you who wins or loses. If morality is (are) seven billion utility functions, then a legal system will be a poor match for it (them).

  Not all decisions are individual. There’s a while set of questions about whether societal actions are justified, whether societies have rights over individuals, and so

  For instance societies have systems of punishment and reward, which, hopefully, have an ethical basis. Putting people in jail is just wanton cruelty if they have done nothing wrong. But if ethics just “is” subjective value, and values vary, as they obviously do, who lands in jail.? It’s easy enough to say the murderer and the thief, and to justify that by saying that murder and theft are against people’s widely shared preferences...but remember that the three word theory is “flat”, and treats all values the same. Should the vanilla lover or the tutti frutti lover, the little endian or the big endian go to jail, if others don’t share their preferences? Voting allows you to decide, the issue, but it is not enough to justify it, because merely having a minority preference is not a crime. on .. which aren’t answered by the simplistic there word theory.

  One can go farther and argue that such societal issues are the essence of ethics. If we consider the case of someone who is alone on a desert island, they have no need, core common-sense morality, rules and against murder because there is no one to murder, and no need of rules against theft because there is no one to steal, and from and so on … in their situation ethics isn’t even definable.

• TAG 30 Oct 2025 17:40 UTC

  2 points

  0

  in reply to: Shmi’s comment on: Wei Dai’s Shortform

  You cant solve philosophy without solving epistemology, and you can’t solve philosophy without solving epistemology. And you can’t solve epistemology because of the Problem of the Criterion, which is pretty is pretty much the same as the Münchhausen Trilemma.

  “Moreover, its [philosophy’s] central tool is intuition, and this displays a near-total ignorance of how brains work. As Michael Vassar observes, philosophers are “spectacularly bad” at understanding that their intuitions are generated by cognitive algorithms.”—Rob Bensinger, Philosophy, a diseased discipline.

  What’s the problem?

  It’s not that philosophers weirdly and unreasonably prefer intuition to empirical facts and mathematical/​logical reasoning, it is that those things either don’t go far enough, or are themselves based on intuition.

  “Just use empiricism” doesn’t work, because philosophy is about interpreting empirical data.

  “Just use maths/​logic” doesn’t work , because those things are based on axioms justified by intuitive appeal.

  “Just use reductionism” doesn’t work , because its not clear what lies at the bottom of the stack, or if anything does. Logic, epistemology and ontology have been held to be First Philosophy at different times. Logic, epistemology and ontology also seen to interact. Correct ontology depends on direct epistemology..but what minds are capable of knowing depends on ontology. Logic possibly depends on ontology too, since quantum.mechanics arguable challenges traditional bivalent logic.

  Philosophers don’t embrace intuitions because they think they are particularly reliable,but because they have reasoned that they can’t do without them. (At least, the other options allowed by the MuNchausen trilemna, circulatory and regress, are distinctly unattractive )That is the essence of the Inconvenient Ineradicability of Intuition. An unfounded foundation is what philosophers mean by “intuition”...and not a supernatural insight that could not have been produced by a material brain

• TAG 28 Oct 2025 10:12 UTC

  −8 points

  −11

  on: On Flesh­ling Safety: A De­bate by Klurl and Tra­pau­cius.

  Humans are intrinsically corrigible, as a social animals, and have specific motions, pride and shame, which are used by society to shape their systems.(as explained in the https://​​en.wikipedia.org/​​wiki/​​The_Emotion_Machine) Society provides the value systems, which can be very complex and variable … whereas, nature only provides the simple , but vital , “hooks” that value-shaping depends on. And we can tell that the hooks are simple, because toddlers (and some animals) can implement them. The basis of corrigibility in organisms is also n in rational, which answer the basic objection

  ‘We call an AI system “corrigible” if it cooperates with what its creators regard as a corrective intervention, despite default incentives for rational agents to resist attempts to shut them down or modify their preferences’. Rationality is intrinsically high level:it never goes all the way down. That being the case , there is no need to solve the problem of making a rational agent corrigible.

  The complexity of the target, final value system is irrelevant to the complexity of the hooks. Humans can learn deference and respect in culturally complex ways, eg. to defer to various gods and ancestors, but that’s still based on the same simple hooks

  Arguments against alignment can’t be redeployed as arguments against corrigbility, because they are not the same.

  So far as I can tell, there are still a number of EAs out there who did not get the idea of “the stuff you do with gradient descent does not pin down the thing you want to teach the AI, because it’s a large space and your dataset underspecifies that internal motivation” and who go, “Aha, but you have not considered that by TRAINING the AI we are providing a REASON for the AI to have the internal motivations I want! And have you also considered that gradient descent doesn’t locate a RANDOM element of the space?”

  ..is an argument against alignment , not corrigibility.

• TAG 25 Oct 2025 17:30 UTC

  10 points

  −3

  on: Re­minder: Mo­ral­ity is unsolved

  Humans haven’t figured out meta ethics well enough to show that moral realism is true. So there is a probability, not a certainty , that an AI will realise moral truths. The argument also requires the AI to be motivated by the truths it discovers, and it requires preserving human life to be an objective moral imperative. (The latter point sn’t obvious—there’s a standard Sci Fi plot where a powerful AI is tasked with solving the world’s problems, and decides humans are the problem). So three uncertain premises have to be true simultaneously, so moral realism is far from a surefire solution to AI safety.

  For the these reasons, AI safety theorists focus on friendliness the preservation of humans, as a direct goal, rather than objective goodness.

• TAG 20 Oct 2025 16:23 UTC

  2 points

  2

  on: Con­tra-Zom­bies? Con­tra-Zom­bies!: Chalmers as a par­allel to Hume

  Hume points out that nothing guarantees causal induction

  The success of that argument depends on how strongly “guarantee” is taken.

  Induction, considered as a form of deduction takes a finite list of similar observations, (A ^ B ^ C.) and models.the general law it leads to as ( A^ B ^ C ^ D ..). Since even the D is not deductively valid, the whole thing is invalid. Or is it?

  Inductivists aren’t persuaded. The traditional view is that induction is a separate magisterium anyway. The problem with that approach is ..where does it end? Is tea leaf reading a separate epistemic magisterium?

  The problem of induction can be rendered easier to solve by aiming lower: one can regard a set of observations as rendering a continuation of the pattern more likely, without rendering it certain. In further good news, that can be fast as kind of deductive justification, because it can be justified by probabilistic loguc, which is An extension if deductive logic.

  I think Eliezer Yudkowsky and Chalmers agree in a similar way to how Hume agreed with The Enlightenment. Both Yudkowsky and Chalmers see the issue of p-zombies as unimportant, something to be assumed away

  But it is not unimportant to Chalmers who needs it to argue against materialism.

  And avoidance of Epiphenomenality is still important to Yudkowsky. I don’t see how the Humean argument addresses it.

  We do know how Chalmers addresses it …

  I endorse Z[ombies], but I don’t endorse E[piphenomenalism].

  ...up to a point. On the other hand, he’s pretty ambiguous about the exact solution …

  I think that the correct conclusion of zombie-style arguments is the disjunction of the type-D, type-E, and type-F views,

  ...making difficult to say with certainty that:-

  Chalmers is a minimal dualist, arguably practically a physicalist

  @James Diacoumis

  If an author pops up in your comment section saying “Hey, you’ve misinterpreted my post!” That’s generally strong evidence that you have indeed misinterpreted their post.

  Yes, well … I’ve been quoting Chalmers’ comment, where he objects to EY’s misreading of his book. I notice that people are still quoting EY’s version as accurate.eg

• TAG 18 Oct 2025 10:14 UTC

  2 points

  0

  on: Non­triv­ial pillars of IABIED

  The cruxes you have picked out are not the ones I would have.

  ---

  The complete argument for complete extinction rests on assumptions...assumptions about the nature of intelligence, the motivations of an artificial intelligence and the means of bringing about extinction. And the conjunctive part of the argument consists of claims which need to be of high probability individually for the conclusion to be of high probability.

  1. Artificial Intelligence greater than human intelligence is possible.

  2. The AI will be an agent, it have goals/​values in the first place.

  3. The goals will be misaligned, however subtly, to be unfavorable to humanity.

  4. That the misalignment between the AI’s goals, and what we want, cannot be corrected incrementally (incorrigibility), because

  5a. …the AI will self modify in way too fast to stop.(With a sub assumption that the AI can achieve value stability under self modification.)

  Or

  5b. …the AI will.The engage in deception about its powers or motivations.

  1. That most misaligned values in the resulting ASI are highly dangerous (even goals that aren’t directly inimical to humans can be a problem for humans)

  2. And that the AI will have extensive opportunities to wreak havoc: biological warfare (custom DNA can be ordered by email), crashing economic systems (trading can be done online), taking over weapon systems, weaponing other technology and so on.

  Obviously the problem is that to claim a high overall probability of doom, each claim in the chain needs to have a high probability. It is not enough for some of the stages to be highly probable, all must be. In my opinion, the weakest parts of the argument are the ones dealing with the motivation, steps 2 to 6, not the ones dealing with the natures of intelligence and the means of destruction, (1 and 7). There’s an obvious problem in making specific high probability claims about systems that don’t yet exist.

  @Mishka

  It’s kind of strange. Why do we want to have a technical ability for any Mr.X from the defense department of a superpower Y to impart his goals and values to some ASI? It’s very easy to imagine how this could be detrimental

  Yes, but not everybody’s-dead detrimental.

  Doomers are concerned about imparting values because they believe that we are going to end up with an incorrigible Sovereign AI running everything, not a multipolar scenario with superpowers using aligned-with-themselves superintelligences as superweapons.