CFAR’s “Adjust Your Seat” principle and associated story is probably one of my most frequently referenced concepts when teaching rationality techniques.
I used to think that slower takeoff implied shorter timelines, because slow takeoff means that pre-AGI AI is more economically valuable, which means that economy advances faster, which means that we get AGI sooner. But there’s a countervailing consideration, which is that in slow takeoff worlds, you can make arguments like ‘it’s unlikely that we’re close to AGI, because AI can’t do X yet’, where X might be ‘make a trillion dollars a year’ or ‘be as competent as a bee’. I now overall think that arguments for fast takeoff should update you towards shorter timelines.
So slow takeoffs cause shorter timelines, but are evidence for longer timelines.
This graph is a version of this argument: if we notice that current capabilities are at the level of the green line, then if we think we’re on the fast takeoff curve we’ll deduce we’re much further ahead than we’d think on the slow takeoff curve.
For the “slow takeoffs mean shorter timelines” argument, see here: https://sideways-view.com/2018/02/24/takeoff-speeds/This point feels really obvious now that I’ve written it down, and I suspect it’s obvious to many AI safety people, including the people whose writings I’m referencing here. Thanks to Caroline Ellison for pointing this out to me, and various other people for helpful comments.
I think that this is why belief in slow takeoffs is correlated with belief in long timelines among the people I know who think a lot about AI safety.
Because I’m dumb, I would have found it easier to interpret the graph if the takeoff curves crossed at “current capabilities” (and thus get to high levels of capabilities at different times)
‘Realism about rationality’ is Normative Realism
‘Realism about rationality’ discussed in the context of AI safety and some of its driving assumptions may already have a name in existing philosophy literature. I think that what it’s really referring to is ‘normative realism’ overall—the notion that there are any facts about what we have most reason to believe or do. Moral facts, if they exist, are a subset of normative facts. Epistemic facts, facts about what we have most reason to believe (e.g. if there is a correct decision theory that we should use, that would be an epistemic fact), are a different subset of normative facts.
These considerations (from the original article) seem to clearly indicate ‘realism about epistemic facts’ in the metaethical sense:
The idea that there is an “ideal” decision theory.
The idea that, given certain evidence for a proposition, there’s an “objective” level of subjective credence which you should assign to it, even under computational constraints.
The idea that having having contradictory preferences or beliefs is really bad, even when there’s no clear way that they’ll lead to bad consequences (and you’re very good at avoiding dutch books and money pumps and so on).
These seem to imply normative (if not exactly moral) realism in general:
The idea that morality is quite like mathematics, in that there are certain types of moral reasoning that are just correct.
The idea that defining coherent extrapolated volition in terms of an idealised process of reflection roughly makes sense, and that it converges in a way which doesn’t depend very much on morally arbitrary factors.
If this ‘realism about rationality’ really is rather like “realism about epistemic reasons/‘epistemic facts’”, then you have the ‘normative web argument’ to contend with, that the above two may be connected:
These and other points of analogy between the moral and epistemic domains might well invite the suspicion that the respective prospects of realism and anti-realism in the two domains are not mutually independent, that what is most plausibly true of the one is likewise most plausibly true of the other. This suspicion is developed in Cuneo’s “core argument” which runs as follows (p. 6):
(1) If moral facts do not exist, then epistemic facts do not exist.
(2) Epistemic facts exist.
(3) So moral facts exist.
(4) If moral facts exist, then moral realism is true.
(5) So moral realism is true.
If ‘realism about rationality’ is really just normative realism in general, or realism about epistemic facts, then there is already an extensive literature on whether it is correct or not. I’m going to discuss some of it below.
Normative realism implies identification with system 2
There’s a further implication that normative realism has—it makes things like this (from the original article, again) seem incoherent:
Implicit in this metaphor is the localization of personal identity primarily in the system 2 rider. Imagine reversing that, so that the experience and behaviour you identify with are primarily driven by your system 1, with a system 2 that is mostly a Hansonian rationalization engine on top (one which occasionally also does useful maths). Does this shift your intuitions about the ideas above, e.g. by making your CEV feel less well-defined?
I find this very interesting because locating personal identity in system 1 feels conceptually impossible or deeply confusing. No matter how much rationalization goes on, it never seems intuitive to identify myself with system 1. How can you identify with the part of yourself that isn’t doing the explicit thinking, including the decision about which part of yourself to identify with? It reminds me of Nagel’s The Last Word.
Normative Realism by degrees
Further to the whole question of Normative / moral realism, there is this post on Moral Anti-Realism. While I don’t really agree with it, I do recommend reading it—one thing that it convinced me of is that there is a close connection between your particular normative ethical theory and moral realism. If you claim to be a moral realist but don’t make ethical claims beyond ‘self-evident’ ones like pain is bad, given the background implausibility of making such a claim about mind-independent facts, you don’t have enough ‘material to work with’ for your theory to plausibly refer to anything . The Moral Anti-Realism post presents this dilemma for the moral realist:
There are instances where just a handful of examples or carefully selected “pointers” can convey all the meaning needed for someone to understand a far-reaching and well-specified concept. I will give two cases where this seems to work (at least superficially) to point out how—absent a compelling object-level theory—we cannot say the same about “normativity.”
...these thought experiments illustrate that under the right circumstances, it’s possible for just a few carefully selected examples to successfully pinpoint fruitful and well-specified concepts in their entirety. We don’t have the philosophical equivalent of a background understanding of chemistry or formal systems… To maintain that normativity—reducible or not—is knowable at least in theory, and to separate it from merely subjective reasons, we have to be able to make direct claims about the structure of normative reality, explaining how the concept unambiguously targets salient features in the space of possible considerations. It is only in this way that the ambitious concept of normativity could attain successful reference. As I have shown in previous sections, absent such an account, we are dealing with a concept that is under-defined, meaningless, or forever unknowable.
The challenge for normative realists is to explain how irreducible reasons can go beyond self-evident principles and remain well-defined and speaker-independent at the same time.
To a large degree, I agree with this claim—I think that many moral realists do as well. Convergence type arguments oftenappear in more recent metaethics (Hare and Parfit are in those previous lists) - so this may already have been recognised. The post discusses such a response to antirealism at the end:
I titled this post “Against Irreducible Normativity.” However, I believe that I have not yet refuted all versions of irreducible normativity. Despite the similarity Parfit’s ethical views share with moral naturalism, Parfit was a proponent of irreducible normativity. Judging by his “climbing the same mountain” analogy, it seems plausible to me that his account of moral realism escapes the main force of my criticism thus far.
But there’s one point I want to make which is in disagreement with that post. I agree that how much you can concretely say about your supposed mind-independent domain of facts affects how plausible its existence should seem, and even how coherent the concept is, but I think that this can come by degrees. This should not be surprising—we’ve known since Quine and Kripke that you can have evidential considerations for/against and degrees of uncertainty about a priori questions. The correct method in such a situation is Bayesian—tally the plausibility points for and against admitting the new thing into your ontology. This can work even if we don’t have an entirely coherent understanding of normative facts, as long as it is coherent enough.
Suppose you’re an Ancient Egyptian who knows a few practical methods for trigonometry and surveying, doesn’t know anything about formal systems or proofs, and someone asks you if there are ‘mathematical facts’. You would say something like “I’m not totally sure what this ‘maths’ thing consists of, but it seems at least plausible that there are some underlying reasons why we keep hitting on the same answers”. You’d be less confident than a modern mathematician, but you could still give a justification for the claim that there are right and wrong answers to mathematical claims. I think that the general thrust of convergence arguments puts us in a similar position with respect to ethical facts.
If we think about how words obtain their meaning, it should be apparent that in order to defend this type of normative realism, one has to commit to a specific normative-ethical theory. If the claim is that normative reality sticks out at us like Mount Fuji on a clear summer day, we need to be able to describe enough of its primary features to be sure that what we’re seeing really is a mountain. If all we are seeing is some rocks (“self-evident principles”) floating in the clouds, it would be premature to assume that they must somehow be connected and form a full mountain.
So, we don’t see the whole mountain, but nor are we seeing simply a few free-floating rocks that might be a mirage. Instead, what we see is maybe part of one slope and a peak.
Let’s be concrete, now—the 5 second, high level description of both Hare’s and Parfit’s convergence arguments goes like this:
If we are going to will the maxim of our action to be a universal law, it must be, to use the jargon, universalizable. I have, that is, to will it not only for the present situation, in which I occupy the role that I do, but also for all situations resembling this in their universal properties, including those in which I occupy all the other possible roles. But I cannot will this unless I am willing to undergo what I should suffer in all those roles, and of course also get the good things that I should enjoy in others of the roles. The upshot is that I shall be able to will only such maxims as do the best, all in all, impartially, for all those affected by my action. And this, again, is utilitarianism.
and
An act is wrong just when such acts are disallowed by some principle that is optimific, uniquely universally willable, and not reasonably rejectable
In other words, the principles that (whatever our particular wants) would produce the best outcome in terms of satisfying our goals, could be willed to be a universal law by all of us and would not be rejected as the basis for a contract, are all the same principles. That is at least suspicious levels of agreement between ethical theories. This is something substantive that can be said—out of every major attempt to get at a universal ethics that has in fact been attempted in history: what produces the best outcome, what can you will to be a universal law, what would we all agree on, seem to produce really similar answers.
The particular convergence arguments given by Parfit and Hare are a lot more complex, I can’t speak to their overall validity. If we thought they were valid then we’d be seeing the entire mountain precisely. Since they just seem quite persuasive, we’re seeing the vague outline of something through the fog, but that’s not the same as just spotting a few free-floating rocks.
Now, run through these same convergence arguments but for decision theory and utility theory, and you have a far stronger conclusion. there might be a bit of haze at the top of that mountain, but we can clearly see which way the slope is headed.
This is why I think that ethical realism should be seen as plausible and realism about some normative facts, like epistemic facts, should be seen as more plausible still. There is some regularity here in need of explanation, and it seems somewhat more natural on the realist framework.
I agree that this ‘theory’ is woefully incomplete, and has very little to say about what the moral facts actually consist of beyond ‘the thing that makes there be a convergence’, but that’s often the case when we’re dealing with difficult conceptual terrain.
I wouldn’t necessarily describe myself as a realist. I get that realism is a weird position. It’s both metaphysically and epistemologically suspicious. What is this mysterious property of “should-ness” that certain actions are meant to possess—and why would our intuitions about which actions possess it be reliable? But I am also very sympathetic to realism and, in practice, tend to reason about normative questions as though I was a full-throated realist.
From the perspective of x, x is not self-defeating
It’s correct that anti-realism means that none of our beliefs are justified in the realist sense of justification. The same goes for our belief in normative anti-realism itself. According to the realist sense of justification, anti-realism is indeed self-defeating.
However, the entire discussion is about whether the realist way of justification makes any sense in the first place—it would beg the question to postulate that it does.
From the perspective of Theism, God is an excellent explanation for the universe’s existence since he is a person with the freedom to choose to create a contingent entity at any time, while existing necessarily himself. From the perspective of almost anyone likely to read this post, that is obvious nonsense since ‘persons’ and ‘free will’ are not primitive pieces of our ontology, and a ‘necessarily existent person’ makes as much sense as ‘necessarily existent cabbage’- so you can’t call it a compelling argument for the atheist to become a theist.
By the same logic, it is true that saying ‘anti-realism is unjustified on the realist sense of justification’ is question-begging by the realist. The anti-realist has nothing much to say to it except ‘so what’. But you can convert that into a Quinean, non-question begging plausibility argument by saying something like:
We have two competing ways of understanding how beliefs are justified. One is where we have anti-realist ‘justification’ for our beliefs, in purely descriptive terms, the other in which there are mind-independent facts about which of our beliefs are justified, and the latter is a more plausible, parsimonious account of the structure of our beliefs.
This won’t compel the anti-realist, but I think it would compel someone weighing up the two alternative theories of how justification works. If you are uncertain about whether there are mind-independent facts about our beliefs being justified, the argument that anti-realism is self-defeating pulls you in the direction of realism.
I think that it is a more damaging mistake to think moral antirealism is true when realism is true than vice versa, but I agree with you that the difference is nowhere near infinite, and doesn’t give you a strong wager.However, I do think that normative anti-realism is self-defeating, assuming you start out with normative concepts (though not an assumption that those concepts apply to anything). I consider this argument to be step 1 in establishing moral realism, nowhere near the whole argument.
Epistemic anti-realism
Cool, I’m happy that this argument appeals to a moral realist! ….
...I don’t think this argument (“anti-realism is self-defeating”) works well in this context. If anti-realism is just the claim “the rocks or free-floating mountain slopes that we’re seeing don’t connect to form a full mountain,” I don’t see what’s self-defeating about that...
To summarize: There’s no infinitely strong wager for moral realism.
I agree that there is no infinitely strong wager for moral realism. As soon as moral realists start making empirical claims about the consequences of realism (that convergence is likely), you can’t say that moral realism is true necessarily or that there is an infinitely strong prior in favour of it. An AI that knows that your idealised preferences don’t cohere could always show up and prove you wrong, just as you say. If I were Bob in this dialogue, I’d happily concede that moral anti-realism is true.If (supposing it were the case) there were not much consensus on anything to do with morality (“The rocks don’t connect...”), someone who pointed that out and said ‘from that I infer that moral realism is unlikely’ wouldn’t be saying anything self-defeating. Moral anti-realism is not self-defeating, either on its own terms or on the terms of a ‘mixed view’ like I describe here:
We have two competing ways of understanding how beliefs are justified. One is where we have anti-realist ‘justification’ for our beliefs, in purely descriptive terms, the other in which there are mind-independent facts about which of our beliefs are justified...
However, I do think that there is an infinitely strong wager in favour of normative realism and that normative anti-realism is self-defeating on the terms of a ‘mixed view’ that starts out considering the two alternatives like that given above. This wager is because of the subset of normative facts that are epistemic facts.The example that I used was about ‘how beliefs are justified’. Maybe I wasn’t clear, but I was referring to beliefs in general, not to beliefs about morality. Epistemic facts, e.g. that you should believe something if there is sufficient amount of evidence, are a kind of normative fact. You noted them on your list here.So, the infinite wager argument goes like this -
1) On normative anti-realism there are no facts about which beliefs are justified. So there are no facts about whether normative anti-realism is justified. Therefore, normative anti-realism is self-defeating.
Except that doesn’t work! Because on normative anti-realism, the whole idea of external facts about which beliefs are justified is mistaken, and instead we all just have fundamental principles (whether moral or epistemic) that we use but don’t question, which means that holding a belief without (the realist’s notion of) justification is consistent with anti-realism.So the wager argument for normative realism actually goes like this -
2) We have two competing ways of understanding how beliefs are justified. One is where we have anti-realist ‘justification’ for our beliefs, in purely descriptive terms of what we will probably end up believing given basic facts about how our minds work in some idealised situation. The other is where there are mind-independent facts about which of our beliefs are justified. The latter is more plausible because of 1).
Evidence for epistemic facts?
I find it interesting the imagined scenario you give in #5 essentially skips over argument 2) as something that is impossible to judge:
AI: Only in a sense I don’t endorse as such! We’ve gone full circle. I take it that you believe that just like there might be irreducibly normative facts about how to do good, the same goes for irreducible normative facts about how to reason?
Bob: Indeed, that has always been my view.
AI: Of course, that concept is just as incomprehensible to me.
The AI doesn’t give evidence against there being irreducible normative facts about how to reason, it just states it finds the concept incoherent, unlike the (hypothetical) evidence that the AI piles on against moral realism (for example, that people’s moral preferences don’t cohere).Either you think some basic epistemic facts have to exist for reasoning to get off the ground and therefore that epistemic anti-realism is self-defeating, or you are an epistemic anti-realist and don’t care about the realist’s sense of ‘self-defeating’. The AI is in the latter camp, but not because of evidence, the way that it’s a moral anti-realist (...However, you haven’t established that all normative statements work the same way—that was just an intuition...), but just because it’s constructed in such a way that it lacks the concept of an epistemic reason.So, if this AI is constructed such that irreducibly normative facts about how to reason aren’t comprehensible to it, it only has access to argument 1), which doesn’t work. It can’t imagine 2).However, I think that we humans are in a situation where 2) is open to consideration, where we have the concept of a reason for believing something, but aren’t sure if it applies—and if we are in that situation, I think we are dragged towards thinking that it must apply, because otherwise our beliefs wouldn’t be justified.However, this doesn’t establish moral realism—as you said earlier, moral anti-realism is not self-defeating.
If anti-realism is just the claim “the rocks or free-floating mountain slopes that we’re seeing don’t connect to form a full mountain,” I don’t see what’s self-defeating about that
Combining convergence arguments and the infinite wager
If you want to argue for moral realism, then you need evidence for moral realism, which comes in the form of convergence arguments. But the above argument is still relevant, because the convergence and ‘infinite wager’ arguments support each other. The reason 2) would be bolstered by the success of convergence arguments (in epistemology, or ethics, or any other normative domain) is that convergence arguments increase our confidence that normativity is a coherent concept—which is what 2) needs to work. It certainly seems coherent to me, but this cannot be taken as self-evident since various people have claimed that they or others don’t have the concept.I also think that 2) is some evidence in favour of moral realism, because it undermines some of the strongest antirealist arguments.
By contrast, for versions of normativity that depend on claims about a normative domain’s structure, the partners-in-crime arguments don’t even apply. After all, just because philosophers might—hypothetically, under idealized circumstances—agree on the answers to all (e.g.) decision-theoretic questions doesn’t mean that they would automatically also find agreement on moral questions.[29] On this interpretation of realism, all domains have to be evaluated separately
I don’t think this is right. What I’m giving here is such a ‘partners-in-crime’ argument with a structure, with epistemic facts at the base. Realism about normativity certainly should lower the burden of proof on moral realism to prove total convergence now, because we already have reason to believe normative facts exist. For most anti-realists, the very strongest argument is the ‘queerness argument’ that normative facts are incoherent or too strange to be allowed into our ontology. The ‘partners-in-crime’/‘infinite wager’ undermines this strong argument against moral realism. So some sort of very strong hint of a convergence structure might be good enough—depending on the details.
I agree that it then shifts the arena to convergence arguments. I will discuss them in posts 6 and 7.
So, with all that out of the way, when we start discussing the convergence arguments, the burden of proof on them is not colossal. If we already have reason to suspect that there are normative facts out there, perhaps some of them are moral facts. But if we found a random morass of different considerations under the name ‘morality’ then we’d be stuck concluding that there might be some normative facts, but maybe they are only epistemic facts, with nothing else in the domain of normativity.I don’t think this is the case, but I will have to wait until your posts on that topic—I look forward to them!All I’ll say is that I don’t consider strongly conflicting intuitions in e.g. population ethics to be persuasive reasons for thinking that convergence will not occur. As long as the direction of travel is consistent, and we can mention many positive examples of convergence, the preponderance of evidence is that there are elements of our morality that reach high-level agreement. (I say elements because realism is not all-or-nothing—there could be an objective ‘core’ to ethics, maybe axiology, and much ethics could be built on top of such a realist core—that even seems like the most natural reading of the evidence, if the evidence is that there is convergence only on a limited subset of questions.) If Kant could have been a utilitarian and never realised it, then those who are appalled by the repugnant conclusion could certainly converge to accept it after enough ideal reflection!
Belief in God, or in many gods, prevented the free development of moral reasoning. Disbelief in God, openly admitted by a majority, is a recent event, not yet completed. Because this event is so recent, Non-Religious Ethics is at a very early stage. We cannot yet predict whether, as in Mathematics, we will all reach agreement. Since we cannot know how Ethics will develop, it is not irrational to have high hopes.
How to make anti-realism existentially satisfying
Instead of “utilitarianism as the One True Theory,” we consider it as “utilitarianism as a personal, morally-inspired life goal...
”While this concession is undoubtedly frustrating, proclaiming others to be objectively wrong rarely accomplished anything anyway. It’s not as though moral disagreements—or disagreements in people’s life choices—would go away if we adopted moral realism.
If your goal here is to convince those inclined towards moral realism to see anti-realism as existentially satisfying, I would recommend a different framing of it. I think that framing morality as a ‘personal life goal’ makes it seem as though it is much more a matter of choice or debate than it in fact is, and will probably ring alarm bells in the mind of a realist and make them think of moral relativism.Speaking as someone inclined towards moral realism, the most inspiring presentations I’ve ever seen of anti-realism are those given by Peter Singer in The Expanding Circle and Eliezer Yudkowsky in his metaethics sequence. Probably not by coincidence—both of these people are inclined to be realists. Eliezer said as much, and Singer later became a realist after reading Parfit. Eliezer Yudkowsky on ‘The Meaning of Right’:
The apparent objectivity of morality has just been explained—and not explained away. For indeed, if someone slipped me a pill that made me want to kill people, nonetheless, it would not be right to kill people. Perhaps I would actually kill people, in that situation—but that is because something other than morality would be controlling my actions.
Morality is not just subjunctively objective, but subjectively objective. I experience it as something I cannot change. Even after I know that it’s myself who computes this 1-place function, and not a rock somewhere—even after I know that I will not find any star or mountain that computes this function, that only upon me is it written—even so, I find that I wish to save lives, and that even if I could change this by an act of will, I would not choose to do so. I do not wish to reject joy, or beauty, or freedom. What else would I do instead? I do not wish to reject the Gift that natural selection accidentally barfed into me.
And Singer in the Expanding Circle:
“Whether particular people with the capacity to take an objective point of view actually do take this objective viewpoint into account when they act will depend on the strength of their desire to avoid inconsistency between the way they reason publicly and the way they act.”
These are both anti-realist claims. They define ‘right’ descriptively and procedurally as arising from what we would want to do under some ideal circumstances, and rigidifies on the output of that idealization, not on what we want. To a realist, this is far more appealing than a mere “personal, morally-inspired life goal”, and has the character of ‘external moral constraint’, even if it’s not really ultimately external, but just the result of immovable or basic facts about how your mind will, in fact work, including facts about how your mind finds inconsistencies in its own beliefs. This is a feature, not a bug:
According to utilitarianism, what people ought to spend their time on depends not on what they care about but also on how they can use their abilities to do the most good. What people most want to do only factors into the equation in the form of motivational constraints, constraints about which self-concepts or ambitious career paths would be long-term sustainable. Williams argues that this utilitarian thought process alienates people from their actions since it makes it no longer the case that actions flow from the projects and attitudes with which these people most strongly identify...
The exact thing that Williams calls ‘alienating’ is the thing that Singer, Yudkowsky, Parfit and many other realists and anti-realists consider to be the most valuable thing about morality! But you can keep this ‘alienation’ if you reframe morality as being the result of the basic, deterministic operations of your moral reasoning, the same way you’d reframe epistemic or practical reasoning on the anti-realist view. Then it seems more ‘external’ and less relativistic.One thing this framing makes clearer, which you don’t deny but don’t mention, is that anti-realism does not imply relativism.
In that case, normative discussions can remain fruitful. Unfortunately, this won’t work in all instances. There will be cases where no matter how outrageous we find someone’s choices, we cannot say that they are committing an error of reasoning.
What we can say, on anti-realism as characterised by Singer and Yudkowsky, is that they are making an error of morality. We are not obligated (how could we be?) towards relativism, permissiveness or accepting values incompatible with our own on anti-realism. Ultimately, you can just say that ‘I am right and you are wrong’.That’s one of the major upsides of anti-realism to the realist—you still get to make universal, prescriptive claims and follow them through, and follow them through because they are morally right, and if people disagree with you then they are morally wrong and you aren’t obligated to listen to their arguments if they arise from fundamentally incompatible values. Put that way, anti-realism is much more appealing to someone with realist inclinations.
I appear to be accidentally writing a sequence on moral realism, or at least explaining what moral realists like about moral realism—for those who are perplexed about why it would be worth wanting or how anyone could find it plausible.
Many philosophers outside this community have an instinct that normative anti-realism (about any irreducible facts about what you should do) is self-defeating, because it includes a denial that there are any final, buck-stopping answers to why we should believe something based on evidence, and therefore no truly, ultimately impartial way to even express the claim that you ought to believe something. I think that this is a good, but not perfect, argument. My experience has been that traditional analytic philosophers find this sort of reasoning appealing, in part because of the legacy of how Kant tried to deduce the logically necessary preconditions for having any kind of judgement or experience. I don’t find it particularly appealing, but I think that there’s a case for it here, if there ever was.
Irreducible Normativity and Recursive Justification
On normative antirealism, what ‘you shouldn’t believe that 2+2=5’ really means is just that someone else’s mind has different basic operations to yours. It is obvious that we can’t stop using normative concepts, and couldn’t use the concept ‘should’ to mean ‘in accordance with the basic operations of my mind’, but this isn’t an easy case of reduction like Water=H20. There is a deep sense in which normative terms really can’t mean what we think they mean if normative antirealism is true. This must be accounted for by either a deep and comprehensive question-dissolving, or by irreducible normative facts.
This ‘normative indispensability’ is not an argument, but it can be made into one:
1) On normative anti-realism there are no facts about which beliefs are justified. So there are no facts about whether normative anti-realism is justified. Therefore, normative anti-realism is self-defeating.
Except that doesn’t work! Because on normative anti-realism, the whole idea of external facts about which beliefs are justified is mistaken, and instead we all just have fundamental principles (whether moral or epistemic) that we use but don’t question, which means that holding a belief without (the realist’s notion of) justification is consistent with anti-realism. So the wager argument for normative realism actually goes like this -
2) We have two competing ways of understanding how beliefs are justified. One is where we have anti-realist ‘justification’ for our beliefs, in purely descriptive terms of what we will probably end up believing given basic facts about how our minds work in some idealised situation. The other is where there are mind-independent facts about which of our beliefs are justified. The latter is more plausible because of 1).
If you’ve read the sequences, you are not going to like this argument, at all—it sounds like the ‘zombie’ argument, and it sounds like someone asking for an exception to reductionism—which is just what it is. This is the alternative:
Where moral judgment is concerned, it’s logic all the way down. ALL the way down. Any frame of reference where you’re worried that it’s really no better to do what’s right then to maximize paperclips… well, that really part has a truth-condition (or what does the “really” mean?) and as soon as you write out the truth-condition you’re going to end up with yet another ordering over actions or algorithms or meta-algorithms or something. And since grinding up the universe won’t and shouldn’t yield any miniature ‘>’ tokens, it must be a logical ordering. And so whatever logical ordering it is you’re worried about, it probably does produce ‘life > paperclips’ - but Clippy isn’t computing that logical fact any more than your pocket calculator is computing it.
Logical facts have no power to directly affect the universe except when some part of the universe is computing them, and morality is (and should be) logic, not physics.
If it’s truly ‘logic all the way down’ and there are no ‘> tokens’ over particular functional arrangements of matter, including the ones you used to form your beliefs, then you have to give up on knowing reality as it is. This isn’t the classic sense in which we all have an ‘imperfect model’ of reality as it is. If you give up on irreducible epistemic facts you give up knowing anything, probabilistically or otherwise, about reality-as-it-is, because there are no fundamentally, objectively, mind-independent ways you should or shouldn’t form beliefs about external reality. So you can’t say you’re better than the pebble with ‘2+2=5’ written on it, except descriptively, in that the causal process that produced the pebble contradict the one that produced 2+2=4 in your brain.
What’s the alternative? If we don’t deny this consequence of normative antirealism, we have two options. One is the route of dissolving the question, by analogy with how reductionism has worked in the past, the other is to say that there are irreducible normative facts. In order to dissolve the question correctly, it needs to be in a way that shows a denial of epistemic facts isn’t damaging, doesn’t lead to epistemological relativism or scepticism. We can’t simply declare that normative facts can’t possibly exist—otherwise you’re vulnerable to the argument 2). David Chalmers talks about question-dissolving for qualia:
You’ve also got to explain why we have these experiences. I guess Dennett’s line is to reject the idea there are these first-person data and say all you do, if you’ve can explain why you believe why you say there are those things. Why do you believe there are those things? Then that’s good enough. I find that line which Dennett has pursued inconsistently over the years, but insofar as that’s his line, I find that a fascinating and powerful line. I do find it ultimately unbelievable because I just don’t think it explains the data, but it does if developed properly, have the view that it could actually explain why people find it unbelievable, and that would be a virtue in its favor.
David Chalmers of all people says that, even if he can’t conceive of how a deep reduction of Qualia might make their non-existence non-paradoxical, he might change his mind if he ever actually saw such a reduction! I say the same about epistemic and therefore normative facts. But crucially, no-one has solved this ‘meta problem’ for Qualia or for normative facts. There are partial hints of explanations for both, but there’s no full debunking argument that makes epistemic antirealism seem completely non-damaging and thus removes 2). I can’t imagine what such an account could look like, but the point of the ‘dissolving the question’ strategy is that it often isn’t imaginable in advance because your concepts are confused, so I’ll just leave that point. In the moral domain, the convergence arguments point against question-dissolving because they suggest the concept of normativity is solid and reliable. If those arguments fall, then question-dissolving looks more likely.
That’s one route. What of the other?
The alternative is to say that there are irreducible normative facts. This is counter-reductionist, counter-intuitive and strange. Two things that can make it less strange: these facts are not supposed to be intrinsically motivational (that violates the orthogonality thesis and is not permitted by the laws of physics) and they are not required to be facts about objects, like Platonic forms outside of time and space. They can be logical facts of the sort Eliezer talked about, but just a particular kind of logical fact, that has the property of being normative, the one you should follow. They don’t need to ‘exist’ as such. What epistemic facts would do is say certain reflective equilibria, certain arrangements of ‘reflecting on your own beliefs, using your current mind’ are the right ones, and others are the wrong ones. It doesn’t deny that this is the case:
So what I did in practice, does not amount to declaring a sudden halt to questioning and justification. I’m not halting the chain of examination at the point that I encounter Occam’s Razor, or my brain, or some other unquestionable. The chain of examination continues—but it continues, unavoidably, using my current brain and my current grasp on reasoning techniques. What else could I possibly use?
Indeed, no matter what I did with this dilemma, it would be me doing it. Even if I trusted something else, like some computer program, it would be my own decision to trust it.
Irreducible normativity just says that there is a meaningful, mind-independent difference between the virtuous and degenerate cases of recursive justification of your beliefs, rather than just ways of recursively justifying our beliefs that are… different.
If you buy that anti-realism is self-defeating, and think that we can know something about the normative domain via moral and non-moral convergence, then we have actual positive reasons to believe that normative facts are knowable (the convergence arguments help establish that moral facts aren’t and couldn’t be random things like stacking pebbles in prime-numbered heaps).
These two arguments are quite different—one is empirical (that our practical, epistemic and moral reasons tend towards agreement over time and after conceptual analysis and reflective justification) and the other is conceptual (that if you start out with normative concepts you are forced into using them).
Depending on which of the arguments you accept, there are four basic options. These are extremes of a spectrum, as while the Normativity argument is all-or-nothing, the Convergence argument can come by degrees for different types of normative claims (epistemic, practical and moral):
• Accept Convergence and Reject Normativity: prescriptivist anti-realism. There are (probably) no mind-independent moral facts, but the nature of rationality is such that our values usually cohere and are stable, so we can treat morality as a more-or-less inflexible logical ordering over outcomes. • Accept Convergence and Accept Normativity: moral realism. There are moral facts and we can know them • Reject Convergence and Reject Normativity: nihilist anti-realism. Morality is seen as a ‘personal life project’ about which we can’t expect much agreement or even within-person coherence • Reject Convergence and Accept Normativity: sceptical moral realism. Normative facts exist, but moral facts may not exist, or may be forever unknowable.
Even if what exactly normative facts are is hard to conceive, perhaps we can still know some things about them. Eliezer ended his post arguing for universalized, prescriptive anti-realism with a quote from HPMOR. Here’s a different quote:
“Sometimes,” Professor Quirrell said in a voice so quiet it almost wasn’t there, “when this flawed world seems unusually hateful, I wonder whether there might be some other place, far away, where I should have been. I cannot seem to imagine what that place might be, and if I can’t even imagine it then how can I believe it exists? And yet the universe is so very, very wide, and perhaps it might exist anyway? …
An extremely unscientific and incomplete list of people who fall into the various categories I gave in the previous post:
1.Accept Convergence and Reject Normativity: Eliezer Yudkowsky, Sam Harris (Interpretation 1), Peter Singer in The Expanding Circle, RM Hare and similar philosophers, HJPEV
2.Accept Convergence and Accept Normativity: Derek Parfit, Sam Harris (Interpretation 2), Peter Singer today, the majority of moral philosophers, Dumbledore
3.Reject Convergence and Reject Normativity: Robin Hanson, Richard Ngo (?), Lucas Gloor (?)most Error Theorists, Quirrell
4.Reject Convergence and Accept Normativity: A few moral philosophers, maybe Ayn Rand and objectivists?
The difference in practical, normative terms between 2), 4) and 3) is clear enough − 2 is a moral realist in the classic sense, 4 is a sceptic about morality but agrees that irreducible normativity exists, and 3 is a classic ‘antirealist’ who sees morality as of a piece with our other wants. What is less clear is the difference between 1) and 3). In my caricature above, I said Quirrell and Harry Potter from HPMOR were non-prescriptive and prescriptive anti-realists, respectively, while Dumbledore is a realist. Here is a dialogue between them that illustrates the difference.
Harry floundered for words and then decided to simply go with the obvious. “First of all, just because I want to hurt someone doesn’t mean it’s right—”
“What makes something right, if not your wanting it?”
“Ah,” Harry said, “preference utilitarianism.”
“Pardon me?” said Professor Quirrell.
“It’s the ethical theory that the good is what satisfies the preferences of the most people—”
“No,” Professor Quirrell said. His fingers rubbed the bridge of his nose. “I don’t think that’s quite what I was trying to say. Mr. Potter, in the end people all do what they want to do. Sometimes people give names like ‘right’ to things they want to do, but how could we possibly act on anything but our own desires?”
The relevant issue here is that Harry draws a distinction between moral and non-moral reasons even though he doesn’t believe in irreducible normativity. In particular, he’s committed to a normative ethical theory, preference utilitarianism, as a fundamental part of how he values things.
Here is another illustration of the difference. Lucas Gloor (3) explains the case for suffering-focussed ethics, based on the claim that our moral intuitions assign diminishing returns to happiness vs suffering.
While there are some people who argue for accepting the repugnant conclusion (Tännsjö, 2004), most people would probably prefer the smaller but happier civilization – at least under some circumstances. One explanation for this preference might lie in intuition one discussed above, “Making people happy rather than making happy people.” However, this is unlikely to be what is going on for everyone who prefers the smaller civilization: If there was a way to double the size of the smaller population while keeping the quality of life perfect, many people would likely consider this option both positive and important. This suggests that some people do care (intrinsically) about adding more lives and/or happiness to the world. But considering that they would not go for the larger civilization in the Repugnant Conclusion thought experiment above, it also seems that they implicitly place diminishing returns on additional happiness, i.e. that the bigger you go, the more making an overall happy population larger is no longer (that) important.
By contrast, people are much less likely to place diminishing returns on reducing suffering – at least17 insofar as the disvalue of extreme suffering, or the suffering in lives that on the whole do not seem worth living, is concerned. Most people would say that no matter the size of a (finite) population of suffering beings, adding more suffering beings would always remain equally bad.
It should be noted that incorporating diminishing returns to things of positive value into a normative theory is difficult to do in ways that do not seem unsatisfyingly arbitrary. However, perhaps the need to fit all one’s moral intuitions into an overarching theory based solely on intuitively appealing axioms simply cannot be fulfilled.
And what are those difficulties mentioned? The most obvious is the absurd conclusion—that scaling up a population can turn it from axiologically good to bad:
Hence, given the reasonable assumption that the negative value of adding extra lives with negative welfare does not decrease relatively to population size, a proportional expansion in the population size can turn a good population into a bad one—a version of the so-called “Absurd Conclusion” (Parfit 1984). A population of one million people enjoying very high positive welfare and one person with negative welfare seems intuitively to be a good population. However, since there is a limit to the positive value of positive welfare but no limit to the negative value of negative welfare, proportional expansions (two million lives with positive welfare and two lives with negative welfare, three million lives with positive welfare and three lives with negative welfare, and so forth) will in the end yield a bad population.
Here, then, is the difference—If you believe, as a matter of fact, that our values cohere and place fundamental importance on coherence, whether because you think that is the way to get at the moral truth (2) or because you judge that human values do cohere to a large degree for whatever other reason and place fundamental value on coherence (1), you will not be satisfied with leaving your moral theory inconsistent. If, on the other hand, you see morality as continuous with your other life plans and goals (3), then there is no pressure to be consistent. So to (3), focussing on suffering-reduction and denying the absurd conclusion is fine, but this would not satisfy (1).
I think that, on closer inspection, (3) is unstable—unless you are Quirrell and explicitly deny any role for ethics in decision-making, we want to make some universal moral claims. The case for suffering-focussed ethics argues that the only coherent way to make sense of many of our moral intuitions is to conclude a fundamental asymmetry between suffering and happiness, but then explicitly throws up a stop sign when we take that argument slightly further—to the absurd conclusion, because ‘the need to fit all one’s moral intuitions into an overarching theory based solely on intuitively appealing axioms simply cannot be fulfilled’. Why begin the project in the first place, unless you place strong terminal value on coherence (1)/(2) - in which case you cannot arbitrarily halt it.
I think the mountain analogy really is the center of the rationality anti-realist argument.
It’s very intuitive to think of us perceiving facts about e.g. epistemology as if gazing upon a mountain. There is a clean separation between us, the gazer, and that external mountain, which we perceive in a way that we can politely pretend is more or less directly. We receive rapid, rich data about it, through a sensory channel whose principles of operation we well understand and trust, and that data tends to cohere well together with everything else, except when sometimes it doesn’t but let’s not worry about that. Etc.
The rationality anti-realist position is that perceiving facts about epistemology is very little like looking at a mountain. I’m reminded of a Dennett quote about the quality of personal experience:
Just about every author who has written about consciousness has made what we might call the first-person-plural presumption: Whatever mysteries consciousness may hold, we (you, gentle reader, and I) may speak comfortably together about our mutual acquaintances, the things we both find in our streams of consciousness. And with a few obstreperous exceptions, readers have always gone along with the conspiracy.
This would be fine if it weren’t for the embarrassing fact that controversy and contradiction bedevil the claims made under these conditions of polite mutual agreement. We are fooling ourselves about something. Perhaps we are fooling ourselves about the extent to which we are all basically alike. Perhaps when people first encounter the different schools of thought on phenomenology, they join the school that sounds right to them, and each school of phenomenological description is basically right about its own members’ sorts of inner life, and then just innocently overgeneralizes, making unsupported claims about how it is with everyone.
So, the mountain disanalogy: sometimes there are things we have opinions about, and yet there is no clean separation between us and the thing. We don’t perceive it in a way that we can agree is trusted or privileged. We receive vague, sparse data about it, and the subject is plagued by disagreement, self-doubt, and claims that other people are doing it all wrong.
This isn’t to say that we should give up entirely, but it means that we might have to shift our expectations of what sort of explanation or justification we are “entitled” to. Everyone would absolutely love it if they could objectively dunk on all those other people who disagree with them, but it’s probably going to turn out that a thorough explanation will sound more like “here’s how things got the way they are” rather than “here’s why you’re right and everyone else is wrong.”
So, the mountain disanalogy: sometimes there are things we have opinions about, and yet there is no clean separation between us and the thing. We don’t perceive it in a way that we can agree is trusted or privileged. We receive vague, sparse data about it, and the subject is plagued by disagreement, self-doubt, and claims that other people are doing it all wrong.
This isn’t to say that we should give up entirely, but it means that we might have to shift our expectations of what sort of explanation or justification we are “entitled” to.
So this depends on two things—first, how likely (in advance of assessing the ‘evidence’) something like normative realism is, and then how good that evidence is (how coherent it is). If we have really good reasons in advance to think there’s ‘no separation between us and the thing’ then no matter how coherent the ‘thing’ is we have to conclude that while we might all be able to agree on what it is, it isn’t mind independent.
So, is it coherent, and is it mind-independent? How coherent it needs to be for us to be confident we can know it, depends on how confident we are that its mind-independent, and vice versa.
The argument for coherence comes in the form of convergence (not among people, to be clear, but among normative frameworks), but as you say that doesn’t establish its mind independent (it might give you some strong hint, though, if its really strongly consistent and coherent), and the argument that normativity is mind-independent comes from the normativity argument. These three posts deal with the difference between those two arguments and how strong they are, and how they interact:
It seems like there’s room for the theory of logical-inductor-like agents with limited computational resources, and I’m not sure if this has already been figured out. The entire trick seems to be that when you try to build a logical inductor agent, it’s got some estimation process for math problems like “what does my model predict will happen?” and it’s got some search process to find good actions, and you don’t want the search process to be more powerful than the estimator because then it will find edge cases. In fact, you want them to be linked somehow, so that the search process is never in the position of taking advantage of the estimator’s mistakes—if you, a human, are making some plan and notice a blind spot in your predictions, you don’t “take advantage” of yourself, you do further estimating as part of the search process.
The hard part is formalizing this handwavy argument, and figuring out what other strong conditions need to be met to get nice guarantees like bounded regret.
As an example, imagine a board that’s lodged directly by the wall (no other support structures). If you make it twice as wide, then it will be twice as stiff, but if you make it twice as thick, then it will be eight times as stiff. On the other hand, if you make it twice as long, it will be eight times more compliant.
In a similar way, different action parameters will have scaling exponents (or more generally, functions). So one way to decrease the risk of high-impact actions would be to make sure that the scaling exponent is bounded above by a certain amount.
Anyway, to even do this, you still need to make sure the agent’s model is honestly evaluating the scaling exponent. And you would still need to define this stuff a lot more rigorously. I think this idea is more useful in the case where you already have an AI with high-level corrigible intent and want to give it a general “common sense” about the kinds of experiments it might think to try.
So it’s probably not that useful, but I wanted to throw it out there.
Somewhat delighted to see that google scholar now includes direct links to PDFs when it can find them instead of making you figure out how to use a given journal website.
This has been true for years. At least six, I think? I think I started using Google scholar around when I started my PhD, and I do not recall a time when it did not link to pdfs.
The results of Bob Jacob’s LessWrong survey are quite interesting. It’s a pity the sample is so small.
The visualized results (link in his post) are univariate, but I would like to highlight some things:
49 out of 56 respondents identifying as “White”, 53 out of 59 respondents born male and 46 out of 58 identifying male cisgender 47 of 59 identifying as heterosexual (comparison: https://en.wikipedia.org/wiki/Demographics_of_sexual_orientation) 1 out of 55 working in a “blue collar” profession Most people identify as “left of center” in some sense. At the same time, 30 out of 55 identify as “libertarian”, but there were multiple answers allowed. 31 of 59 respondents think they are at least “upper middle class”; 22 of 59 think the family they were raised in was “upper middle class”. (Background: In social science surveys, wealthy people usually underestimate their position, and poor people overestimate it but to a lesser extent.)
I would not have guessed the left-of-center identification, and I would have slightly underestimated the share of male (cisgender).
I would not have guessed the left-of-center identification
If you have 9 people who identify as left-wing and 1 person who identifies as right-wing, many people will hysterically denounce the entire group as “extreme right”, based on the fact that the 1 person wasn’t banned.
Furthermore, if you have people who identify as left-wing, but don’t fully buy the current Twitter left-wing orthodoxy, they too will be denounced by some as “extreme right”.
The survey was non-binary. Your first claim does not distinguish extremes and moderates.
The survey was anonymous. You cannot ban anonymous people.
I see no reason why people should have overstated their leftishness.
If your statement is meant to explain why my perception differs from the result, it does not fit. My perception based on posts and comments would have been relatively more rightwing, less liberal / social democratic / green etc.
I don’t see where leftwing lesswrongers are denounced as rightwing extremists. In particular, I don’t see where this explains people identifying as leftwing in the survey.
My model is that in USA most intelligent people are left-wing. Especially when you define “left-wing” to mean the 50% of the political spectrum, not just the extreme. And there seem to be many Americans on Less Wrong, just like on most English-speaking websites.
(Note that I am not discussing here why this is so. Maybe the left-wing is inherently correct. Or maybe the intelligent people are just more likely to attend universities where they get brainwashed by the establishment. I am not discussing the cause here, merely observing the outcome.)
So, I would expect Less Wrong to be mostly left-wing (in the 50% sense). My question is, why were you surprised by this outcome?
I don’t see where leftwing lesswrongers are denounced as rightwing extremists.
For example, “neoreaction” is the only flavor of politics that is mentioned in the Wikipedia article about LessWrong. It does not claim that it is the predominant political belief, and it even says that Yudkowsky disagrees with them. Nonetheless, it is the only political opinion mentioned in connection with Less Wrong. (This is about making associations rather than making arguments.) So a reader who does not know how to read between the lines properly, might leave with an impression that LW is mostly right-wing. (Which is exactly the intended outcome, in my opinion.) And Wikipedia is not the only place where this game of associations is played.
“My model is that in USA most intelligent people are left-wing. Especially when you define “left-wing” to mean the 50% of the political spectrum, not just the extreme.”
I agree. (I assume that by political spectrum you refer to something “objective”?)
And there seem to be many Americans on Less Wrong, just like on most English-speaking websites.
Given the whole Bay-area thing, I would have expected a higher share. In the survey, 37 out of 60 say they are residing in the US.
So, I would expect Less Wrong to be mostly left-wing (in the 50% sense). My question is, why were you surprised by this outcome?
Having been in this forum for a while, my impressions based on posts and comments led me to believe that less than 50% of people on lessrong would say of themselves that they are on values 1-5 of 1-10 scale from left-wing to right-wing. In fact, 41⁄56 did so.
For example, “neoreaction” is the only flavor of politics that is mentioned in the Wikipedia article about LessWrong. It does not claim that it is the predominant political belief, and it even says that Yudkowsky disagrees with them. Nonetheless, it is the only political opinion mentioned in connection with Less Wrong. (This is about making associations rather than making arguments.) So a reader who does not know how to read between the lines properly, might leave with an impression that LW is mostly right-wing. (Which is exactly the intended outcome, in my opinion.) And Wikipedia is not the only place where this game of associations is played.
The wikipedia article, as far as I can see, explains in that paragraph where the neoreactionary movement originated. I don’t agree on the “intended outcome”, or rather, I do not see why I should believe that.
The wikipedia article, as far as I can see, explains in that paragraph where the neoreactionary movement originated.
It’s not true, though! The article claims: “The neoreactionary movement first grew on LessWrong, attracted by discussions on the site of eugenics and evolutionary psychology”.
I mean, okay, it’s true that we’ve had discussions on eugenics and evolutionary psychology, and it’s true that a few of the contrarian nerds who enthusiastically read Overcoming Bias back in the late ’aughts were also a few of the contrarian nerds who enthusiastically read Unqualified Reservations. But “first grew” (Wikipedia) and “originated” (your comment) really doesn’t seem like a fair summary of that kind of minor overlap in readership. No one was doing neoreactionary political theorizing on this website. Okay, I don’t have a exact formalization of what I mean by “no one” in the previous sentence because I haven’t personally read and remembered every post in our archives; maybe there are nonzero posts with nonnegative karma that could be construed to match this description. Still, in essence, you can only make the claim “true” by gerrymandering the construal of those words.
Interesting. I had maybe read the Wikipedia article a long time ago, but it did not leave any impression in my memory. Now rereading it, I did not find it dramatic, but I see your point.
Tbh, I stilĺ do not fully understand how Wikipedia works (that is, I do not have a model who determines how an article develops). And the “originated” (ok maybe that is only almost and not fully identical to “first grew”) is just what I got from the article. The problem with the association is that it is hard to definitely determine what even makes things mentionable, but once somebody publibly has to distance himself from something, this indicates a public kind of association.
Further reading the article, my impression is that it indeed cites things that in Wikipedia count as sources for its claims. If the impression of lesswrong is distorted, then this may be a problem of what kinds of thing on lesswrong are covered by media publications? Or maybe it is all just selective citing, but then it should be possible to cite other things.
In theory, Wikipedia strives to be impartial. In practice, the rules are always only as good as the judges who uphold them. (All legal systems involve some degree of human judgment somewhere in the loop, because it is impossible to write a set of rules that covers everything and doesn’t allow some clever abuse. That’s why we talk about the letter and the spirit of the law.)
How to become a Wikipedia admin? You need to spend a lot of time editing Wikipedia in a way other admins consider helpful, and you need to be interested in getting the role. (Probably a few more technical details I forgot.) The good thing is that by doing a lot of useful work you send a costly signal that you care about Wikipedia. The bad thing is that if certain political opinion becomes dominant among the existing admins, there is no mechanism to fix this bias; it’s actually the other way round, because edits disagreeing with the consensus would be judged as harmful, and would probably disqualify their author from becoming an admin in the future.
I don’t assume bad faith from most of Wikipedia editors. Being wrong about something feels the same from inside as being right; and if other people agree with you, that is usually a good sign. But if you have a few bad actors who can play it smart, who can pretend that their personal grudges are how they actually see the world… considering that other admins already see them as part of the same team, and the same political bias means they already roughly agree on who are the good guys and who are the bad guys… it is not difficult to defend their decisions in front of jury of their peers. An outsider has no chance in this fight, because the insider is fluent with local lingo. Whatever they want to argue, they can find a wiki-rule pointing in that direction; of course it would be just as easy for them to find a wiki-rule pointing in the opposite direction (e.g. if you want to edit an article about something you are personally involved with, you have a “conflict of interest”, which is a bad thing; if I want to do the same thing, my personal involvement makes me a “subject-matter expert”, which is a good thing; your repetitive editing of the article to make your point is “vandalism”, my repetitive editing of the article to make an opposite point is “reverting vandalism”); and then the other admins will nod and say: “of course, if this is what the wiki-rules say, our job is to obey them”.
The specific admin that is so obsessed with Less Wrong is David Gerard from RationalWiki. He keeps a grudge for almost a decade, when he added Less Wrong to his website as an example of pseudoscience, mostly because of the quantum physics sequence. After being explained that actually “many worlds” is one of the mainstream interpretations among the scientists, he failed to say oops, and continued in the spirit of: well, maybe I was technically wrong about the quantum thing, but still… and spent the last decade trying to find and document everything that is wrong with Less Wrong. (Roko’s Basilisk—a controversial comment that was posted on LW once, deleted by Eliezer along with the whole thread, then posted on RationalWiki as “this is what people at Less Wrong actually believe”. Because the fact that it was deleted is somehow a proof that deep inside we actually agree with it, but we don’t want the world to know. Neoreaction—a small group of people who enjoyed debating their edgy beliefs on Less Wrong, were considered entertaining for a while, then became boring and were kicked out. Again, the fact that they were not kicked out sooner is evidence of something dark.) Now if you look who makes most edits on the Wikipedia page about Less Wrong: it’s David Gerard. If you go through the edit history and look at the individual changes, most of them are small and innocent, but they are all in the same direction: the basilisk and neoreaction must remain in the article, no matter how minuscule they are from perspective of someone who actually reads Less Wrong; on the other hand, mentions of effective altruism must be kept as short as possible. All of this is technically true and defensible, but… I’d argue that the Less Wrong described by the Wikipedia article does not resemble the Less Wrong its readers know, and that we have David Gerard and his decade-long work to thank for this fact.
If the impression of lesswrong is distorted, then this may be a problem of what kinds of thing on lesswrong are covered by media publications?
True, but most of the information in media originates from RationalWiki, where it was written by David Gerard. A decade ago, RationalWiki used to be quite high in google rankings, if I remember correctly; any journalist who did a simple background check would find it. Then he or she would ask about the juicy things in the interview, and regardless of the answer, the juicy things would be mentioned in the article. Which means that the next journalist would now find them both at RationalWiki and in the previous article, which means that he or she would again make a part of the interview about it, reinforcing the connection. It is hard to find an article about Less Wrong that does not mention Roko’s Basilisk, despite the fact that it is discussed here rarely, and usually in the context of “guys, I have read about this thing called Roko’s Basilisk in the media, and I can’t find anything about it here, could you please explain me what this is about?”
Part of this is the clickbait nature of media: given the choice between debating neoreaction and debating technical details of the latest decision theory, it doesn’t matter which topic is more relevant to Less Wrong per se, they know that their audience doesn’t care about the latter. And part of the problem with Wikipedia is that it is downstream of the clickbait journalism. They try to use more serious sources, but sometimes there is simply no other source on the topic.
Thanks for the history overview! Very interesting. Concerning the wikipedia dynamics, I agree that this is plausible, as it is a plausible development of nearly every volunteer organization, in particular if they try to be grassroots-democratic. The wikipedia-media problem is known (https://xkcd.com/978/) though in this particular case I was a bit surprised about the “original research” and “reliable source” distinction. Many articles there did not seem very “serious”. On the other hand, during this whole “lost in hyperspace”, I also found “A frequent poster to LessWrong was Michael Anissimov, who was MIRI’s media director until 2013.”(https://splinternews.com/the-strange-and-conflicting-world-views-of-silicon-vall-1793857715) which was news to me. In internet years, all this is so long ago that I did not have any such associations. (I would rather have expected lesswrong to be notable for demanding the dissolution of the WHO, but probably that is not yet clickbaity enough.)
It’s happened again: I’ve realized that one of my old beliefs (pre-LW) is just plain dumb.
I used to look around at all the various diet (Paleo, Keto, low carb, low fat, etc.) and feel angry at people for having such low epistemic standards. Like, there’s a new theory of nutrition every two years, and people still put faith in them every time? Everybody swears by a different diet and this is common knowledge, but people still swear by diets? And the reasoning is that “fat” (the nutrient) has the same name as “fat” (the body part people are trying to get rid of)?
Then I encountered the “calories in = calories out” theory, which says that the only thing you need to do to lose weight is to make sure that you burn more calories than you eat.
And I thought to myself, “yeah, obviously.”.
Because, you see, if the orthodox asserts X and the heterodox asserts Y, and the orthodox is dumb, then Y must be true!
Anyway, I hadn’t thought about this belief in a while, but I randomly remembered it a few minutes ago, and as soon as I remembered its origins, I chucked it out the window.
Oops!
(PS: I wouldn’t be flabbergasted if the belief turned out true anyway. But I’ve reverted my map from the “I know how the world is” state to the “I’m awaiting additional evidence” state.)
Then I encountered the “calories in = calories out” theory, which says that the only thing you need to do to lose weight is to make sure that you burn more calories than you eat.
And I thought to myself, “yeah, obviously.”.
On one hand, yeah, obviously.
On the other hand, “burning calories” is not an elementary action. Suppose I tell you to burn 500 calories now; how exactly would you achieve it? If your answer is that you would exercise or do something physically demanding, such actions spend ATP in the cells, so what if you don’t have enough ATP in your muscle cells; what is your plan B for burning calories? From the opposite side, you can limit the intake of calories. What if your metabolism is such that if you don’t provide enough calories, you will gradually fall in coma and die.
Your metabolism can make it impossible for you to reduce the “calories in” or increase the “calories out”, if it is somehow set up in a way that does not convert the calories into useful energy in your muscle cells, and it starts damaging your organs if the calories are missing in general. So while the theory is almost tautologically true, it may still be impossible to use it to lose weight. And the problem is that the proponents of “calories in = calories out” usually smugly pretend that it is an actionable advice, instead of mere description.
The actionable advice needs to be about how the metabolism works. And the things that impact it, such as what you eat, and who knows what else. Also, if you have some hormonal imbalance, or alergy, or whatever, your metabolism may differ from other people even if you eat the same things and try to live the same lifestyle. So, while e.g. eating less refined sugar would probably help everyone, no advice would guarantee a perfect outcome for everyone.
You make a good point—even if my belief was technically true, it could still have been poorly framed and inactionable (is there a name for this failure mode?).
But in fact, I think it’s not even obvious that it was technically true. If we say “calories in” is the sum of the calorie counts on the labels of each food item you eat (let’s assume the labels are accurate) then could there not still be some nutrient X that needs to be present for your body to extract the calories? Say, you need at least an ounce of X to process 100 calories? If so, then one could eat the same amount of food, but less X, and potentially lose weight.
Or perhaps the human body can only process food between four and eight hours after eating it, and it doesn’t try as hard to extract calories if you aren’t being active, so scheduling your meals to take place four hours before you sit around doing nothing would make them “count less”.
Calories are (presumably?) a measure of chemical potential energy, but remember that matter itself can also be converted into energy. There’s no antimatter engine inside my gut, so my body fails to extract all of the energy present in each piece of food. Couldn’t the mechanism of digestion also fail to extract all the chemical potential energy of species “calorie”?
Yes, there are many steps in the metabolism that are not under your conscious control. I am not an expert, so I don’t want to speculate too much about the technical details, but I think that gut bacteria probably also play a role. Simply, not everything you put in your mouth ends up necessarily in your bloodstream, and not everything that you absorbed is necessarily available in form of muscle energy.
is there a name for this failure mode?
I don’t know any standard name. Seems to me the problem is confusing “rephrasing of the desired outcome” with “an algorithm that actually get you there”. Something like:
Q: How can I get lot of money?
A: Become a millionaire!
Like, yeah, technically, everyone who successfully followed this advice ended up with lots of money, and everyone who didn’t can be accused of not following the advice properly, but that’s simply because those are two ways to describe the same thing.
Q: How can I lose weight?
A: Get rid of the extra atoms! I mean, extra calories!
Charitably, the advice is not absolutely bad, because for a hypothetical completely clueless listener it would provide some little information. But then, using this advice in practice means insinuating that your target is completely clueless, which is probably not be the case.
I want to give a big thumbs up of positive reinforcement. I thinks its great that I got to read an “oops! That was dumb, but now I’ve changed my mind.”
Thanks for the feedback! Here’s another one for ya. A relatively long time ago I used to be pretty concerned about Pascal’s wager, but then I devised some clever reasoning why it all cancels out and I don’t need to think about it. I reasoned that one of three things must be true:
I don’t have an immortal soul. In this case, I might as well be a good person.
I have an immortal soul, and after my bodily death I will be assigned to one of a handful of infinite fates, depending on how good of a person I was. In this case it’s very important that I be a good person.
Same as above, but the decision process is something else. In this case I have no way of knowing how my infinite fate will be decided, so I might as well be a good person during my mortal life and hope for the best.
But then, post-LW, I realized that there are two issues with this:
It doesn’t make any sense to separate out case 2 from the enormous ocean of possibilities allowed for by case 3. Or rather, I can separate it, but then I need to probabilistically penalize it relative to case 3, and I also need to slightly shift the “expected judgment criterion” found in case 3 away from “being a good person is the way to get a good infinite fate”, and it all balances out.
More importantly, this argument flippantly supposes that I have no way of discerning what process, if any, will be used to assign me an infinite fate. An infinite fate, mind you. I ought to be putting in more thought than this even if I thought the afterlife only lasted an hour, let alone eternity.
So now I am back to being rather concerned about Pascal’s wager, or more generally, the possibility that I have an immortal soul and need to worry about where it eventually ends up.
From my first read-through of the sequences I remember that it claims to show that the idea of there being a god is somewhat nonsensical, but I didn’t quite catch it the first time around. So my first line of attack is to read through the sequences again, more carefully this time, and see if they really do give a valid reason to believe that.
From my first read-through of the sequences I remember that it claims to show that the idea of there being a god is somewhat nonsensical, but I didn’t quite catch it the first time around.
I think the first step is to ask yourself what do you even mean by saying “god”.
Because if you have a definition like “the spirit of our dead chieftain, who sometimes visits me in my dreams”, I have no problem with that. Like, I find it completely plausible that you see the image of the chieftain in your dreams; nothing unscientific about that. It’s just that the image is generated by your brain, so if try to communicate with it, it can only give you advice that your brain generated, and it can only grant your wishes in the sense that you accomplish that yourself. But if you agree that this is exactly what you meant, then such god is perfectly okay for me.
But the modern definition is more like: an intelligent being that exists outside of our universe, but can observe it and change it. Suspending disbelief for a moment; how exactly can a being be “intelligent”, whether inside our universe or not? Intelligence implies processing data. That would imply that the god has some… supernatural neurons, or some other architecture capable of processing data. So the god is a mechanism (in the sense that a human is a biological mechanism) composed of parts, although those parts do not necessarily have to be found in our physics. Skill kinda plausible, maybe gods are made out of dark matter, who knows. But then, how did this improbably complicated mechanism come into existence? Humans were made by evolution, were gods too? But then again those gods are not the gods of religion; they are merely powerful aliens. But powerful aliens are neither creators of the universe, nor are they omniscient.
Then the theologists try to come with some smart-sounding arguments, like “god is actually supremely simple”—therefore Occam’s razor does not disprove him, and no evolution was needed, because simple things are more likely a priori, so a supreme being is supremely likely. Or something like that. But this is nonsense, because “supremely simple” and “capable of processing information” are incompatible.
The last argument would be: okay, we do not have a coherent explanation of god, but you don’t have a coherent explanation how the universe can exist without one. And a potential answer is that maybe existence is relative. Like, if you run a simulation of Conway’s Game of Life, it could possibly contain evolved intelligent life, and that life would ask “how was our universe created”? And it’s not like you created the universe by simulating it, because you are merely following the mathematical rules; so it’s more like the math created that universe and you are only observing it. If the beings in that mathematical universe will pray to gods, there is no way for anyone outside to intervene (while simultaneously following the mathematical rules). So the universe inside the Game of Life is a perfectly godless universe, based on math. But does it exist? Well, that rules of math describe the intelligent beings inside that universe that live, think, interact with their world. Maybe that’s all that is needed for existence; or rather, the question is what else could be needed? And then, maybe our world is also just an implication of some mathematical equations.
tl;dr—maybe math itself implies existence, and if you try to turn the concept of (intelligent, preceiving, acting) god into something coherent it is no longer the god of religion but merely a powerful alien
That would imply that the god has some… supernatural neurons, or some other architecture capable of processing data.
The supernatural isn’t supposed to be the natural done all over again. The typical theological claim is that God’s wisdom or whatever is an intrinisic quality, not something with moving parts.
Well, “wisdom as an intrinsic quality” is a mysterious answer. And what is “wisdom without moving parts”? A precomputed table of wise answers to all possible questions? Who precomputed it and how?
I agree that this is how theology usually answers it, but it is an answer that doesn’t make any sense when you look at it closer; it’s just some good-sounding words randomly glued together. And if you try to make it refer to something, even a hypothetical something, the whole explanation falls apart.
Reductionism is a combination of three claims.
That many thing are made of smaller components
That those particular can be understood in terms of the operations of their components
There’s an irreduciby basic level. Its not turtles all way down.
If it’s always the case that something that isn’t explicable in terms of its parts is mysterious, then the lowest level us mysterious. If nothing is mysterious, if you apply the argument against mysterious answer without prejudice,
reductionism is false. There isn’t a consistent set of principles here.
Continued ..
Naturalism is the claim that there is a bunch of fundamental properties that just are, at the bottom of the stack ,and everything is built up from that. Supernaturalism is the claim that the intrinsic stuff is at the top of the stack, and everything else is derived from it top-down. That may be 100% false , but it is the actual claim.
There’s a thing called the principle of charity , where you one party interprets the others statements so as to maximise their truth value. This only enhances communication if the truth is not basically in dispute...that’s the easy case. The hard case is when there is a basic dispute about whats true. In that case, it’s not helpful to fix the other person’s claims by making them more reasonable from your point of view.
Anyway, thats how we ended up with “God must have superneurons in his superbrain”.
Feels like in the top-down universe, science shouldn’t work at all. I mean, when you take a magnifying glass and look at the details, they are supposedly generated on the fly to fit the larger picture. Then you apply munchkinry to the details and invent an atomic bomb or quantum computer… which means… what exactly, from the top-down perspective?
Yeah, you can find an excuse, e.g. that some of those top-down principles are hidden like Easter eggs, waiting to be discovered later. That the Platonic idea of smartphones has been waiting for us since the creation of the universe, but was only revealed to the recent generation. Which would mean that the top-down universe has some reason to pretend to be bottom-up, at least in some aspects...
Okay, the same argument could be made that quantum physics pretends to be classical physics at larger scale, or relativity pretends to be Newtonian mechanics at low speeds… as if the scientists are trying to make up silly excuses for why their latest magic works but totally “doesn’t contradict” what the previous generations of scientists were telling us...
Well, at least it seems like the bottom-up approach is fruitful, whether the true reason is that the universe is bottom-up, or that the universe it top-down in a way that tries really hard to pretend that it is actually bottom-up (either in the sense that when it generates the—inherently meaningless—details for us, it makes sure that all consequences of those details are compatible with the preexisting Platonic ideas that govern the universe… or like a Dungeon Master who allows the players to invent all kinds of crazy stuff and throw the entire game off balance, because he values consistency above everything).
More importantly, in universe where there is magic all the way up, what sense does it make to adopt the essentially half-assed approach, where you believe in the supernatural but also kinda use logic except not too seriously… might as well throw the logic away completely, because in that kind of universe it is not going to do you much good anyway.
The basic claim of a top down universe is a short string that doesn’t contain much information. About the same amount as a basic claim of reductionism.
The top down claim doesnt imply a universe of immutable physical law, but it doesn’t contradict it either.
The same goes for the bottom-up claim. A universe of randomly moving high entropy gas is useless for science and technology, but compatible with reductionism.
But all this is rather beside the point. Even if supernaturalism is indefensible, you can’t refute it by changing it into something else.
I wouldn’t call the dead chieftain a god—that would just be a word game.
But then, how did this improbably complicated mechanism come into existence? Humans were made by evolution, were gods too? But then again those gods are not the gods of religion; they are merely powerful aliens. But powerful aliens are neither creators of the universe, nor are they omniscient.
Wait wait! You say a god-like being created by evolution cannot be a creator of the universe. But that’s only true if you constrain that particular instance of evolution to have occered in *this* universe. Maybe this universe is a simulation designed by a powerful “alien” in another universe, who itself came about from an evolutionary process in its own universe.
It might be “omniscient” in the sense that it can think 1000x as fast as us and has 1000x as much working memory and is familiar with thinking habits that are 1000x as good as ours, but that’s a moot point. The real thing I’m worried about isn’t whether there exists an omniscient-omnipotent-benevolent creature, but rather whether there exists *some* very powerful creature who I might need to understand to avoid getting horrible outcomes.
I haven’t yet put much thought into this, since I only recently came to believe that this topic merits serious thought, but the existence of such a powerful creature seems like a plausible avenue to the conclusion “I have an infinite fate and it depends on me doing/avoiding X”.
[...] Occam’s razor [...]
This is another area where my understanding could stand to be improved (and where I expect it will be during my next read-through of the sequences). I’m not sure exactly what kind of simplicity Occam’s razor uses. Apparently it can be formalized as Kolmogorov complexity, but the only definition I’ve ever found for that term is “the Kolmogorov Complexity of X is the length of the shortest computer program that would output X”. But this definition is itself in need of formalization. Which programming language? And what if X is something other than a stream of bits, such as a dandelion? And even once that’s answered, I’m not quite sure how to arrive at the conclusion that Kolmogorov-ly simpler things are more likely to be encountered.
(All that being said, I’d like to note that I’m keeping in mind that just because I don’t understand these things doesn’t mean there’s nothing to them. Do you know of any good learning resources for someone who has my confusions about these topics?)
And it’s not like you created the universe by simulating it, because you are merely following the mathematical rules; so it’s more like the math created that universe and you are only observing it.
If the beings in that mathematical universe will pray to gods, there is no way for anyone outside to intervene (while simultaneously following the mathematical rules). So the universe inside the Game of Life is a perfectly godless universe, based on math.
That much makes sense, but I think it excludes a possibly important class of universe that is based on math but also depends on a constant stream of data from an outside source. Imagine a Life-like simulation ruleset where the state of the array of cells at time T+1 depended on (1) the state of the array at time T and (2) the on/off state of a light switch in my attic at time T. I could listen to the prayers of the simulated creatures and use the light switch to influence their universe such that they are answered.
I wouldn’t call the dead chieftain a god—that would just be a word game.
Some people in history did, specifically, ancient Romans. But now we don’t. Just making it obvious.
You say a god-like being created by evolution cannot be a creator of the universe. But that’s only true if you constrain that particular instance of evolution to have occered in *this* universe. Maybe this universe is a simulation designed by a powerful “alien” in another universe, who itself came about from an evolutionary process in its own universe.
And this is similar to the chieftain thing. You can have a religion that defines “god” as “an alien from another universe”, but many religions insist that god is not created but eternal.
The real thing I’m worried about isn’t whether there exists an omniscient-omnipotent-benevolent creature, but rather whether there exists *some* very powerful creature who I might need to understand to avoid getting horrible outcomes.
Yes, this is a practical approach. Well, you cannot disprove such thing, because it is logically possible. (Obviously, “possible” does not automatically imply “it happened”.) But unless you assume it is “simulations all the way up”, there must be a universe that is not created by an external alien lifeform. Therefore, it is also logically possible that our universe is like that.
There is no reason to assume that the existing religions have something in common with the simulating alien. When I play Civilization, the “people” in my simulation have a religion, but it doesn’t mean that I believe in it, or even approve it, or that I am somehow going to reward them for it.
It’s just a cosmic horror that you need to learn to live with. There are more.
the only definition I’ve ever found for that term is “the Kolmogorov Complexity of X is the length of the shortest computer program that would output X”. But this definition is itself in need of formalization. Which programming language?
Any programming language; for large enough values it doesn’t matter. If you believe that e.g. Python is much better in this regard than Java, then for sufficiently complicated things the most efficient way to implement them in Java is to implement a Python emulator (a constant-sized piece of code) and implementing the algorithm in Python. So if you chose a wrong language, you pay at most a constant-sized penalty. Which is usually irrelevant, because these things are usually applied to debating what happens in general when the data grow.
But I agree that if you talk about small data, it is underspecified. I don’t really know what could it mean to “have a universe defined by the following three bits: 0, 0, 1”, and maybe no one has a meaningful answer to this. But there are cases where you can have an intuition that for any reasonable definition of a programming language, X should be simpler than Y.
I’m not quite sure how to arrive at the conclusion that Kolmogorov-ly simpler things are more likely to be encountered.
Just an intuition pump: Imagine that there is a multiverse A containing all universes that can be run by programs having exactly 100 lines of code; and there is a multiverse B containing all universes that can be run by programs having exactly 105 lines of code. The universes from A are more likely, because they also appear in B.
For each program A1 describing a universe from A, you have a set of programs describing exactly the same universe in B, simply by adding an “exit” instruction on line 101, and arbitrary instructions on lines 102-105. If we take a program B1 describing a universe from B in such way that each of the 105 lines is used meaningfully, then… in multiverse A we have one A1 vs zero B1, and in multiverse B we have many A1-equivalents vs one B1; in either case A1 wins.
Do you know of any good learning resources for someone who has my confusions about these topics?
A part of this is in standard computer science curriculum, and another part is a philosophical extrapolation. I do not have a recommendation about a specific textbook, I just vaguely remember things like this from university. (Actually, I probably never heard explicitly about Kolmogorov complexity at university, but I learned some related concepts that allowed me to recognize what it means and what it implies, when I found it on Less Wrong.)
Imagine a Life-like simulation ruleset where the state of the array of cells at time T+1 depended on (1) the state of the array at time T and (2) the on/off state of a light switch in my attic at time T. I could listen to the prayers of the simulated creatures and use the light switch to influence their universe such that they are answered.
Then I would say the Kolmogorov complexity of the “Life-only” universe is the complexity of the rules of Life; but the complexity of the “Life+inputs” universe is the complexity of the rules plus the complexity of whoever generates the inputs, and everything they depend on, so it is the complexity of the outside universe.
The Civ analogy makes sense, and I certainly wouldn’t stop at disproving all actually-practiced religions (though at the moment I don’t even feel equipped to do that).
Well, you cannot disprove such thing, because it is logically possible. (Obviously, “possible” does not automatically imply “it happened”.) But unless you assume it is “simulations all the way up”, there must be a universe that is not created by an external alien lifeform. Therefore, it is also logically possible that our universe is like that.
Are you sure it’s logically possible in the strict sense? Maybe there’s some hidden line of reasoning we haven’t yet discovered that shows that this universe isn’t a simulation! (Of course, there’s a lot of question-untangling that has to happen first, like whether “is this a simulation?” is even an appropriate question to ask. See also: Greg Egan’s book Permutation City, a fascinating work of fiction that gives a unique take on what it means for a universe to be a simulation.)
It’s just a cosmic horror that you need to learn to live with. There are more.
This sounds like the kind of thing someone might say who is already relatively confident they won’t suffer eternal damnation. Imagine believing with probability at least 1/1000 that, if you act incorrectly during your life, then...
(WARNING: graphic imagery) …upon your bodily death, your consciousness will be embedded in an indestructible body and put in a 15K degree oven for 100 centuries. (END).
Would you still say it was just another cosmic horror you have to learn to live with? If you wouldn’t still say that, but you say it now because your probability estimate is less than 1/1000, how did you come to have that estimate?
Any programming language; for large enough values it doesn’t matter. If you believe that e.g. Python is much better in this regard than Java, then for sufficiently complicated things the most efficient way to implement them in Java is to implement a Python emulator (a constant-sized piece of code) and implementing the algorithm in Python. So if you chose a wrong language, you pay at most a constant-sized penalty. Which is usually irrelevant, because these things are usually applied to debating what happens in general when the data grow.
The constant-sized penalty makes sense. But I don’t understand the claim that this concept is usually applied in the context of looking at how things grow. Occam’s razor is (apparently) formulated in terms of raw Kolmogorov complexity—the appropriate prior for an event X is 2^(-B), where B is the Kolmogorov Complexity of X.
Let’s say general relativity is being compared against Theory T, and the programming language is Python. Doesn’t it make a huge difference whether you’re allowed to “pip install general-relativity” before you begin?
But there are cases where you can have an intuition that for any reasonable definition of a programming language, X should be simpler than Y.
I agree that these intuitions can exist, but if I’m going to use them, then I detest this process being called a formalization! If I’m allowed to invoke my sense of reasonableness to choose a good programming language to generate my priors, why don’t I instead just invoke my sense of reasonableness to choose good priors? Wisdom of the form “programming languages that generate priors that work tend to have characteristic X” can be transformed into wisdom of the form “priors that work tend to have characteristic X”.
Just an intuition pump: [...]
I have to admit that I kind of bounced off of this. The universe-counting argument makes sense, but it doesn’t seem especially intuitive to me that the whole of reality should consist of one universe for each computer program of a set length written in a set language.
(Actually, I probably never heard explicitly about Kolmogorov complexity at university, but I learned some related concepts that allowed me to recognize what it means and what it implies, when I found it on Less Wrong.)
Can I ask which related concepts you mean?
[...] so it is the complexity of the outside universe.
Oh, that makes sense. In that case, the argument would be that nothing outside MY universe could intervene in the lives of the simulated Life-creatures, since they really just live in the same universe as me. But then my concern just transforms into “what if there’s a powerful entity living in this universe (rather than outside of it) who will punish me if I do X, etc”.
Maybe there’s some hidden line of reasoning we haven’t yet discovered that shows that this universe isn’t a simulation!
I find it difficult to imagine how such argument could even be constructed. “Our universe isn’t a simulation because it has property X” doesn’t explain why the simulator could not simulate X. The usual argument is “because quantum stuff, the simulation would require insane amounts of computing power”, which is true, but we have no idea what the simulating universe looks like, and what kind of physics is has… maybe what’s an insane amount for us is peanuts for them.
But maybe there is some argument why a computing power in principle (like, some mathematical reason) cannot exceed certain value, ever. And the value may turn out to be insufficient to simulate our universe. And we can somehow make sure that our entire universe is simulated in sufficient resolution (not something like: the Earth or perhaps the entire Solar system is simulated in full quantum physics, but everything else is just a credible approximation). Etc. Well, if such thing happens, then I would accept the argument.
This sounds like the kind of thing someone might say who is already relatively confident they won’t suffer eternal damnation. Imagine believing with probability at least 1/1000 that, if you act incorrectly during your life, then...
Yeah, I just… stopped worrying about these kinds of things. (In my case, “these kinds of things” refer e.g. to very unlikely Everett branches, which I still consider more likely than gods.) You just can’t win this game. There are million possible horror scenarios, each of them extremely unlikely, but each of them extremely horrifying, so you would just spend all your life thinking about them; and there is often nothing you can do about them, in some cases you would be required to do contradictory things (you spend your entire life trying to appease the bloodthirsty Jehovah, but it turns out the true master of universe is the goddess Kali and she is very displeased with your Christianity...) or it could be some god you don’t even know because it is a god of Aztecs, or some future god that will only be revealed to humanity in year 3000. Maybe humans are just a precursor to an intelligent species that will exist million years in the future, and from their god’s perspective humans are even less relevant than monkeys are for Christianity. Maybe we are just meant to be food for the space locusts from Andromeda galaxy. Maybe our entire universe is a simulation on a computer in some alien universe with insane computer power, but they don’t care about the intelligent beings or life in general; they just use the flashing galaxies as a screen saver when they are bored. If you put things into perspective, assigning probability 1/1000 to any specific religion is way too much; all kinds of religions, existing and fictional put together, don’t deserve that much.
And by the way, torturing people forever, because they did not believe in your illogical incoherent statements unsupported by evidence, that is 100% compatible with being an omnipotent, omniscient, and omnibenevolent god, right? Yet another theological mystery...
On the other hand, if you assume an evil god, then… maybe the holy texts and promises of heaven are just a sadistic way he is toying with us, and then he will torture all of us forever regardless.
So… you can’t really win this game. Better to focus on things where you actually can gather evidence, and improve your actual outcomes in life.
Psychologically, if you can’t get rid of the idea of supernatural, maybe it would be better to believe in an actually good god. Someone who is at least as reasonable and good as any of us, which should not be an impossibly high standard. Such god certainly wouldn’t spend an eternity torturing random people for “crimes” such as being generally decent people but believing in a wrong religion or no religion at all, or having extramarital sex, etc. (Some theologians would say that this is actually their god. I don’t think so, but whatever.) I don’t really believe in such god either, honestly, but it is a good fallback plan when you can’t get rid of the idea of gods completely.
Let’s say general relativity is being compared against Theory T, and the programming language is Python. Doesn’t it make a huge difference whether you’re allowed to “pip install general-relativity” before you begin?
That would be cheating, obviously. Unless by the length of code you mean also the length of all used libraries, in which case it is okay. It is assumed that the original programming language does not favoritize any specific theory, just provides very basic capabilities for expressing things like “2+2” or “if A then B else B”. (Abstracting from details such as how many pluses are in one “if”.)
If I’m allowed to invoke my sense of reasonableness to choose a good programming language to generate my priors, why don’t I instead just invoke my sense of reasonableness to choose good priors?
Yeah, ok. The meaning of all this is, how can we compare “complexity” of two things where neither is a strict subset of the other. The important part is that “fewer words” does not necessarily mean “smaller complexity”, because it allows obvious cheating (invent a new word that means exactly what you are arguing for, and then insist that your theory is really simple because it could be described by one word—the same trick as with importing a library), but even if it is not your intention to cheat, your theory can still benefit from some concepts having shorter words for historical reasons, or even because words related to humans (or life on Earth in general) are already shorter, so you should account for the complexity that is already included for them. Furthermore it should be obvious to you that “1000 different laws of physics” is more complex than “1 law of physics, applied in the same way to 1000 particles”. If this all is obvious to you, than yes, making the analogy to a programming language does not bring any extra value.
But historical evidence shows that humans are quite bad at this. They will insist that stars are just shining dots in the sky instead of distant solar systems, because “one solar system + thousands of shining dots” seems to them less complex than “thousands of solar systems”. They will insist that Milky Way or Andromeda cannot be composed of stars, because “thousand stars + one Milky Way + one Andromeda” seems to them less complex than “millions of stars”. More recently (and more controversially), they will insist that “quantum physics + collapse + classical physics” is less complex than “quantum physics all the way up”. Programming analogy helps to express this in a simple way: “Complexity is about the size of code, not how large values are stored in the variables.”
Can I ask which related [to Kolmogorov complexity] concepts you mean?
Compression (lossless), like in the “zip” files. Specifically the fact that an average random file cannotbe compressed, no matter how smart is the method used. Like, for any given value N, the number of all files with size N is larger than the number of all files with size smaller than N. So, whatever is your compression method, if it is lossless, it needs to be a bijection, so there must be at least one file of size N that it will not compress into a file of smaller size. Even worse, for each file of size N compressed to a smaller size, there must be a file of size smaller than N that gets “compressed” to a file of size N or more.
So how does compression work in real world? (Because experience shows it does.) It ultimately exploits the fact that most of the files we want to compress are not random, so it is designed to compress non-random files into smaller ones, and random (containing only noise) files into slightly larger ones. Like, you can try it at home; generate a one megabyte file full of randomly generated bits, then try all compression programs you have installed, and see what happens.
Now, each specific compression algorithm recognized and exploits some kinds of regularity, and is blind towards others. This is why people sometimes invent better compression algorithms that exploit more regularities. The question is, what is the asymptote of this progress? If you tried to invent the best compression algorithm ever, one that could exploit literally all kinds of non-randomness, what would it look like?
(You may want to think about this for a moment, before reading more.)
The answer is that the hypothetical best compression algorithm ever would transform each file into the shortest possible program that generates this file. There is only one problem with that: finding the shortest possible program for every input is algorithmically impossible—this is related to halting problem. Regardless, if the file is compressed by this hypothetical ultimate compression algorithm, the size of the compressed file would be the Kolmogorov complexity of the original file.
But then my concern just transforms into “what if there’s a powerful entity living in this universe (rather than outside of it) who will punish me if I do X, etc”.
Then we are no longer talking about gods in the modern sense, but about powerful aliens.
Congratulations—noticing that you are confused is an important step!
What are you doing while “awaiting additional evidence”? This is a topic that doesn’t have a neutral/agnostic position—biology forces you to eat, you have some influence (depending on your willpower model) over what and how much.
This belief wasn’t really affecting my eating habits, so I don’t think I’ll be changing much. My rules are basically:
No meat (I’m a vegetarian for moral reasons).
If I feel hungry but I can see/feel my stomach being full by looking at / touching my belly, I’m probably just bored or thirsty and I should consider not eating anything.
Try to eat at least a meal’s worth of “light” food (like toast or cereal as opposed to pizza or nachos) per day. This last rule is just to keep me from getting stomach aches, which happens if I eat too much “heavy” food in too short a time span.
I think I might contend that this kind of reflects an agnostic position. But I’m glad you asked, because I hadn’t noticed before that rule 2 actually does implicitly assume some relationship between “amount of food” and “weight change”, and is put in place so I don’t gain weight. So I guess I should really have said that what I tossed out the window was the extra detail that calories alone determine the effect food will have on one’s weight. I still believe, for normal cases, that taking the same eating pattern but scaling it up (eating more of everything but keeping the ratios the same) will result in weight gain.
Are rationalist ideas always going to be offensive to just about everybody who doesn’t self-select in?
One loved one was quite receptive to Chesterton’s Fence the other day. Like, it stopped their rant in the middle of its tracks and got them on board with a different way of looking at things immediately.
On the other hand, I routinely feel this weird tension. Like to explain why I think as I do, I‘d need to go through some basic rational concepts. But I expect most people I know would hate it.
I wish we could figure out ways of getting this stuff across that was fun, made it seem agreeable and sensible and non-threatening.
Less negativity—we do sooo much critique. I was originally attracted to LW partly as a place where I didn’t feel obligated to participate in the culture war. Now, I do, just on a set of topics that I didn’t associate with the CW before LessWrong.
My guess? This is totally possible. But it needs a champion. Somebody willing to dedicate themselves to it. Somebody friendly, funny, empathic, a good performer, neat and practiced. And it needs a space for the educative process—a YouTube channel, a book, etc. And it needs the courage of its convictions. The sign of that? Not taking itself too seriously, being known by the fruits of its labors.
Traditionally, things like this are socially achieved by using some form of “good cop, bad cop” strategy. You have someone who explains the concepts clearly and bluntly, regardless of whom it may offend (e.g. Eliezer Yudkowsky), and you have someone who presents the concepts nicely and inoffensively, reaching a wider audience (e.g. Scott Alexander), but ultimately they both use the same framework.
The inoffensiveness of Scott is of course relative, but I would say that people who get offended by him are really not the target audience for rationalist thought. Because, ultimately, saying “2+2=4” means offending people who believe that 2+2=5 and are really sensitive about it; so the only way to be non-offensive is to never say anything specific.
If a movement only has the “bad cops” and no “good cops”, it will be perceived as a group of assholes. Which is not necessarily bad if the members are powerful; people want to join the winning side. But without actual power, it will not gain wide acceptance. Most people don’t want to go into unnecessary conflicts.
On the other hand, a movement with “good cops” without “bad cops” will get its message diluted. First, the diplomatic believers will dilute their message in order not to offend anyone. Their fans will further dilute the message, because even the once-diluted version is too strong for normies’ taste. At the end, the message may gain popular support… kind of… because the version that gains the popular support will actually contain maybe 1% of the original message, but mostly 99% of what the normies already believed, peppered by the new keywords.
The more people will present rationality using different methods, the better. Because each of them will reach a different audience. So I completely approve the approach you suggest… in addition to the existing ones.
I need to try a lot harder to remember that this is just a community full of individuals airing their strongly held personal opinions on a variety of topics.
Those opinions often have something in common—respect for the scientific method, effort to improve one’s rationality, concern about artificial intelligence—and I like to believe it is not just a random idiosyncratic mix (a bunch of random things Eliezer likes), but different manifestations of the same underlying principle (use your intelligence to win, not to defeat yourself). However, not everyone is interested in all of this.
And I would definitely like to see “somebody friendly, funny, empathic, a good performer, neat and practiced” promoting these values in a YouTube channel or in books. But that requires a talent I don’t have, so I can only wait until someone else with the necessary skills does it.
This reminded me of the YouTube channel of Julia Galef, but the latest videos there are 3 years old.
Like to explain why I think as I do, I‘d need to go through some basic rational concepts.
I believe that if the rational concepts are pulling their weight, it should be possible to explain the way the concept is showing up concretely in your thinking, rather than justifying it in the general case first.
As an example, perhaps your friend is protesting your use of anecdotes as data, but you wish to defend it as Bayesian, if not scientific, evidence. Rather than explaining the difference in general, I think you can say “I think that it’s more likely that we hear this many people complaining about an axe murderer downtown if that’s in fact what’s going on, and that it’s appropriate for us to avoid that area today. I agree it’s not the only explanation and you should be able to get a more reliable sort of data for building a scientific theory, but I do think the existence of an axe murderer is a likely enough explanation for these stories that we should act on it”
If I’m right that this is generally possible, then I think this is a route around the feeling of being trapped on the other side of an inferential gap (which is how I interpreted the ‘weird tension’)
I think you’re right, when the issue at hand is agreed on by both parties to be purely a “matter of fact.”
As soon as social or political implications crop in, that’s no longer a guarantee.
But we often pretend like our social/political values are matters of fact. The offense arises when we use rational concepts in a way that gives the lie to that pretense. Finding an indirect and inoffensive way to present the materials and let them deconstruct their pretenses is what I’m wishing for here. LW has a strong culture surrounding how these general-purpose tools get applied, so I’d like to see a presentation of the “pure theory” that’s done in an engaging way not obviously entangled with this blog.
The alternative is to use rationality to try and become savvier social operators. This can be “instrumental rationality” or it can be “dark arts,” depending on how we carry it out. I’m all for instrumental rationality, but I suspect that spreading rational thought further will require that other cultural groups appropriate the tools to refine their own viewpoints rather than us going out and doing the convincing ourselves.
The term for the “fear of truth” is alethophobia. I’m not familiar of many other great terms in this area (curious to hear suggestions).
Apparently “Epistemophobia” is a thing, but that seems quite different; Epistemophobia is more the fear of learning, rather than the fear of facing the truth.
One given definition of alethophobia is, “The inability to accept unflattering facts about your nation, religion, culture, ethnic group, or yourself”
This seems like a incredibly common issue, one that is especially talked about as of recent, but without much specific terminology.
Can you give examples of things you think would fit under this? It seems that there are lots of instances of being resistant to the truth, but I can think of very few that I would categorize as fear of truth. It’s often fear of something else (e.g. fear of changing your identity) or biases (e.g. the halo effect or consistency bias) that cause people to resist. I can think of very few cases where people have a general fear of truth.
Without looking, I’ll predict it’s a neologism—invented in the last ~25 years to apply to someone the inventor disagrees with. Yup, google n-grams shows 0 uses in any indexed book. A little more searching shows almost no actual uses anywhere—mostly automated dictionary sites that steal from each other, presumably with some that accept user submissions.
Yea, I also found the claim, as well as a few results from old books before the claim. The name come straight from the Latin though, so isn’t that original or surprising.
Just because it hasn’t been used much before doesn’t mean we can’t start to use it and adjust the definition as we see fit. I want to see more and better terminology in this area.
From the Greek as it happens. Also, alethephobia would be a double negative, with a-letheia meaning a state of not being hidden; a more natural neologism would avoid that double negative. Also, the greek concept of truth has some differences to our own conceptualization. Bad neologism.
The trend of calling things that aren’t fears “-phobia” seems to me a trend that’s harmful for clear communication. Adjusting the definition only leads to more confusion.
I want to see more and better terminology in this area.
I think I want less terminology in this area, and generally more words and longer descriptions for things that need nuance and understanding. Dressing up insults as diagnoses doesn’t help any goals I understand, and jargon should only be introduced as part of much longer analyses where it illuminates rather than obscures.
An interesting thing about Supernatural Fitness (a VR app kinda like Beat Saber) is that they are leaning hard into being a fitness app rather than a game. You don’t currently get to pick songs, you pick workouts, which come with pep talks and stretching and warmups.
This might make you go “ugh, I just wanna play a song” and go play Beat Saber instead. But, Supernatural Fitness is _way_ prettier and has some conceptual advances over Beat Saber.
And… I mostly endorse this and think it was the right call. I am sympathetic to “if you give people the ability to choose whatever, they mostly choose to be couch potatoes, or goodhart on simple metrics”, and if you want people to do anything complicated and interesting you need to design your app with weird constraints in mind.
(Example: LessWrong deliberately doesn’t show users the view-count of their posts. We already have the entire internet as the control group for what happens if you give people view-counts – they optimize for views, and you get clickbait. Is this patronizing? Yeah. Am I 100% confident it’s the right call? No. But, I do think if you want to build a strong intellectual culture, it matters what kinds of Internet Points you give [or don’t give] people, and this is at least a judgment call you need to be capable of making)
But… I still do think it’s worth looking at third options. Sometimes, I might really want to just jam to some tunes, and I want to pick the specific
In the case of Supernatural Fitness, I think it is quite important that your opening experience puts you in the mindset of coaches and workouts, and that songs are clustered together in groups of 15+ minutes (so you actually get a cardio workout), and that they spend upfront effort teaching you the proper form for squats and encouraging you to maintain that form rather than “minimizing effort” (which I think Beat Saber ends up teaching you, and if you’re coming from Beat Saber, you might have already internalized habits around)
At first I thought “maybe they should make you learn proper form first, but eventually give you the ability to choose individual songs.” Then I simulated myself doing that, and thought “well, I would probably end up just doing less workouts.”
My current guess for my coherent-extrapolated-preference for the game is for something like “Individual song plays are more like rewards for actually completing a workout” (i.e. if you make it all the way through you get credits that can be spent on playing individual songs). Or, alternately, maybe players can assemble workouts out of at least 3 songs.
It’s probably complicated somewhat by the app licensing popular music, and the licenses might be temporary.
Epistemic status: really shaky, but I think there’s something here.
I naturally feel a lot of resistance to the way culture/norm differences are characterized in posts like Ask and Guess and Wait vs Interrupt Culture. I naturally want to give them little pet names, like:
Guess culture = “read my fucking mind, you badwrong idiot” culture.
Ask culture = nothing, because this is just how normal, non-insane people act.
I think this feeling is generated by various negative experiences I’ve had with people around me, who, no matter where I am, always seem to share between them one culture or another that I don’t really understand the rules of. This leads to a lot of interactions where I’m being told by everyone around me that I’m being a jerk, even when I can “clearly see” that their is nothing I could have done that would have been correct in their eyes, or that what they wanted me to do was impossible or unreasonable.
But I’m starting to wonder if I need to let go of this. When I feel someone is treating me unfairly, it could just be because (1) they are speaking in Culture 1, then (2) I am listening in Culture 2 and hearing something they don’t mean to transmit. If I was more tuned in to what people meant to say, my perception of people who use other norms might change.
I feel there’s at least one more important pair of cultures, and although I haven’t mentioned it yet, it’s the one I had in mind most while writing this post. Something like:
Culture 1: Everyone speaks for themselves only, unless explicitly stated otherwise. Putting words in someone’s mouth or saying that they are “implying” something they didn’t literally say is completely unacceptable. False accusations are taken seriously and reflect poorly on the accuser.
Culture 2: The things you say reflect not only on you but also on people “associated” with you. If X is what you believe, you might have to say Y instead if saying X could be taken the wrong way. If someone is being a jerk, you don’t have to extend the courtesy of articulating their mistake to them correctly; you can just shun them off in whatever way is easiest.
I don’t really know how real this dichotomy is, and if it is real, I don’t know for sure how I feel about one being “right” and the other being “wrong”. I tried semi-hard to give a neutral take on the distinction, but I don’t think I succeeded. Can people reading this tell which culture I naturally feel opposed to? Do you think I’ve correctly put my finger on another real dichotomy? Which set of norms, if either, do you feel more in tune with?
I think this feeling is generated by various negative experiences I’ve had with people around me, who, no matter where I am, always seem to share between them one culture or another that I don’t really understand the rules of. This leads to a lot of interactions where I’m being told by everyone around me that I’m being a jerk, even when I can “clearly see” that their is nothing I could have done that would have been correct in their eyes, or that what they wanted me to do was impossible or unreasonable.
Is it because they’re expecting you to read their mind, and go along with their “culture”, instead of asking you?
it (the negative experiences) - Are *they (the negative experiences) the result of (people with a “culture” who’s rules rules you don’t understand) expecting you to read *their mind, and go along with their “culture”, instead of asking you to go along with their culture?
Aha, no, the mind reading part is just one of several cultures I’m mentioning. (Guess Culture, to be exact.) If I default to being an Asker but somebody else is a Guesser, I might have the following interaction with them:
Me: [looking at some cookies they just made] These look delicious! Would it be all right if I ate one?
Them: [obviously uncomfortable] Uhm… uh… I mean, I guess so...
Here, it’s retroactively clear that, in their eyes, I’ve overstepped a boundary just by asking. But I usually can’t tell in advance what things I’m allowed to ask and what things I’m not allowed to ask. There could be some rule that I just haven’t discovered yet, but because I haven’t discovered it yet, it feels to me like each case is arbitrary, and thus it feels like I’m being required to read people’s minds each time. Hence why I’m tempted to call Guess Culture as “Read-my-mind Culture”.
(Contrast this to Ask Culture, where the rule is, to me, very simple and easy to discover: every request is acceptable to make, and if the other person doesn’t want you to do what you’re asking to do, they just say “no”.)
It might be hard to take a normative stance, but if culture 1 makes you feel better AND leads to better results AND helps people individuate and makes adults out of them, then maybe it’s just, y’know, better. Not “better” in the naive mistake-theorist assumption that there is such a thing as a moral truth, but “better” in the correct conflict-theorist assumption that it just suits you and me and we will exert our power to make it more widely adopted, for the sake of us and our enlightened ideals.
I’m experimenting with a format for applying LW tools to personal social-life problems. The goal is to boil down situations so that similar ones will be easy to diagnose and deal with in the future.
To do that, I want to arrive at an acronym that’s memorable, defines an action plan and implies when you’d want to use it. Examples:
OSSEE Activity—“One Short Simple Easy-to-Exit Activity.” A way to plan dates and hangouts that aren’t exhausting or recipes for confusion.
DAHLIA—“Discuss, Assess, Help/Ask, Leave, Intervene, Accept.” An action plan for how to deal with annoying behavior by other people. Discuss with the people you’re with, assess the situation, offer to help or ask the annoying person to stop, leave if possible, intervene if not, and accept the situation if the intervention doesn’t work out.
I came up with these by doing a brief post-mortem analysis on social problems in my life. I did it like this:
Describe the situation as fairly as possible, both what happened and how it felt to me and others.
Use LW concepts to generalize the situation and form an action plan. For example, OSSEE Activity arose from applying the concept of “diminishing marginal returns” to my outings.
Format the action plan into a mnemonic, such as an acronym.
Experiment with applying the action plan mnemonic in life and see if it leads you to behave differently and proves useful.
I’m annoyed that I think so hard about small daily decisions.
Is there a simple and ideally general pattern to not spend 10 minutes doing arithmetic on the cost of making burritos at home vs. buying the equivalent at a restaurant? Or am I actually being smart somehow by spending the time to cost out that sort of thing?
Perhaps:
“Spend no more than 1 minute per $25 spent and 2% of the price to find a better product.”
This heuristic cashes out to:
Over a year of weekly $35 restaurant meals, spend about $35 and an hour and a half finding better restaurants or meals.
For $250 of monthly consumer spending, spend a total of $5 and 10 minutes per month finding a better product.
For bigger buys of around $500 (about 2x/year), spend $10 and 20 minutes on each purchase.
Buying a used car ($15,000) I’d spend $300 and 10 hours. I could use the $300 to hire somebody at $25/hour to test-drive an additional 5-10 cars, a mechanic to inspect it on the lot, a good negotiator to help me secure a lower price.
For work over the next year ($30,000), spend $600 and 20 hours.
Getting a Master’s degree ($100,000 including opportunity costs), spend 66 hours and $2,000 finding the best school.
Choosing from among STEM career options ($100,000 per year), spend about 66 hours and $600 per year exploring career decisions.
Comparing that with my own patterns, that simplifies to:
Spend much less time thinking about daily spending. You’re correctly calibrated for ~$500 buys. Spend much more time considering your biggest buys and decisions.
For some (including younger-me), the opposite advice was helpful—I’d agonize over “big” decisions, without realizing that the oft-repeated small decisions actually had a much larger impact on my life.
To account for that, I might recommend you notice cache-ability and repetition, and budget on longer timeframes. For monthly spending, there’s some portion that’s really $120X decade spending (you can optimize once, then continue to buy monthly for the next 10 years), a bunch that’s probably $12Y of annual spending, and some that’s really $Z that you have to re-consider every month.
Also, avoid the mistake of inflexible permissions. Notice when you’re spending much more (or less!) time optimizing a decision than your average, but there are lots of them that actually benefit from the extra time. And lots that additional time/money doesn’t change the marginal outcome by much, so you should spend less time on.
I wonder if your problem as a youth was in agonizing over big decisions, rather than learning a productive way to methodically think them through. I have lots of evidence that I underthink big decisions and overthink small ones. I also tend to be slow yet ultimately impulsive in making big changes, and fast yet hyper-analytical in making small changes.
Daily choices have low switching and sunk costs. Everybody’s always comparing, so one brand at a given price point tends to be about as good as another.
But big decisions aren’t just big spends. They’re typically choices that you’re likely stuck with for a long time to come. They serve as “anchors” to your life. There are often major switching and sunk costs involved. So it’s really worthwhile anchoring in the right place. Everything else will be influenced or determined by where you’re anchored.
The 1 minute/$25 + 2% of purchase price rule takes only a moment’s thought. It’s a simple but useful rule, and that’s why I like it.
There are a few items or services that are relatively inexpensive, but have high switching costs and are used enough or consequential enough to need extra thought. Examples include pets, tutors, toys for children, wedding rings, mattresses, acoustic pianos, couches, safety gear, and textbooks. A heuristic and acronym for these exceptions might be CHEAPS: “Is it a Curriculum? Is it Heavy? Is it Ergonomic? Is it Alive? Is it Precious? Is it Safety-related?”
Why do men mansplain and why do people (particularly women) hate it? People sometimes struggle to articulate what mansplaining is and why they dislike it, but I’m surely not the discoverer of this argument.
Recently I was talking to a colleage during a strategy game session. He said “You are bad because you made these mistakes” and I said “yes I am bad at these aspects of the game. Alsol, you should have invested more into anti-aircraft guns”. He immediately began repeating a list of mistakes I had made, as evidence that his investment in AA was optimal. But I have seen the AA formula so I made a technical argument for strong AA in the late game. He stepped back and said “Well you were trashtalking me so I had to...”.
Then I realized that we were not explaining the game to one another or discussing the best build. We were really having a dominance fight, through vaguely technical arguments. Once you realize that men do this, you see it fairly often. Recently I said that transaction costs were built into the bitcoin code from the beginning, and a friend argued back that only third party exchanges charge transaction fees. It took me a while to prove him wrong because the content was about dominance, not about bitcoin[1].
Meanwhile I’ve been learning to flirt with women. Originally I though that you cannot disagree with a women while flirting, since it is a dominance play ala man world. But actually you can disagree in a non-conflictive way. For example, on Tinder I asked a women what she reads, and she said “I don’t think reading indicates intelligence or curiosity”. I responded by saying “I accept the argument on intelligence, but I expect curiosity and reading are correlated. Otherwise would be too surprising”. I have found this type of disagreement actually gets a longer response (signal of interest). My theory is that the qualified disagreement shows intelligence, status and social intelligence.
1 - To his credit, he eventually accepted the technical argument
We were really having a dominance fight, through vaguely technical arguments. Once you realize that men do this, you see it fairly often.
This is a great observation! But, as often happens in political debates, “mansplaining” is a motte-and-bailey term, which could mean the thing you just described, or it could mean “a man tried to say something”, or anything in between, depending on who used it and in which context.
Also, it is not an exclusively male behavior, despite the name. I have no strong opinion on whether it is a mostly male behavior, because I assume that most female dominance fights happen out of my sight, for various reasons. But I am pretty sure I have seen women fighting for dominance using supposedly factual arguments a few times.
Originally I though that you cannot disagree with a women while flirting, since it is a dominance play ala man world. But actually you can disagree in a non-conflictive way.
Depends on whether the specific woman finds dominance attractive. And that probably also depends on the type/degree of dominance, her mood, and how well you know each other. Yes, this “partially agree, partially disagree” strategy seems like the golden middle way between being disagreeable and boring.
Depends on whether the specific woman finds dominance attractive. And that probably also depends on the type/degree of dominance, her mood, and how well you know each other. Yes, this “partially agree, partially disagree” strategy seems like the golden middle way between being disagreeable and boring.
I think many women, perhaps a majority, find a more dominant man attractive. Basically ensure any fact-based dominance display doesn’t make the other person feel stupid. Good rule for lots of interactions.
With a term like mansplaining that’s used in quite different scopes by different people it would be helpful to start with a more clear definition before going into why people do it.
True. I should rephrase my thesis “What people often mean when they say “mansplaining” is explanations which are intended to express dominance rather than to mutually arrive at better understanding”.
“Medium Engagement Activities” are the death of culture creation.
Expecting someone to show up for a ~1-hour or more event every week that helps shape your culture is great for culture creation, or requiring them to wear a dress code—large commitments are good in the early stages.
Removing trivial inconveniences to following your values and rules is great for building culture, doing things that require no or low engagement but help shape group cohesion. Design does a lot here—no commitment tools to shape culture are great during early stages.
But medium commitment tools are awful, a series of little things that take 5-50 minutes a week to work on—these things are death to early stage cultures. It’s death by a 1000 cuts of things they can’t see clear immediate benefit for, and which they can see clear immediate cost for.
I don’t know why exactly this is, and haven’t really mapped out what’s behind this intuition, it’s something about the benefits of building identity vs. the time required, it’s ushaped, the tails are a much more effective tradeoff than the middle.
With some frequency, LW gets a new user writing a post that’s sort of… in the middle of having their mind blown by the prospect of quantum immortality and MWI. I’d like to have a single post to link them to that makes a fairly succinct case for “it adds up to normality”, and I don’t have a clear sense of what to do other that link to the entire Quantum Physics sequence.
Any suggestions? Or, anyone feel like writing said post if it doesn’t exist yet?
The wavefunctioncollapse algorithm measures whichever tile currently has the lowest entropy. GPT-3 always just measures the next token. Of course in prose those are usually the same, but I expect some qualitative improvements once we get structured data with holes such that any might have low entropy, a transformer trained to fill holes, and the resulting ability to pick which hole to fill next.
Until then, I expect those prompts/GPT protocols to perform well which happen to present the holes in your data in the order that wfc would have picked, ie ask it to show its work, don’t ask it to write the bottom line of its reasoning process first.
Long shortform short: Include the sequences in your prompt as instructions :)
CFAR’s “Adjust Your Seat” principle and associated story is probably one of my most frequently referenced concepts when teaching rationality techniques.
I wish there was a LW post about it.
I used to think that slower takeoff implied shorter timelines, because slow takeoff means that pre-AGI AI is more economically valuable, which means that economy advances faster, which means that we get AGI sooner. But there’s a countervailing consideration, which is that in slow takeoff worlds, you can make arguments like ‘it’s unlikely that we’re close to AGI, because AI can’t do X yet’, where X might be ‘make a trillion dollars a year’ or ‘be as competent as a bee’. I now overall think that arguments for fast takeoff should update you towards shorter timelines.
So slow takeoffs cause shorter timelines, but are evidence for longer timelines.
This graph is a version of this argument: if we notice that current capabilities are at the level of the green line, then if we think we’re on the fast takeoff curve we’ll deduce we’re much further ahead than we’d think on the slow takeoff curve.
For the “slow takeoffs mean shorter timelines” argument, see here: https://sideways-view.com/2018/02/24/takeoff-speeds/This point feels really obvious now that I’ve written it down, and I suspect it’s obvious to many AI safety people, including the people whose writings I’m referencing here. Thanks to Caroline Ellison for pointing this out to me, and various other people for helpful comments.
I think that this is why belief in slow takeoffs is correlated with belief in long timelines among the people I know who think a lot about AI safety.
Because I’m dumb, I would have found it easier to interpret the graph if the takeoff curves crossed at “current capabilities” (and thus get to high levels of capabilities at different times)
Normative Realism
Normative Realism by Degrees
Normative Anti-realism is self-defeating
Normativity and recursive justification
Prescriptive Anti-realism
‘Realism about rationality’ is Normative Realism
‘Realism about rationality’ discussed in the context of AI safety and some of its driving assumptions may already have a name in existing philosophy literature. I think that what it’s really referring to is ‘normative realism’ overall—the notion that there are any facts about what we have most reason to believe or do. Moral facts, if they exist, are a subset of normative facts. Epistemic facts, facts about what we have most reason to believe (e.g. if there is a correct decision theory that we should use, that would be an epistemic fact), are a different subset of normative facts.
These considerations (from the original article) seem to clearly indicate ‘realism about epistemic facts’ in the metaethical sense:
These seem to imply normative (if not exactly moral) realism in general:
If this ‘realism about rationality’ really is rather like “realism about epistemic reasons/‘epistemic facts’”, then you have the ‘normative web argument’ to contend with, that the above two may be connected:
If ‘realism about rationality’ is really just normative realism in general, or realism about epistemic facts, then there is already an extensive literature on whether it is correct or not. I’m going to discuss some of it below.
Normative realism implies identification with system 2
There’s a further implication that normative realism has—it makes things like this (from the original article, again) seem incoherent:
I find this very interesting because locating personal identity in system 1 feels conceptually impossible or deeply confusing. No matter how much rationalization goes on, it never seems intuitive to identify myself with system 1. How can you identify with the part of yourself that isn’t doing the explicit thinking, including the decision about which part of yourself to identify with? It reminds me of Nagel’s The Last Word.
Normative Realism by degrees
Further to the whole question of Normative / moral realism, there is this post on Moral Anti-Realism. While I don’t really agree with it, I do recommend reading it—one thing that it convinced me of is that there is a close connection between your particular normative ethical theory and moral realism. If you claim to be a moral realist but don’t make ethical claims beyond ‘self-evident’ ones like pain is bad, given the background implausibility of making such a claim about mind-independent facts, you don’t have enough ‘material to work with’ for your theory to plausibly refer to anything . The Moral Anti-Realism post presents this dilemma for the moral realist:
To a large degree, I agree with this claim—I think that many moral realists do as well. Convergence type arguments often appear in more recent metaethics (Hare and Parfit are in those previous lists) - so this may already have been recognised. The post discusses such a response to antirealism at the end:
But there’s one point I want to make which is in disagreement with that post. I agree that how much you can concretely say about your supposed mind-independent domain of facts affects how plausible its existence should seem, and even how coherent the concept is, but I think that this can come by degrees. This should not be surprising—we’ve known since Quine and Kripke that you can have evidential considerations for/against and degrees of uncertainty about a priori questions. The correct method in such a situation is Bayesian—tally the plausibility points for and against admitting the new thing into your ontology. This can work even if we don’t have an entirely coherent understanding of normative facts, as long as it is coherent enough.
Suppose you’re an Ancient Egyptian who knows a few practical methods for trigonometry and surveying, doesn’t know anything about formal systems or proofs, and someone asks you if there are ‘mathematical facts’. You would say something like “I’m not totally sure what this ‘maths’ thing consists of, but it seems at least plausible that there are some underlying reasons why we keep hitting on the same answers”. You’d be less confident than a modern mathematician, but you could still give a justification for the claim that there are right and wrong answers to mathematical claims. I think that the general thrust of convergence arguments puts us in a similar position with respect to ethical facts.
So, we don’t see the whole mountain, but nor are we seeing simply a few free-floating rocks that might be a mirage. Instead, what we see is maybe part of one slope and a peak.
Let’s be concrete, now—the 5 second, high level description of both Hare’s and Parfit’s convergence arguments goes like this:
and
In other words, the principles that (whatever our particular wants) would produce the best outcome in terms of satisfying our goals, could be willed to be a universal law by all of us and would not be rejected as the basis for a contract, are all the same principles. That is at least suspicious levels of agreement between ethical theories. This is something substantive that can be said—out of every major attempt to get at a universal ethics that has in fact been attempted in history: what produces the best outcome, what can you will to be a universal law, what would we all agree on, seem to produce really similar answers.
The particular convergence arguments given by Parfit and Hare are a lot more complex, I can’t speak to their overall validity. If we thought they were valid then we’d be seeing the entire mountain precisely. Since they just seem quite persuasive, we’re seeing the vague outline of something through the fog, but that’s not the same as just spotting a few free-floating rocks.
Now, run through these same convergence arguments but for decision theory and utility theory, and you have a far stronger conclusion. there might be a bit of haze at the top of that mountain, but we can clearly see which way the slope is headed.
This is why I think that ethical realism should be seen as plausible and realism about some normative facts, like epistemic facts, should be seen as more plausible still. There is some regularity here in need of explanation, and it seems somewhat more natural on the realist framework.
I agree that this ‘theory’ is woefully incomplete, and has very little to say about what the moral facts actually consist of beyond ‘the thing that makes there be a convergence’, but that’s often the case when we’re dealing with difficult conceptual terrain.
From Ben’s post:
From the perspective of x, x is not self-defeating
From the antirealism post, referring to the normative web argument:
Sooner or later every theory ends up question-begging.
From the perspective of Theism, God is an excellent explanation for the universe’s existence since he is a person with the freedom to choose to create a contingent entity at any time, while existing necessarily himself. From the perspective of almost anyone likely to read this post, that is obvious nonsense since ‘persons’ and ‘free will’ are not primitive pieces of our ontology, and a ‘necessarily existent person’ makes as much sense as ‘necessarily existent cabbage’- so you can’t call it a compelling argument for the atheist to become a theist.
By the same logic, it is true that saying ‘anti-realism is unjustified on the realist sense of justification’ is question-begging by the realist. The anti-realist has nothing much to say to it except ‘so what’. But you can convert that into a Quinean, non-question begging plausibility argument by saying something like:
This won’t compel the anti-realist, but I think it would compel someone weighing up the two alternative theories of how justification works. If you are uncertain about whether there are mind-independent facts about our beliefs being justified, the argument that anti-realism is self-defeating pulls you in the direction of realism.
I got into a discussion with Lucas Gloor on the EA forum about these issues. I’m copying some of what I wrote here as it’s a continuation of that.
I think that it is a more damaging mistake to think moral antirealism is true when realism is true than vice versa, but I agree with you that the difference is nowhere near infinite, and doesn’t give you a strong wager.However, I do think that normative anti-realism is self-defeating, assuming you start out with normative concepts (though not an assumption that those concepts apply to anything). I consider this argument to be step 1 in establishing moral realism, nowhere near the whole argument.
Epistemic anti-realism
I agree that there is no infinitely strong wager for moral realism. As soon as moral realists start making empirical claims about the consequences of realism (that convergence is likely), you can’t say that moral realism is true necessarily or that there is an infinitely strong prior in favour of it. An AI that knows that your idealised preferences don’t cohere could always show up and prove you wrong, just as you say. If I were Bob in this dialogue, I’d happily concede that moral anti-realism is true.If (supposing it were the case) there were not much consensus on anything to do with morality (“The rocks don’t connect...”), someone who pointed that out and said ‘from that I infer that moral realism is unlikely’ wouldn’t be saying anything self-defeating. Moral anti-realism is not self-defeating, either on its own terms or on the terms of a ‘mixed view’ like I describe here:
However, I do think that there is an infinitely strong wager in favour of normative realism and that normative anti-realism is self-defeating on the terms of a ‘mixed view’ that starts out considering the two alternatives like that given above. This wager is because of the subset of normative facts that are epistemic facts.The example that I used was about ‘how beliefs are justified’. Maybe I wasn’t clear, but I was referring to beliefs in general, not to beliefs about morality. Epistemic facts, e.g. that you should believe something if there is sufficient amount of evidence, are a kind of normative fact. You noted them on your list here.So, the infinite wager argument goes like this -
Except that doesn’t work! Because on normative anti-realism, the whole idea of external facts about which beliefs are justified is mistaken, and instead we all just have fundamental principles (whether moral or epistemic) that we use but don’t question, which means that holding a belief without (the realist’s notion of) justification is consistent with anti-realism.So the wager argument for normative realism actually goes like this -
Evidence for epistemic facts?
I find it interesting the imagined scenario you give in #5 essentially skips over argument 2) as something that is impossible to judge:
The AI doesn’t give evidence against there being irreducible normative facts about how to reason, it just states it finds the concept incoherent, unlike the (hypothetical) evidence that the AI piles on against moral realism (for example, that people’s moral preferences don’t cohere).Either you think some basic epistemic facts have to exist for reasoning to get off the ground and therefore that epistemic anti-realism is self-defeating, or you are an epistemic anti-realist and don’t care about the realist’s sense of ‘self-defeating’. The AI is in the latter camp, but not because of evidence, the way that it’s a moral anti-realist (...However, you haven’t established that all normative statements work the same way—that was just an intuition...), but just because it’s constructed in such a way that it lacks the concept of an epistemic reason.So, if this AI is constructed such that irreducibly normative facts about how to reason aren’t comprehensible to it, it only has access to argument 1), which doesn’t work. It can’t imagine 2).However, I think that we humans are in a situation where 2) is open to consideration, where we have the concept of a reason for believing something, but aren’t sure if it applies—and if we are in that situation, I think we are dragged towards thinking that it must apply, because otherwise our beliefs wouldn’t be justified.However, this doesn’t establish moral realism—as you said earlier, moral anti-realism is not self-defeating.
Combining convergence arguments and the infinite wager
If you want to argue for moral realism, then you need evidence for moral realism, which comes in the form of convergence arguments. But the above argument is still relevant, because the convergence and ‘infinite wager’ arguments support each other. The reason 2) would be bolstered by the success of convergence arguments (in epistemology, or ethics, or any other normative domain) is that convergence arguments increase our confidence that normativity is a coherent concept—which is what 2) needs to work. It certainly seems coherent to me, but this cannot be taken as self-evident since various people have claimed that they or others don’t have the concept.I also think that 2) is some evidence in favour of moral realism, because it undermines some of the strongest antirealist arguments.
I don’t think this is right. What I’m giving here is such a ‘partners-in-crime’ argument with a structure, with epistemic facts at the base. Realism about normativity certainly should lower the burden of proof on moral realism to prove total convergence now, because we already have reason to believe normative facts exist. For most anti-realists, the very strongest argument is the ‘queerness argument’ that normative facts are incoherent or too strange to be allowed into our ontology. The ‘partners-in-crime’/‘infinite wager’ undermines this strong argument against moral realism. So some sort of very strong hint of a convergence structure might be good enough—depending on the details.
So, with all that out of the way, when we start discussing the convergence arguments, the burden of proof on them is not colossal. If we already have reason to suspect that there are normative facts out there, perhaps some of them are moral facts. But if we found a random morass of different considerations under the name ‘morality’ then we’d be stuck concluding that there might be some normative facts, but maybe they are only epistemic facts, with nothing else in the domain of normativity.I don’t think this is the case, but I will have to wait until your posts on that topic—I look forward to them!All I’ll say is that I don’t consider strongly conflicting intuitions in e.g. population ethics to be persuasive reasons for thinking that convergence will not occur. As long as the direction of travel is consistent, and we can mention many positive examples of convergence, the preponderance of evidence is that there are elements of our morality that reach high-level agreement. (I say elements because realism is not all-or-nothing—there could be an objective ‘core’ to ethics, maybe axiology, and much ethics could be built on top of such a realist core—that even seems like the most natural reading of the evidence, if the evidence is that there is convergence only on a limited subset of questions.) If Kant could have been a utilitarian and never realised it, then those who are appalled by the repugnant conclusion could certainly converge to accept it after enough ideal reflection!
How to make anti-realism existentially satisfying
If your goal here is to convince those inclined towards moral realism to see anti-realism as existentially satisfying, I would recommend a different framing of it. I think that framing morality as a ‘personal life goal’ makes it seem as though it is much more a matter of choice or debate than it in fact is, and will probably ring alarm bells in the mind of a realist and make them think of moral relativism.Speaking as someone inclined towards moral realism, the most inspiring presentations I’ve ever seen of anti-realism are those given by Peter Singer in The Expanding Circle and Eliezer Yudkowsky in his metaethics sequence. Probably not by coincidence—both of these people are inclined to be realists. Eliezer said as much, and Singer later became a realist after reading Parfit. Eliezer Yudkowsky on ‘The Meaning of Right’:
And Singer in the Expanding Circle:
These are both anti-realist claims. They define ‘right’ descriptively and procedurally as arising from what we would want to do under some ideal circumstances, and rigidifies on the output of that idealization, not on what we want. To a realist, this is far more appealing than a mere “personal, morally-inspired life goal”, and has the character of ‘external moral constraint’, even if it’s not really ultimately external, but just the result of immovable or basic facts about how your mind will, in fact work, including facts about how your mind finds inconsistencies in its own beliefs. This is a feature, not a bug:
The exact thing that Williams calls ‘alienating’ is the thing that Singer, Yudkowsky, Parfit and many other realists and anti-realists consider to be the most valuable thing about morality! But you can keep this ‘alienation’ if you reframe morality as being the result of the basic, deterministic operations of your moral reasoning, the same way you’d reframe epistemic or practical reasoning on the anti-realist view. Then it seems more ‘external’ and less relativistic.One thing this framing makes clearer, which you don’t deny but don’t mention, is that anti-realism does not imply relativism.
What we can say, on anti-realism as characterised by Singer and Yudkowsky, is that they are making an error of morality. We are not obligated (how could we be?) towards relativism, permissiveness or accepting values incompatible with our own on anti-realism. Ultimately, you can just say that ‘I am right and you are wrong’.That’s one of the major upsides of anti-realism to the realist—you still get to make universal, prescriptive claims and follow them through, and follow them through because they are morally right, and if people disagree with you then they are morally wrong and you aren’t obligated to listen to their arguments if they arise from fundamentally incompatible values. Put that way, anti-realism is much more appealing to someone with realist inclinations.
I appear to be accidentally writing a sequence on moral realism, or at least explaining what moral realists like about moral realism—for those who are perplexed about why it would be worth wanting or how anyone could find it plausible.
Many philosophers outside this community have an instinct that normative anti-realism (about any irreducible facts about what you should do) is self-defeating, because it includes a denial that there are any final, buck-stopping answers to why we should believe something based on evidence, and therefore no truly, ultimately impartial way to even express the claim that you ought to believe something. I think that this is a good, but not perfect, argument. My experience has been that traditional analytic philosophers find this sort of reasoning appealing, in part because of the legacy of how Kant tried to deduce the logically necessary preconditions for having any kind of judgement or experience. I don’t find it particularly appealing, but I think that there’s a case for it here, if there ever was.
Irreducible Normativity and Recursive Justification
On normative antirealism, what ‘you shouldn’t believe that 2+2=5’ really means is just that someone else’s mind has different basic operations to yours. It is obvious that we can’t stop using normative concepts, and couldn’t use the concept ‘should’ to mean ‘in accordance with the basic operations of my mind’, but this isn’t an easy case of reduction like Water=H20. There is a deep sense in which normative terms really can’t mean what we think they mean if normative antirealism is true. This must be accounted for by either a deep and comprehensive question-dissolving, or by irreducible normative facts.
This ‘normative indispensability’ is not an argument, but it can be made into one:
If you’ve read the sequences, you are not going to like this argument, at all—it sounds like the ‘zombie’ argument, and it sounds like someone asking for an exception to reductionism—which is just what it is. This is the alternative:
If it’s truly ‘logic all the way down’ and there are no ‘> tokens’ over particular functional arrangements of matter, including the ones you used to form your beliefs, then you have to give up on knowing reality as it is. This isn’t the classic sense in which we all have an ‘imperfect model’ of reality as it is. If you give up on irreducible epistemic facts you give up knowing anything, probabilistically or otherwise, about reality-as-it-is, because there are no fundamentally, objectively, mind-independent ways you should or shouldn’t form beliefs about external reality. So you can’t say you’re better than the pebble with ‘2+2=5’ written on it, except descriptively, in that the causal process that produced the pebble contradict the one that produced 2+2=4 in your brain.
What’s the alternative? If we don’t deny this consequence of normative antirealism, we have two options. One is the route of dissolving the question, by analogy with how reductionism has worked in the past, the other is to say that there are irreducible normative facts. In order to dissolve the question correctly, it needs to be in a way that shows a denial of epistemic facts isn’t damaging, doesn’t lead to epistemological relativism or scepticism. We can’t simply declare that normative facts can’t possibly exist—otherwise you’re vulnerable to the argument 2). David Chalmers talks about question-dissolving for qualia:
David Chalmers of all people says that, even if he can’t conceive of how a deep reduction of Qualia might make their non-existence non-paradoxical, he might change his mind if he ever actually saw such a reduction! I say the same about epistemic and therefore normative facts. But crucially, no-one has solved this ‘meta problem’ for Qualia or for normative facts. There are partial hints of explanations for both, but there’s no full debunking argument that makes epistemic antirealism seem completely non-damaging and thus removes 2). I can’t imagine what such an account could look like, but the point of the ‘dissolving the question’ strategy is that it often isn’t imaginable in advance because your concepts are confused, so I’ll just leave that point. In the moral domain, the convergence arguments point against question-dissolving because they suggest the concept of normativity is solid and reliable. If those arguments fall, then question-dissolving looks more likely.
That’s one route. What of the other?
The alternative is to say that there are irreducible normative facts. This is counter-reductionist, counter-intuitive and strange. Two things that can make it less strange: these facts are not supposed to be intrinsically motivational (that violates the orthogonality thesis and is not permitted by the laws of physics) and they are not required to be facts about objects, like Platonic forms outside of time and space. They can be logical facts of the sort Eliezer talked about, but just a particular kind of logical fact, that has the property of being normative, the one you should follow. They don’t need to ‘exist’ as such. What epistemic facts would do is say certain reflective equilibria, certain arrangements of ‘reflecting on your own beliefs, using your current mind’ are the right ones, and others are the wrong ones. It doesn’t deny that this is the case:
Irreducible normativity just says that there is a meaningful, mind-independent difference between the virtuous and degenerate cases of recursive justification of your beliefs, rather than just ways of recursively justifying our beliefs that are… different.
If you buy that anti-realism is self-defeating, and think that we can know something about the normative domain via moral and non-moral convergence, then we have actual positive reasons to believe that normative facts are knowable (the convergence arguments help establish that moral facts aren’t and couldn’t be random things like stacking pebbles in prime-numbered heaps).
These two arguments are quite different—one is empirical (that our practical, epistemic and moral reasons tend towards agreement over time and after conceptual analysis and reflective justification) and the other is conceptual (that if you start out with normative concepts you are forced into using them).
Depending on which of the arguments you accept, there are four basic options. These are extremes of a spectrum, as while the Normativity argument is all-or-nothing, the Convergence argument can come by degrees for different types of normative claims (epistemic, practical and moral):
• Accept Convergence and Reject Normativity: prescriptivist anti-realism. There are (probably) no mind-independent moral facts, but the nature of rationality is such that our values usually cohere and are stable, so we can treat morality as a more-or-less inflexible logical ordering over outcomes.
• Accept Convergence and Accept Normativity: moral realism. There are moral facts and we can know them
• Reject Convergence and Reject Normativity: nihilist anti-realism. Morality is seen as a ‘personal life project’ about which we can’t expect much agreement or even within-person coherence
• Reject Convergence and Accept Normativity: sceptical moral realism. Normative facts exist, but moral facts may not exist, or may be forever unknowable.
Even if what exactly normative facts are is hard to conceive, perhaps we can still know some things about them. Eliezer ended his post arguing for universalized, prescriptive anti-realism with a quote from HPMOR. Here’s a different quote:
Prescriptive Anti-realism
An extremely unscientific and incomplete list of people who fall into the various categories I gave in the previous post:
1. Accept Convergence and Reject Normativity: Eliezer Yudkowsky, Sam Harris (Interpretation 1), Peter Singer in The Expanding Circle, RM Hare and similar philosophers, HJPEV
2. Accept Convergence and Accept Normativity: Derek Parfit, Sam Harris (Interpretation 2), Peter Singer today, the majority of moral philosophers, Dumbledore
3. Reject Convergence and Reject Normativity: Robin Hanson, Richard Ngo (?), Lucas Gloor (?) most Error Theorists, Quirrell
4. Reject Convergence and Accept Normativity: A few moral philosophers, maybe Ayn Rand and objectivists?
The difference in practical, normative terms between 2), 4) and 3) is clear enough − 2 is a moral realist in the classic sense, 4 is a sceptic about morality but agrees that irreducible normativity exists, and 3 is a classic ‘antirealist’ who sees morality as of a piece with our other wants. What is less clear is the difference between 1) and 3). In my caricature above, I said Quirrell and Harry Potter from HPMOR were non-prescriptive and prescriptive anti-realists, respectively, while Dumbledore is a realist. Here is a dialogue between them that illustrates the difference.
The relevant issue here is that Harry draws a distinction between moral and non-moral reasons even though he doesn’t believe in irreducible normativity. In particular, he’s committed to a normative ethical theory, preference utilitarianism, as a fundamental part of how he values things.
Here is another illustration of the difference. Lucas Gloor (3) explains the case for suffering-focussed ethics, based on the claim that our moral intuitions assign diminishing returns to happiness vs suffering.
And what are those difficulties mentioned? The most obvious is the absurd conclusion—that scaling up a population can turn it from axiologically good to bad:
Here, then, is the difference—If you believe, as a matter of fact, that our values cohere and place fundamental importance on coherence, whether because you think that is the way to get at the moral truth (2) or because you judge that human values do cohere to a large degree for whatever other reason and place fundamental value on coherence (1), you will not be satisfied with leaving your moral theory inconsistent. If, on the other hand, you see morality as continuous with your other life plans and goals (3), then there is no pressure to be consistent. So to (3), focussing on suffering-reduction and denying the absurd conclusion is fine, but this would not satisfy (1).
I think that, on closer inspection, (3) is unstable—unless you are Quirrell and explicitly deny any role for ethics in decision-making, we want to make some universal moral claims. The case for suffering-focussed ethics argues that the only coherent way to make sense of many of our moral intuitions is to conclude a fundamental asymmetry between suffering and happiness, but then explicitly throws up a stop sign when we take that argument slightly further—to the absurd conclusion, because ‘the need to fit all one’s moral intuitions into an overarching theory based solely on intuitively appealing axioms simply cannot be fulfilled’. Why begin the project in the first place, unless you place strong terminal value on coherence (1)/(2) - in which case you cannot arbitrarily halt it.
I think the mountain analogy really is the center of the rationality anti-realist argument.
It’s very intuitive to think of us perceiving facts about e.g. epistemology as if gazing upon a mountain. There is a clean separation between us, the gazer, and that external mountain, which we perceive in a way that we can politely pretend is more or less directly. We receive rapid, rich data about it, through a sensory channel whose principles of operation we well understand and trust, and that data tends to cohere well together with everything else, except when sometimes it doesn’t but let’s not worry about that. Etc.
The rationality anti-realist position is that perceiving facts about epistemology is very little like looking at a mountain. I’m reminded of a Dennett quote about the quality of personal experience:
So, the mountain disanalogy: sometimes there are things we have opinions about, and yet there is no clean separation between us and the thing. We don’t perceive it in a way that we can agree is trusted or privileged. We receive vague, sparse data about it, and the subject is plagued by disagreement, self-doubt, and claims that other people are doing it all wrong.
This isn’t to say that we should give up entirely, but it means that we might have to shift our expectations of what sort of explanation or justification we are “entitled” to. Everyone would absolutely love it if they could objectively dunk on all those other people who disagree with them, but it’s probably going to turn out that a thorough explanation will sound more like “here’s how things got the way they are” rather than “here’s why you’re right and everyone else is wrong.”
So this depends on two things—first, how likely (in advance of assessing the ‘evidence’) something like normative realism is, and then how good that evidence is (how coherent it is). If we have really good reasons in advance to think there’s ‘no separation between us and the thing’ then no matter how coherent the ‘thing’ is we have to conclude that while we might all be able to agree on what it is, it isn’t mind independent.
So, is it coherent, and is it mind-independent? How coherent it needs to be for us to be confident we can know it, depends on how confident we are that its mind-independent, and vice versa.
The argument for coherence comes in the form of convergence (not among people, to be clear, but among normative frameworks), but as you say that doesn’t establish its mind independent (it might give you some strong hint, though, if its really strongly consistent and coherent), and the argument that normativity is mind-independent comes from the normativity argument. These three posts deal with the difference between those two arguments and how strong they are, and how they interact:
Normative Anti-realism is self-defeating
Normativity and recursive justification
Prescriptive Anti-realism
Collection of discussions of epistemic modesty, “rationalist/EA exceptionalism”, and similar
These are currently in reverse-chronological order.
Some thoughts on deference and inside-view models—Buck Shlegeris
Epistemic learned helplessness—Scott Alexander, 2019
AI, global coordination, and epistemic humility—Jaan Tallinn, 2018
In defence of epistemic modesty—Greg Lewis, 2017
Inadequate Equilibria—Eliezer Yudkowsky, 2017
Common sense as a prior—Nick Beckstead, 2013
From memory, I think a decent amount of Rationality: A-Z by Eliezer Yudkowsky is relevant
Philosophical Majoritarianism—Hal Finney, 2007
Somewhat less relevant/substantial
This comment/question—Michael Aird (i.e., me), 2020
Naming Beliefs—Hal Finney, 2008
Likely relevant, but I’m not yet sure how relevant as I haven’t yet read it
Are Disagreements Honest? - Cowen & Hanson, 2004
Uncommon Priors Require Origin Disputes—Robin Hanson, 2006
Aumann’s agreement theorem—Wikipedia
I intend to add to this list over time. If you know of other relevant work, please mention it in a comment.
See also EA reading list: cluelessness and epistemic modesty.
It seems like there’s room for the theory of logical-inductor-like agents with limited computational resources, and I’m not sure if this has already been figured out. The entire trick seems to be that when you try to build a logical inductor agent, it’s got some estimation process for math problems like “what does my model predict will happen?” and it’s got some search process to find good actions, and you don’t want the search process to be more powerful than the estimator because then it will find edge cases. In fact, you want them to be linked somehow, so that the search process is never in the position of taking advantage of the estimator’s mistakes—if you, a human, are making some plan and notice a blind spot in your predictions, you don’t “take advantage” of yourself, you do further estimating as part of the search process.
The hard part is formalizing this handwavy argument, and figuring out what other strong conditions need to be met to get nice guarantees like bounded regret.
Half-baked idea for low-impact AI:
As an example, imagine a board that’s lodged directly by the wall (no other support structures). If you make it twice as wide, then it will be twice as stiff, but if you make it twice as thick, then it will be eight times as stiff. On the other hand, if you make it twice as long, it will be eight times more compliant.
In a similar way, different action parameters will have scaling exponents (or more generally, functions). So one way to decrease the risk of high-impact actions would be to make sure that the scaling exponent is bounded above by a certain amount.
Anyway, to even do this, you still need to make sure the agent’s model is honestly evaluating the scaling exponent. And you would still need to define this stuff a lot more rigorously. I think this idea is more useful in the case where you already have an AI with high-level corrigible intent and want to give it a general “common sense” about the kinds of experiments it might think to try.
So it’s probably not that useful, but I wanted to throw it out there.
Somewhat delighted to see that google scholar now includes direct links to PDFs when it can find them instead of making you figure out how to use a given journal website.
There’s a plug in that will look for PDFs for you that match the page you’re on or the text you have highlighted.
This has been true for years. At least six, I think? I think I started using Google scholar around when I started my PhD, and I do not recall a time when it did not link to pdfs.
The results of Bob Jacob’s LessWrong survey are quite interesting. It’s a pity the sample is so small.
The visualized results (link in his post) are univariate, but I would like to highlight some things:
49 out of 56 respondents identifying as “White”,
53 out of 59 respondents born male and 46 out of 58 identifying male cisgender
47 of 59 identifying as heterosexual (comparison: https://en.wikipedia.org/wiki/Demographics_of_sexual_orientation)
1 out of 55 working in a “blue collar” profession
Most people identify as “left of center” in some sense. At the same time, 30 out of 55 identify as “libertarian”, but there were multiple answers allowed.
31 of 59 respondents think they are at least “upper middle class”; 22 of 59 think the family they were raised in was “upper middle class”. (Background: In social science surveys, wealthy people usually underestimate their position, and poor people overestimate it but to a lesser extent.)
I would not have guessed the left-of-center identification, and I would have slightly underestimated the share of male (cisgender).
If you have 9 people who identify as left-wing and 1 person who identifies as right-wing, many people will hysterically denounce the entire group as “extreme right”, based on the fact that the 1 person wasn’t banned.
Furthermore, if you have people who identify as left-wing, but don’t fully buy the current Twitter left-wing orthodoxy, they too will be denounced by some as “extreme right”.
This skews the perception.
I don’t think that fits what I am talking about:
The survey was non-binary. Your first claim does not distinguish extremes and moderates.
The survey was anonymous. You cannot ban anonymous people.
I see no reason why people should have overstated their leftishness.
If your statement is meant to explain why my perception differs from the result, it does not fit. My perception based on posts and comments would have been relatively more rightwing, less liberal / social democratic / green etc.
I don’t see where leftwing lesswrongers are denounced as rightwing extremists. In particular, I don’t see where this explains people identifying as leftwing in the survey.
My model is that in USA most intelligent people are left-wing. Especially when you define “left-wing” to mean the 50% of the political spectrum, not just the extreme. And there seem to be many Americans on Less Wrong, just like on most English-speaking websites.
(Note that I am not discussing here why this is so. Maybe the left-wing is inherently correct. Or maybe the intelligent people are just more likely to attend universities where they get brainwashed by the establishment. I am not discussing the cause here, merely observing the outcome.)
So, I would expect Less Wrong to be mostly left-wing (in the 50% sense). My question is, why were you surprised by this outcome?
For example, “neoreaction” is the only flavor of politics that is mentioned in the Wikipedia article about LessWrong. It does not claim that it is the predominant political belief, and it even says that Yudkowsky disagrees with them. Nonetheless, it is the only political opinion mentioned in connection with Less Wrong. (This is about making associations rather than making arguments.) So a reader who does not know how to read between the lines properly, might leave with an impression that LW is mostly right-wing. (Which is exactly the intended outcome, in my opinion.) And Wikipedia is not the only place where this game of associations is played.
I agree. (I assume that by political spectrum you refer to something “objective”?)
Given the whole Bay-area thing, I would have expected a higher share. In the survey, 37 out of 60 say they are residing in the US.
Having been in this forum for a while, my impressions based on posts and comments led me to believe that less than 50% of people on lessrong would say of themselves that they are on values 1-5 of 1-10 scale from left-wing to right-wing. In fact, 41⁄56 did so.
The wikipedia article, as far as I can see, explains in that paragraph where the neoreactionary movement originated. I don’t agree on the “intended outcome”, or rather, I do not see why I should believe that.
It’s not true, though! The article claims: “The neoreactionary movement first grew on LessWrong, attracted by discussions on the site of eugenics and evolutionary psychology”.
I mean, okay, it’s true that we’ve had discussions on eugenics and evolutionary psychology, and it’s true that a few of the contrarian nerds who enthusiastically read Overcoming Bias back in the late ’aughts were also a few of the contrarian nerds who enthusiastically read Unqualified Reservations. But “first grew” (Wikipedia) and “originated” (your comment) really doesn’t seem like a fair summary of that kind of minor overlap in readership. No one was doing neoreactionary political theorizing on this website. Okay, I don’t have a exact formalization of what I mean by “no one” in the previous sentence because I haven’t personally read and remembered every post in our archives; maybe there are nonzero posts with nonnegative karma that could be construed to match this description. Still, in essence, you can only make the claim “true” by gerrymandering the construal of those words.
And yet the characterization will remain in Wikipedia’s view of us—glancing at the talk page, I don’t expect to win an edit war with David Gerard.
Interesting. I had maybe read the Wikipedia article a long time ago, but it did not leave any impression in my memory. Now rereading it, I did not find it dramatic, but I see your point.
Tbh, I stilĺ do not fully understand how Wikipedia works (that is, I do not have a model who determines how an article develops). And the “originated” (ok maybe that is only almost and not fully identical to “first grew”) is just what I got from the article. The problem with the association is that it is hard to definitely determine what even makes things mentionable, but once somebody publibly has to distance himself from something, this indicates a public kind of association.
Further reading the article, my impression is that it indeed cites things that in Wikipedia count as sources for its claims. If the impression of lesswrong is distorted, then this may be a problem of what kinds of thing on lesswrong are covered by media publications? Or maybe it is all just selective citing, but then it should be possible to cite other things.
In theory, Wikipedia strives to be impartial. In practice, the rules are always only as good as the judges who uphold them. (All legal systems involve some degree of human judgment somewhere in the loop, because it is impossible to write a set of rules that covers everything and doesn’t allow some clever abuse. That’s why we talk about the letter and the spirit of the law.)
How to become a Wikipedia admin? You need to spend a lot of time editing Wikipedia in a way other admins consider helpful, and you need to be interested in getting the role. (Probably a few more technical details I forgot.) The good thing is that by doing a lot of useful work you send a costly signal that you care about Wikipedia. The bad thing is that if certain political opinion becomes dominant among the existing admins, there is no mechanism to fix this bias; it’s actually the other way round, because edits disagreeing with the consensus would be judged as harmful, and would probably disqualify their author from becoming an admin in the future.
I don’t assume bad faith from most of Wikipedia editors. Being wrong about something feels the same from inside as being right; and if other people agree with you, that is usually a good sign. But if you have a few bad actors who can play it smart, who can pretend that their personal grudges are how they actually see the world… considering that other admins already see them as part of the same team, and the same political bias means they already roughly agree on who are the good guys and who are the bad guys… it is not difficult to defend their decisions in front of jury of their peers. An outsider has no chance in this fight, because the insider is fluent with local lingo. Whatever they want to argue, they can find a wiki-rule pointing in that direction; of course it would be just as easy for them to find a wiki-rule pointing in the opposite direction (e.g. if you want to edit an article about something you are personally involved with, you have a “conflict of interest”, which is a bad thing; if I want to do the same thing, my personal involvement makes me a “subject-matter expert”, which is a good thing; your repetitive editing of the article to make your point is “vandalism”, my repetitive editing of the article to make an opposite point is “reverting vandalism”); and then the other admins will nod and say: “of course, if this is what the wiki-rules say, our job is to obey them”.
The specific admin that is so obsessed with Less Wrong is David Gerard from RationalWiki. He keeps a grudge for almost a decade, when he added Less Wrong to his website as an example of pseudoscience, mostly because of the quantum physics sequence. After being explained that actually “many worlds” is one of the mainstream interpretations among the scientists, he failed to say oops, and continued in the spirit of: well, maybe I was technically wrong about the quantum thing, but still… and spent the last decade trying to find and document everything that is wrong with Less Wrong. (Roko’s Basilisk—a controversial comment that was posted on LW once, deleted by Eliezer along with the whole thread, then posted on RationalWiki as “this is what people at Less Wrong actually believe”. Because the fact that it was deleted is somehow a proof that deep inside we actually agree with it, but we don’t want the world to know. Neoreaction—a small group of people who enjoyed debating their edgy beliefs on Less Wrong, were considered entertaining for a while, then became boring and were kicked out. Again, the fact that they were not kicked out sooner is evidence of something dark.) Now if you look who makes most edits on the Wikipedia page about Less Wrong: it’s David Gerard. If you go through the edit history and look at the individual changes, most of them are small and innocent, but they are all in the same direction: the basilisk and neoreaction must remain in the article, no matter how minuscule they are from perspective of someone who actually reads Less Wrong; on the other hand, mentions of effective altruism must be kept as short as possible. All of this is technically true and defensible, but… I’d argue that the Less Wrong described by the Wikipedia article does not resemble the Less Wrong its readers know, and that we have David Gerard and his decade-long work to thank for this fact.
True, but most of the information in media originates from RationalWiki, where it was written by David Gerard. A decade ago, RationalWiki used to be quite high in google rankings, if I remember correctly; any journalist who did a simple background check would find it. Then he or she would ask about the juicy things in the interview, and regardless of the answer, the juicy things would be mentioned in the article. Which means that the next journalist would now find them both at RationalWiki and in the previous article, which means that he or she would again make a part of the interview about it, reinforcing the connection. It is hard to find an article about Less Wrong that does not mention Roko’s Basilisk, despite the fact that it is discussed here rarely, and usually in the context of “guys, I have read about this thing called Roko’s Basilisk in the media, and I can’t find anything about it here, could you please explain me what this is about?”
Part of this is the clickbait nature of media: given the choice between debating neoreaction and debating technical details of the latest decision theory, it doesn’t matter which topic is more relevant to Less Wrong per se, they know that their audience doesn’t care about the latter. And part of the problem with Wikipedia is that it is downstream of the clickbait journalism. They try to use more serious sources, but sometimes there is simply no other source on the topic.
Thanks for the history overview! Very interesting. Concerning the wikipedia dynamics, I agree that this is plausible, as it is a plausible development of nearly every volunteer organization, in particular if they try to be grassroots-democratic. The wikipedia-media problem is known (https://xkcd.com/978/) though in this particular case I was a bit surprised about the “original research” and “reliable source” distinction. Many articles there did not seem very “serious”. On the other hand, during this whole “lost in hyperspace”, I also found “A frequent poster to LessWrong was Michael Anissimov, who was MIRI’s media director until 2013.”(https://splinternews.com/the-strange-and-conflicting-world-views-of-silicon-vall-1793857715) which was news to me. In internet years, all this is so long ago that I did not have any such associations. (I would rather have expected lesswrong to be notable for demanding the dissolution of the WHO, but probably that is not yet clickbaity enough.)
It’s happened again: I’ve realized that one of my old beliefs (pre-LW) is just plain dumb.
I used to look around at all the various diet (Paleo, Keto, low carb, low fat, etc.) and feel angry at people for having such low epistemic standards. Like, there’s a new theory of nutrition every two years, and people still put faith in them every time? Everybody swears by a different diet and this is common knowledge, but people still swear by diets? And the reasoning is that “fat” (the nutrient) has the same name as “fat” (the body part people are trying to get rid of)?
Then I encountered the “calories in = calories out” theory, which says that the only thing you need to do to lose weight is to make sure that you burn more calories than you eat.
And I thought to myself, “yeah, obviously.”.
Because, you see, if the orthodox asserts X and the heterodox asserts Y, and the orthodox is dumb, then Y must be true!
Anyway, I hadn’t thought about this belief in a while, but I randomly remembered it a few minutes ago, and as soon as I remembered its origins, I chucked it out the window.
Oops!
(PS: I wouldn’t be flabbergasted if the belief turned out true anyway. But I’ve reverted my map from the “I know how the world is” state to the “I’m awaiting additional evidence” state.)
On one hand, yeah, obviously.
On the other hand, “burning calories” is not an elementary action. Suppose I tell you to burn 500 calories now; how exactly would you achieve it? If your answer is that you would exercise or do something physically demanding, such actions spend ATP in the cells, so what if you don’t have enough ATP in your muscle cells; what is your plan B for burning calories? From the opposite side, you can limit the intake of calories. What if your metabolism is such that if you don’t provide enough calories, you will gradually fall in coma and die.
Your metabolism can make it impossible for you to reduce the “calories in” or increase the “calories out”, if it is somehow set up in a way that does not convert the calories into useful energy in your muscle cells, and it starts damaging your organs if the calories are missing in general. So while the theory is almost tautologically true, it may still be impossible to use it to lose weight. And the problem is that the proponents of “calories in = calories out” usually smugly pretend that it is an actionable advice, instead of mere description.
The actionable advice needs to be about how the metabolism works. And the things that impact it, such as what you eat, and who knows what else. Also, if you have some hormonal imbalance, or alergy, or whatever, your metabolism may differ from other people even if you eat the same things and try to live the same lifestyle. So, while e.g. eating less refined sugar would probably help everyone, no advice would guarantee a perfect outcome for everyone.
You make a good point—even if my belief was technically true, it could still have been poorly framed and inactionable (is there a name for this failure mode?).
But in fact, I think it’s not even obvious that it was technically true. If we say “calories in” is the sum of the calorie counts on the labels of each food item you eat (let’s assume the labels are accurate) then could there not still be some nutrient X that needs to be present for your body to extract the calories? Say, you need at least an ounce of X to process 100 calories? If so, then one could eat the same amount of food, but less X, and potentially lose weight.
Or perhaps the human body can only process food between four and eight hours after eating it, and it doesn’t try as hard to extract calories if you aren’t being active, so scheduling your meals to take place four hours before you sit around doing nothing would make them “count less”.
Calories are (presumably?) a measure of chemical potential energy, but remember that matter itself can also be converted into energy. There’s no antimatter engine inside my gut, so my body fails to extract all of the energy present in each piece of food. Couldn’t the mechanism of digestion also fail to extract all the chemical potential energy of species “calorie”?
Yes, there are many steps in the metabolism that are not under your conscious control. I am not an expert, so I don’t want to speculate too much about the technical details, but I think that gut bacteria probably also play a role. Simply, not everything you put in your mouth ends up necessarily in your bloodstream, and not everything that you absorbed is necessarily available in form of muscle energy.
I don’t know any standard name. Seems to me the problem is confusing “rephrasing of the desired outcome” with “an algorithm that actually get you there”. Something like:
Q: How can I get lot of money?
A: Become a millionaire!
Like, yeah, technically, everyone who successfully followed this advice ended up with lots of money, and everyone who didn’t can be accused of not following the advice properly, but that’s simply because those are two ways to describe the same thing.
Q: How can I lose weight?
A: Get rid of the extra atoms! I mean, extra calories!
Charitably, the advice is not absolutely bad, because for a hypothetical completely clueless listener it would provide some little information. But then, using this advice in practice means insinuating that your target is completely clueless, which is probably not be the case.
But atoms aren’t similar to calories, are they? I maintain that this hypothesis could be literally false, rather than simply unhelpful.
Okay, it’s not the same. But the idea is that the answer is equally unhelpful, for similar reasons.
I want to give a big thumbs up of positive reinforcement. I thinks its great that I got to read an “oops! That was dumb, but now I’ve changed my mind.”
Thanks for helping to normalize this.
Thanks for the feedback! Here’s another one for ya. A relatively long time ago I used to be pretty concerned about Pascal’s wager, but then I devised some clever reasoning why it all cancels out and I don’t need to think about it. I reasoned that one of three things must be true:
I don’t have an immortal soul. In this case, I might as well be a good person.
I have an immortal soul, and after my bodily death I will be assigned to one of a handful of infinite fates, depending on how good of a person I was. In this case it’s very important that I be a good person.
Same as above, but the decision process is something else. In this case I have no way of knowing how my infinite fate will be decided, so I might as well be a good person during my mortal life and hope for the best.
But then, post-LW, I realized that there are two issues with this:
It doesn’t make any sense to separate out case 2 from the enormous ocean of possibilities allowed for by case 3. Or rather, I can separate it, but then I need to probabilistically penalize it relative to case 3, and I also need to slightly shift the “expected judgment criterion” found in case 3 away from “being a good person is the way to get a good infinite fate”, and it all balances out.
More importantly, this argument flippantly supposes that I have no way of discerning what process, if any, will be used to assign me an infinite fate. An infinite fate, mind you. I ought to be putting in more thought than this even if I thought the afterlife only lasted an hour, let alone eternity.
So now I am back to being rather concerned about Pascal’s wager, or more generally, the possibility that I have an immortal soul and need to worry about where it eventually ends up.
From my first read-through of the sequences I remember that it claims to show that the idea of there being a god is somewhat nonsensical, but I didn’t quite catch it the first time around. So my first line of attack is to read through the sequences again, more carefully this time, and see if they really do give a valid reason to believe that.
I think the first step is to ask yourself what do you even mean by saying “god”.
Because if you have a definition like “the spirit of our dead chieftain, who sometimes visits me in my dreams”, I have no problem with that. Like, I find it completely plausible that you see the image of the chieftain in your dreams; nothing unscientific about that. It’s just that the image is generated by your brain, so if try to communicate with it, it can only give you advice that your brain generated, and it can only grant your wishes in the sense that you accomplish that yourself. But if you agree that this is exactly what you meant, then such god is perfectly okay for me.
But the modern definition is more like: an intelligent being that exists outside of our universe, but can observe it and change it. Suspending disbelief for a moment; how exactly can a being be “intelligent”, whether inside our universe or not? Intelligence implies processing data. That would imply that the god has some… supernatural neurons, or some other architecture capable of processing data. So the god is a mechanism (in the sense that a human is a biological mechanism) composed of parts, although those parts do not necessarily have to be found in our physics. Skill kinda plausible, maybe gods are made out of dark matter, who knows. But then, how did this improbably complicated mechanism come into existence? Humans were made by evolution, were gods too? But then again those gods are not the gods of religion; they are merely powerful aliens. But powerful aliens are neither creators of the universe, nor are they omniscient.
Then the theologists try to come with some smart-sounding arguments, like “god is actually supremely simple”—therefore Occam’s razor does not disprove him, and no evolution was needed, because simple things are more likely a priori, so a supreme being is supremely likely. Or something like that. But this is nonsense, because “supremely simple” and “capable of processing information” are incompatible.
The last argument would be: okay, we do not have a coherent explanation of god, but you don’t have a coherent explanation how the universe can exist without one. And a potential answer is that maybe existence is relative. Like, if you run a simulation of Conway’s Game of Life, it could possibly contain evolved intelligent life, and that life would ask “how was our universe created”? And it’s not like you created the universe by simulating it, because you are merely following the mathematical rules; so it’s more like the math created that universe and you are only observing it. If the beings in that mathematical universe will pray to gods, there is no way for anyone outside to intervene (while simultaneously following the mathematical rules). So the universe inside the Game of Life is a perfectly godless universe, based on math. But does it exist? Well, that rules of math describe the intelligent beings inside that universe that live, think, interact with their world. Maybe that’s all that is needed for existence; or rather, the question is what else could be needed? And then, maybe our world is also just an implication of some mathematical equations.
tl;dr—maybe math itself implies existence, and if you try to turn the concept of (intelligent, preceiving, acting) god into something coherent it is no longer the god of religion but merely a powerful alien
The supernatural isn’t supposed to be the natural done all over again. The typical theological claim is that God’s wisdom or whatever is an intrinisic quality, not something with moving parts.
Well, “wisdom as an intrinsic quality” is a mysterious answer. And what is “wisdom without moving parts”? A precomputed table of wise answers to all possible questions? Who precomputed it and how?
I agree that this is how theology usually answers it, but it is an answer that doesn’t make any sense when you look at it closer; it’s just some good-sounding words randomly glued together. And if you try to make it refer to something, even a hypothetical something, the whole explanation falls apart.
Reductionism is a combination of three claims.
That many thing are made of smaller components
That those particular can be understood in terms of the operations of their components
There’s an irreduciby basic level. Its not turtles all way down.
If it’s always the case that something that isn’t explicable in terms of its parts is mysterious, then the lowest level us mysterious. If nothing is mysterious, if you apply the argument against mysterious answer without prejudice, reductionism is false. There isn’t a consistent set of principles here.
Continued ..
Naturalism is the claim that there is a bunch of fundamental properties that just are, at the bottom of the stack ,and everything is built up from that. Supernaturalism is the claim that the intrinsic stuff is at the top of the stack, and everything else is derived from it top-down. That may be 100% false , but it is the actual claim.
There’s a thing called the principle of charity , where you one party interprets the others statements so as to maximise their truth value. This only enhances communication if the truth is not basically in dispute...that’s the easy case. The hard case is when there is a basic dispute about whats true. In that case, it’s not helpful to fix the other person’s claims by making them more reasonable from your point of view.
Anyway, thats how we ended up with “God must have superneurons in his superbrain”.
Feels like in the top-down universe, science shouldn’t work at all. I mean, when you take a magnifying glass and look at the details, they are supposedly generated on the fly to fit the larger picture. Then you apply munchkinry to the details and invent an atomic bomb or quantum computer… which means… what exactly, from the top-down perspective?
Yeah, you can find an excuse, e.g. that some of those top-down principles are hidden like Easter eggs, waiting to be discovered later. That the Platonic idea of smartphones has been waiting for us since the creation of the universe, but was only revealed to the recent generation. Which would mean that the top-down universe has some reason to pretend to be bottom-up, at least in some aspects...
Okay, the same argument could be made that quantum physics pretends to be classical physics at larger scale, or relativity pretends to be Newtonian mechanics at low speeds… as if the scientists are trying to make up silly excuses for why their latest magic works but totally “doesn’t contradict” what the previous generations of scientists were telling us...
Well, at least it seems like the bottom-up approach is fruitful, whether the true reason is that the universe is bottom-up, or that the universe it top-down in a way that tries really hard to pretend that it is actually bottom-up (either in the sense that when it generates the—inherently meaningless—details for us, it makes sure that all consequences of those details are compatible with the preexisting Platonic ideas that govern the universe… or like a Dungeon Master who allows the players to invent all kinds of crazy stuff and throw the entire game off balance, because he values consistency above everything).
More importantly, in universe where there is magic all the way up, what sense does it make to adopt the essentially half-assed approach, where you believe in the supernatural but also kinda use logic except not too seriously… might as well throw the logic away completely, because in that kind of universe it is not going to do you much good anyway.
The basic claim of a top down universe is a short string that doesn’t contain much information. About the same amount as a basic claim of reductionism.
The top down claim doesnt imply a universe of immutable physical law, but it doesn’t contradict it either.
The same goes for the bottom-up claim. A universe of randomly moving high entropy gas is useless for science and technology, but compatible with reductionism.
But all this is rather beside the point. Even if supernaturalism is indefensible, you can’t refute it by changing it into something else.
I wouldn’t call the dead chieftain a god—that would just be a word game.
Wait wait! You say a god-like being created by evolution cannot be a creator of the universe. But that’s only true if you constrain that particular instance of evolution to have occered in *this* universe. Maybe this universe is a simulation designed by a powerful “alien” in another universe, who itself came about from an evolutionary process in its own universe.
It might be “omniscient” in the sense that it can think 1000x as fast as us and has 1000x as much working memory and is familiar with thinking habits that are 1000x as good as ours, but that’s a moot point. The real thing I’m worried about isn’t whether there exists an omniscient-omnipotent-benevolent creature, but rather whether there exists *some* very powerful creature who I might need to understand to avoid getting horrible outcomes.
I haven’t yet put much thought into this, since I only recently came to believe that this topic merits serious thought, but the existence of such a powerful creature seems like a plausible avenue to the conclusion “I have an infinite fate and it depends on me doing/avoiding X”.
This is another area where my understanding could stand to be improved (and where I expect it will be during my next read-through of the sequences). I’m not sure exactly what kind of simplicity Occam’s razor uses. Apparently it can be formalized as Kolmogorov complexity, but the only definition I’ve ever found for that term is “the Kolmogorov Complexity of X is the length of the shortest computer program that would output X”. But this definition is itself in need of formalization. Which programming language? And what if X is something other than a stream of bits, such as a dandelion? And even once that’s answered, I’m not quite sure how to arrive at the conclusion that Kolmogorov-ly simpler things are more likely to be encountered.
(All that being said, I’d like to note that I’m keeping in mind that just because I don’t understand these things doesn’t mean there’s nothing to them. Do you know of any good learning resources for someone who has my confusions about these topics?)
That much makes sense, but I think it excludes a possibly important class of universe that is based on math but also depends on a constant stream of data from an outside source. Imagine a Life-like simulation ruleset where the state of the array of cells at time T+1 depended on (1) the state of the array at time T and (2) the on/off state of a light switch in my attic at time T. I could listen to the prayers of the simulated creatures and use the light switch to influence their universe such that they are answered.
Some people in history did, specifically, ancient Romans. But now we don’t. Just making it obvious.
And this is similar to the chieftain thing. You can have a religion that defines “god” as “an alien from another universe”, but many religions insist that god is not created but eternal.
Yes, this is a practical approach. Well, you cannot disprove such thing, because it is logically possible. (Obviously, “possible” does not automatically imply “it happened”.) But unless you assume it is “simulations all the way up”, there must be a universe that is not created by an external alien lifeform. Therefore, it is also logically possible that our universe is like that.
There is no reason to assume that the existing religions have something in common with the simulating alien. When I play Civilization, the “people” in my simulation have a religion, but it doesn’t mean that I believe in it, or even approve it, or that I am somehow going to reward them for it.
It’s just a cosmic horror that you need to learn to live with. There are more.
Any programming language; for large enough values it doesn’t matter. If you believe that e.g. Python is much better in this regard than Java, then for sufficiently complicated things the most efficient way to implement them in Java is to implement a Python emulator (a constant-sized piece of code) and implementing the algorithm in Python. So if you chose a wrong language, you pay at most a constant-sized penalty. Which is usually irrelevant, because these things are usually applied to debating what happens in general when the data grow.
But I agree that if you talk about small data, it is underspecified. I don’t really know what could it mean to “have a universe defined by the following three bits: 0, 0, 1”, and maybe no one has a meaningful answer to this. But there are cases where you can have an intuition that for any reasonable definition of a programming language, X should be simpler than Y.
Just an intuition pump: Imagine that there is a multiverse A containing all universes that can be run by programs having exactly 100 lines of code; and there is a multiverse B containing all universes that can be run by programs having exactly 105 lines of code. The universes from A are more likely, because they also appear in B.
For each program A1 describing a universe from A, you have a set of programs describing exactly the same universe in B, simply by adding an “exit” instruction on line 101, and arbitrary instructions on lines 102-105. If we take a program B1 describing a universe from B in such way that each of the 105 lines is used meaningfully, then… in multiverse A we have one A1 vs zero B1, and in multiverse B we have many A1-equivalents vs one B1; in either case A1 wins.
A part of this is in standard computer science curriculum, and another part is a philosophical extrapolation. I do not have a recommendation about a specific textbook, I just vaguely remember things like this from university. (Actually, I probably never heard explicitly about Kolmogorov complexity at university, but I learned some related concepts that allowed me to recognize what it means and what it implies, when I found it on Less Wrong.)
Then I would say the Kolmogorov complexity of the “Life-only” universe is the complexity of the rules of Life; but the complexity of the “Life+inputs” universe is the complexity of the rules plus the complexity of whoever generates the inputs, and everything they depend on, so it is the complexity of the outside universe.
The Civ analogy makes sense, and I certainly wouldn’t stop at disproving all actually-practiced religions (though at the moment I don’t even feel equipped to do that).
Are you sure it’s logically possible in the strict sense? Maybe there’s some hidden line of reasoning we haven’t yet discovered that shows that this universe isn’t a simulation! (Of course, there’s a lot of question-untangling that has to happen first, like whether “is this a simulation?” is even an appropriate question to ask. See also: Greg Egan’s book Permutation City, a fascinating work of fiction that gives a unique take on what it means for a universe to be a simulation.)
This sounds like the kind of thing someone might say who is already relatively confident they won’t suffer eternal damnation. Imagine believing with probability at least 1/1000 that, if you act incorrectly during your life, then...
(WARNING: graphic imagery) …upon your bodily death, your consciousness will be embedded in an indestructible body and put in a 15K degree oven for 100 centuries. (END).
Would you still say it was just another cosmic horror you have to learn to live with? If you wouldn’t still say that, but you say it now because your probability estimate is less than 1/1000, how did you come to have that estimate?
The constant-sized penalty makes sense. But I don’t understand the claim that this concept is usually applied in the context of looking at how things grow. Occam’s razor is (apparently) formulated in terms of raw Kolmogorov complexity—the appropriate prior for an event X is 2^(-B), where B is the Kolmogorov Complexity of X.
Let’s say general relativity is being compared against Theory T, and the programming language is Python. Doesn’t it make a huge difference whether you’re allowed to “pip install general-relativity” before you begin?
I agree that these intuitions can exist, but if I’m going to use them, then I detest this process being called a formalization! If I’m allowed to invoke my sense of reasonableness to choose a good programming language to generate my priors, why don’t I instead just invoke my sense of reasonableness to choose good priors? Wisdom of the form “programming languages that generate priors that work tend to have characteristic X” can be transformed into wisdom of the form “priors that work tend to have characteristic X”.
I have to admit that I kind of bounced off of this. The universe-counting argument makes sense, but it doesn’t seem especially intuitive to me that the whole of reality should consist of one universe for each computer program of a set length written in a set language.
Can I ask which related concepts you mean?
Oh, that makes sense. In that case, the argument would be that nothing outside MY universe could intervene in the lives of the simulated Life-creatures, since they really just live in the same universe as me. But then my concern just transforms into “what if there’s a powerful entity living in this universe (rather than outside of it) who will punish me if I do X, etc”.
I find it difficult to imagine how such argument could even be constructed. “Our universe isn’t a simulation because it has property X” doesn’t explain why the simulator could not simulate X. The usual argument is “because quantum stuff, the simulation would require insane amounts of computing power”, which is true, but we have no idea what the simulating universe looks like, and what kind of physics is has… maybe what’s an insane amount for us is peanuts for them.
But maybe there is some argument why a computing power in principle (like, some mathematical reason) cannot exceed certain value, ever. And the value may turn out to be insufficient to simulate our universe. And we can somehow make sure that our entire universe is simulated in sufficient resolution (not something like: the Earth or perhaps the entire Solar system is simulated in full quantum physics, but everything else is just a credible approximation). Etc. Well, if such thing happens, then I would accept the argument.
Yeah, I just… stopped worrying about these kinds of things. (In my case, “these kinds of things” refer e.g. to very unlikely Everett branches, which I still consider more likely than gods.) You just can’t win this game. There are million possible horror scenarios, each of them extremely unlikely, but each of them extremely horrifying, so you would just spend all your life thinking about them; and there is often nothing you can do about them, in some cases you would be required to do contradictory things (you spend your entire life trying to appease the bloodthirsty Jehovah, but it turns out the true master of universe is the goddess Kali and she is very displeased with your Christianity...) or it could be some god you don’t even know because it is a god of Aztecs, or some future god that will only be revealed to humanity in year 3000. Maybe humans are just a precursor to an intelligent species that will exist million years in the future, and from their god’s perspective humans are even less relevant than monkeys are for Christianity. Maybe we are just meant to be food for the space locusts from Andromeda galaxy. Maybe our entire universe is a simulation on a computer in some alien universe with insane computer power, but they don’t care about the intelligent beings or life in general; they just use the flashing galaxies as a screen saver when they are bored. If you put things into perspective, assigning probability 1/1000 to any specific religion is way too much; all kinds of religions, existing and fictional put together, don’t deserve that much.
And by the way, torturing people forever, because they did not believe in your illogical incoherent statements unsupported by evidence, that is 100% compatible with being an omnipotent, omniscient, and omnibenevolent god, right? Yet another theological mystery...
On the other hand, if you assume an evil god, then… maybe the holy texts and promises of heaven are just a sadistic way he is toying with us, and then he will torture all of us forever regardless.
So… you can’t really win this game. Better to focus on things where you actually can gather evidence, and improve your actual outcomes in life.
Psychologically, if you can’t get rid of the idea of supernatural, maybe it would be better to believe in an actually good god. Someone who is at least as reasonable and good as any of us, which should not be an impossibly high standard. Such god certainly wouldn’t spend an eternity torturing random people for “crimes” such as being generally decent people but believing in a wrong religion or no religion at all, or having extramarital sex, etc. (Some theologians would say that this is actually their god. I don’t think so, but whatever.) I don’t really believe in such god either, honestly, but it is a good fallback plan when you can’t get rid of the idea of gods completely.
That would be cheating, obviously. Unless by the length of code you mean also the length of all used libraries, in which case it is okay. It is assumed that the original programming language does not favoritize any specific theory, just provides very basic capabilities for expressing things like “2+2” or “if A then B else B”. (Abstracting from details such as how many pluses are in one “if”.)
Yeah, ok. The meaning of all this is, how can we compare “complexity” of two things where neither is a strict subset of the other. The important part is that “fewer words” does not necessarily mean “smaller complexity”, because it allows obvious cheating (invent a new word that means exactly what you are arguing for, and then insist that your theory is really simple because it could be described by one word—the same trick as with importing a library), but even if it is not your intention to cheat, your theory can still benefit from some concepts having shorter words for historical reasons, or even because words related to humans (or life on Earth in general) are already shorter, so you should account for the complexity that is already included for them. Furthermore it should be obvious to you that “1000 different laws of physics” is more complex than “1 law of physics, applied in the same way to 1000 particles”. If this all is obvious to you, than yes, making the analogy to a programming language does not bring any extra value.
But historical evidence shows that humans are quite bad at this. They will insist that stars are just shining dots in the sky instead of distant solar systems, because “one solar system + thousands of shining dots” seems to them less complex than “thousands of solar systems”. They will insist that Milky Way or Andromeda cannot be composed of stars, because “thousand stars + one Milky Way + one Andromeda” seems to them less complex than “millions of stars”. More recently (and more controversially), they will insist that “quantum physics + collapse + classical physics” is less complex than “quantum physics all the way up”. Programming analogy helps to express this in a simple way: “Complexity is about the size of code, not how large values are stored in the variables.”
Compression (lossless), like in the “zip” files. Specifically the fact that an average random file cannot be compressed, no matter how smart is the method used. Like, for any given value N, the number of all files with size N is larger than the number of all files with size smaller than N. So, whatever is your compression method, if it is lossless, it needs to be a bijection, so there must be at least one file of size N that it will not compress into a file of smaller size. Even worse, for each file of size N compressed to a smaller size, there must be a file of size smaller than N that gets “compressed” to a file of size N or more.
So how does compression work in real world? (Because experience shows it does.) It ultimately exploits the fact that most of the files we want to compress are not random, so it is designed to compress non-random files into smaller ones, and random (containing only noise) files into slightly larger ones. Like, you can try it at home; generate a one megabyte file full of randomly generated bits, then try all compression programs you have installed, and see what happens.
Now, each specific compression algorithm recognized and exploits some kinds of regularity, and is blind towards others. This is why people sometimes invent better compression algorithms that exploit more regularities. The question is, what is the asymptote of this progress? If you tried to invent the best compression algorithm ever, one that could exploit literally all kinds of non-randomness, what would it look like?
(You may want to think about this for a moment, before reading more.)
The answer is that the hypothetical best compression algorithm ever would transform each file into the shortest possible program that generates this file. There is only one problem with that: finding the shortest possible program for every input is algorithmically impossible—this is related to halting problem. Regardless, if the file is compressed by this hypothetical ultimate compression algorithm, the size of the compressed file would be the Kolmogorov complexity of the original file.
Then we are no longer talking about gods in the modern sense, but about powerful aliens.
Congratulations—noticing that you are confused is an important step!
What are you doing while “awaiting additional evidence”? This is a topic that doesn’t have a neutral/agnostic position—biology forces you to eat, you have some influence (depending on your willpower model) over what and how much.
This belief wasn’t really affecting my eating habits, so I don’t think I’ll be changing much. My rules are basically:
No meat (I’m a vegetarian for moral reasons).
If I feel hungry but I can see/feel my stomach being full by looking at / touching my belly, I’m probably just bored or thirsty and I should consider not eating anything.
Try to eat at least a meal’s worth of “light” food (like toast or cereal as opposed to pizza or nachos) per day. This last rule is just to keep me from getting stomach aches, which happens if I eat too much “heavy” food in too short a time span.
I think I might contend that this kind of reflects an agnostic position. But I’m glad you asked, because I hadn’t noticed before that rule 2 actually does implicitly assume some relationship between “amount of food” and “weight change”, and is put in place so I don’t gain weight. So I guess I should really have said that what I tossed out the window was the extra detail that calories alone determine the effect food will have on one’s weight. I still believe, for normal cases, that taking the same eating pattern but scaling it up (eating more of everything but keeping the ratios the same) will result in weight gain.
Are rationalist ideas always going to be offensive to just about everybody who doesn’t self-select in?
One loved one was quite receptive to Chesterton’s Fence the other day. Like, it stopped their rant in the middle of its tracks and got them on board with a different way of looking at things immediately.
On the other hand, I routinely feel this weird tension. Like to explain why I think as I do, I‘d need to go through some basic rational concepts. But I expect most people I know would hate it.
I wish we could figure out ways of getting this stuff across that was fun, made it seem agreeable and sensible and non-threatening.
Less negativity—we do sooo much critique. I was originally attracted to LW partly as a place where I didn’t feel obligated to participate in the culture war. Now, I do, just on a set of topics that I didn’t associate with the CW before LessWrong.
My guess? This is totally possible. But it needs a champion. Somebody willing to dedicate themselves to it. Somebody friendly, funny, empathic, a good performer, neat and practiced. And it needs a space for the educative process—a YouTube channel, a book, etc. And it needs the courage of its convictions. The sign of that? Not taking itself too seriously, being known by the fruits of its labors.
Traditionally, things like this are socially achieved by using some form of “good cop, bad cop” strategy. You have someone who explains the concepts clearly and bluntly, regardless of whom it may offend (e.g. Eliezer Yudkowsky), and you have someone who presents the concepts nicely and inoffensively, reaching a wider audience (e.g. Scott Alexander), but ultimately they both use the same framework.
The inoffensiveness of Scott is of course relative, but I would say that people who get offended by him are really not the target audience for rationalist thought. Because, ultimately, saying “2+2=4” means offending people who believe that 2+2=5 and are really sensitive about it; so the only way to be non-offensive is to never say anything specific.
If a movement only has the “bad cops” and no “good cops”, it will be perceived as a group of assholes. Which is not necessarily bad if the members are powerful; people want to join the winning side. But without actual power, it will not gain wide acceptance. Most people don’t want to go into unnecessary conflicts.
On the other hand, a movement with “good cops” without “bad cops” will get its message diluted. First, the diplomatic believers will dilute their message in order not to offend anyone. Their fans will further dilute the message, because even the once-diluted version is too strong for normies’ taste. At the end, the message may gain popular support… kind of… because the version that gains the popular support will actually contain maybe 1% of the original message, but mostly 99% of what the normies already believed, peppered by the new keywords.
The more people will present rationality using different methods, the better. Because each of them will reach a different audience. So I completely approve the approach you suggest… in addition to the existing ones.
You’re both assuming that you have a set of correct ideas coupled with bad PR...but how well are Bayes, Aumann and MWI (eg.) actually doing?
You’re right.
I need to try a lot harder to remember that this is just a community full of individuals airing their strongly held personal opinions on a variety of topics.
Those opinions often have something in common—respect for the scientific method, effort to improve one’s rationality, concern about artificial intelligence—and I like to believe it is not just a random idiosyncratic mix (a bunch of random things Eliezer likes), but different manifestations of the same underlying principle (use your intelligence to win, not to defeat yourself). However, not everyone is interested in all of this.
And I would definitely like to see “somebody friendly, funny, empathic, a good performer, neat and practiced” promoting these values in a YouTube channel or in books. But that requires a talent I don’t have, so I can only wait until someone else with the necessary skills does it.
This reminded me of the YouTube channel of Julia Galef, but the latest videos there are 3 years old.
I believe that if the rational concepts are pulling their weight, it should be possible to explain the way the concept is showing up concretely in your thinking, rather than justifying it in the general case first.
As an example, perhaps your friend is protesting your use of anecdotes as data, but you wish to defend it as Bayesian, if not scientific, evidence. Rather than explaining the difference in general, I think you can say “I think that it’s more likely that we hear this many people complaining about an axe murderer downtown if that’s in fact what’s going on, and that it’s appropriate for us to avoid that area today. I agree it’s not the only explanation and you should be able to get a more reliable sort of data for building a scientific theory, but I do think the existence of an axe murderer is a likely enough explanation for these stories that we should act on it”
If I’m right that this is generally possible, then I think this is a route around the feeling of being trapped on the other side of an inferential gap (which is how I interpreted the ‘weird tension’)
I think you’re right, when the issue at hand is agreed on by both parties to be purely a “matter of fact.”
As soon as social or political implications crop in, that’s no longer a guarantee.
But we often pretend like our social/political values are matters of fact. The offense arises when we use rational concepts in a way that gives the lie to that pretense. Finding an indirect and inoffensive way to present the materials and let them deconstruct their pretenses is what I’m wishing for here. LW has a strong culture surrounding how these general-purpose tools get applied, so I’d like to see a presentation of the “pure theory” that’s done in an engaging way not obviously entangled with this blog.
The alternative is to use rationality to try and become savvier social operators. This can be “instrumental rationality” or it can be “dark arts,” depending on how we carry it out. I’m all for instrumental rationality, but I suspect that spreading rational thought further will require that other cultural groups appropriate the tools to refine their own viewpoints rather than us going out and doing the convincing ourselves.
The term for the “fear of truth” is alethophobia. I’m not familiar of many other great terms in this area (curious to hear suggestions).
Apparently “Epistemophobia” is a thing, but that seems quite different; Epistemophobia is more the fear of learning, rather than the fear of facing the truth.
One given definition of alethophobia is,
“The inability to accept unflattering facts about your nation, religion, culture, ethnic group, or yourself”
This seems like a incredibly common issue, one that is especially talked about as of recent, but without much specific terminology.
https://www.yourdictionary.com/alethophobia
https://books.google.co.uk/books?id=teUEAAAAQAAJ&pg=PA10&dq=alethophobia&redir_esc=y&hl=en&fbclid=IwAR19KAulFSCERMWGR2UCDyiUQrF5ai_PpgxV-cQ0vM7ggaBaHQQlHhuy9sU#v=onepage&q=alethophobia&f=false
Can you give examples of things you think would fit under this? It seems that there are lots of instances of being resistant to the truth, but I can think of very few that I would categorize as fear of truth. It’s often fear of something else (e.g. fear of changing your identity) or biases (e.g. the halo effect or consistency bias) that cause people to resist. I can think of very few cases where people have a general fear of truth.
Without looking, I’ll predict it’s a neologism—invented in the last ~25 years to apply to someone the inventor disagrees with. Yup, google n-grams shows 0 uses in any indexed book. A little more searching shows almost no actual uses anywhere—mostly automated dictionary sites that steal from each other, presumably with some that accept user submissions.
I did find one claim to invention, in 2017: http://www.danielpipes.org/comments/236053 . Oh, and earlier (2008), there’s a book with that title: https://www.amazon.com/Alethophobia-Manoucher-Parvin/dp/1588140474 .
I still submit that this is a word in search of a need, which mostly exists as a schoolyard insult dressed up in Latin.
Yea, I also found the claim, as well as a few results from old books before the claim. The name come straight from the Latin though, so isn’t that original or surprising.
Just because it hasn’t been used much before doesn’t mean we can’t start to use it and adjust the definition as we see fit. I want to see more and better terminology in this area.
> The name comes straight from the Latin though
From the Greek as it happens. Also, alethephobia would be a double negative, with a-letheia meaning a state of not being hidden; a more natural neologism would avoid that double negative. Also, the greek concept of truth has some differences to our own conceptualization. Bad neologism.
Ah, good to know. Do you have recommendations for other words?
The trend of calling things that aren’t fears “-phobia” seems to me a trend that’s harmful for clear communication. Adjusting the definition only leads to more confusion.
I think I want less terminology in this area, and generally more words and longer descriptions for things that need nuance and understanding. Dressing up insults as diagnoses doesn’t help any goals I understand, and jargon should only be introduced as part of much longer analyses where it illuminates rather than obscures.
An interesting thing about Supernatural Fitness (a VR app kinda like Beat Saber) is that they are leaning hard into being a fitness app rather than a game. You don’t currently get to pick songs, you pick workouts, which come with pep talks and stretching and warmups.
This might make you go “ugh, I just wanna play a song” and go play Beat Saber instead. But, Supernatural Fitness is _way_ prettier and has some conceptual advances over Beat Saber.
And… I mostly endorse this and think it was the right call. I am sympathetic to “if you give people the ability to choose whatever, they mostly choose to be couch potatoes, or goodhart on simple metrics”, and if you want people to do anything complicated and interesting you need to design your app with weird constraints in mind.
(Example: LessWrong deliberately doesn’t show users the view-count of their posts. We already have the entire internet as the control group for what happens if you give people view-counts – they optimize for views, and you get clickbait. Is this patronizing? Yeah. Am I 100% confident it’s the right call? No. But, I do think if you want to build a strong intellectual culture, it matters what kinds of Internet Points you give [or don’t give] people, and this is at least a judgment call you need to be capable of making)
But… I still do think it’s worth looking at third options. Sometimes, I might really want to just jam to some tunes, and I want to pick the specific
In the case of Supernatural Fitness, I think it is quite important that your opening experience puts you in the mindset of coaches and workouts, and that songs are clustered together in groups of 15+ minutes (so you actually get a cardio workout), and that they spend upfront effort teaching you the proper form for squats and encouraging you to maintain that form rather than “minimizing effort” (which I think Beat Saber ends up teaching you, and if you’re coming from Beat Saber, you might have already internalized habits around)
At first I thought “maybe they should make you learn proper form first, but eventually give you the ability to choose individual songs.” Then I simulated myself doing that, and thought “well, I would probably end up just doing less workouts.”
My current guess for my coherent-extrapolated-preference for the game is for something like “Individual song plays are more like rewards for actually completing a workout” (i.e. if you make it all the way through you get credits that can be spent on playing individual songs). Or, alternately, maybe players can assemble workouts out of at least 3 songs.
It’s probably complicated somewhat by the app licensing popular music, and the licenses might be temporary.
Epistemic status: really shaky, but I think there’s something here.
I naturally feel a lot of resistance to the way culture/norm differences are characterized in posts like Ask and Guess and Wait vs Interrupt Culture. I naturally want to give them little pet names, like:
Guess culture = “read my fucking mind, you badwrong idiot” culture.
Ask culture = nothing, because this is just how normal, non-insane people act.
I think this feeling is generated by various negative experiences I’ve had with people around me, who, no matter where I am, always seem to share between them one culture or another that I don’t really understand the rules of. This leads to a lot of interactions where I’m being told by everyone around me that I’m being a jerk, even when I can “clearly see” that their is nothing I could have done that would have been correct in their eyes, or that what they wanted me to do was impossible or unreasonable.
But I’m starting to wonder if I need to let go of this. When I feel someone is treating me unfairly, it could just be because (1) they are speaking in Culture 1, then (2) I am listening in Culture 2 and hearing something they don’t mean to transmit. If I was more tuned in to what people meant to say, my perception of people who use other norms might change.
I feel there’s at least one more important pair of cultures, and although I haven’t mentioned it yet, it’s the one I had in mind most while writing this post. Something like:
Culture 1: Everyone speaks for themselves only, unless explicitly stated otherwise. Putting words in someone’s mouth or saying that they are “implying” something they didn’t literally say is completely unacceptable. False accusations are taken seriously and reflect poorly on the accuser.
Culture 2: The things you say reflect not only on you but also on people “associated” with you. If X is what you believe, you might have to say Y instead if saying X could be taken the wrong way. If someone is being a jerk, you don’t have to extend the courtesy of articulating their mistake to them correctly; you can just shun them off in whatever way is easiest.
I don’t really know how real this dichotomy is, and if it is real, I don’t know for sure how I feel about one being “right” and the other being “wrong”. I tried semi-hard to give a neutral take on the distinction, but I don’t think I succeeded. Can people reading this tell which culture I naturally feel opposed to? Do you think I’ve correctly put my finger on another real dichotomy? Which set of norms, if either, do you feel more in tune with?
Is it because they’re expecting you to read their mind, and go along with their “culture”, instead of asking you?
I couldn’t parse this question. Which part are you referring to by “it”, and what do you mean by “instead of asking you”?
it (the negative experiences) - Are *they (the negative experiences) the result of (people with a “culture” who’s rules rules you don’t understand) expecting you to read *their mind, and go along with their “culture”, instead of asking you to go along with their culture?
Aha, no, the mind reading part is just one of several cultures I’m mentioning. (Guess Culture, to be exact.) If I default to being an Asker but somebody else is a Guesser, I might have the following interaction with them:
Me: [looking at some cookies they just made] These look delicious! Would it be all right if I ate one?
Them: [obviously uncomfortable] Uhm… uh… I mean, I guess so...
Here, it’s retroactively clear that, in their eyes, I’ve overstepped a boundary just by asking. But I usually can’t tell in advance what things I’m allowed to ask and what things I’m not allowed to ask. There could be some rule that I just haven’t discovered yet, but because I haven’t discovered it yet, it feels to me like each case is arbitrary, and thus it feels like I’m being required to read people’s minds each time. Hence why I’m tempted to call Guess Culture as “Read-my-mind Culture”.
(Contrast this to Ask Culture, where the rule is, to me, very simple and easy to discover: every request is acceptable to make, and if the other person doesn’t want you to do what you’re asking to do, they just say “no”.)
It might be hard to take a normative stance, but if culture 1 makes you feel better AND leads to better results AND helps people individuate and makes adults out of them, then maybe it’s just, y’know, better. Not “better” in the naive mistake-theorist assumption that there is such a thing as a moral truth, but “better” in the correct conflict-theorist assumption that it just suits you and me and we will exert our power to make it more widely adopted, for the sake of us and our enlightened ideals.
I’m experimenting with a format for applying LW tools to personal social-life problems. The goal is to boil down situations so that similar ones will be easy to diagnose and deal with in the future.
To do that, I want to arrive at an acronym that’s memorable, defines an action plan and implies when you’d want to use it. Examples:
OSSEE Activity—“One Short Simple Easy-to-Exit Activity.” A way to plan dates and hangouts that aren’t exhausting or recipes for confusion.
DAHLIA—“Discuss, Assess, Help/Ask, Leave, Intervene, Accept.” An action plan for how to deal with annoying behavior by other people. Discuss with the people you’re with, assess the situation, offer to help or ask the annoying person to stop, leave if possible, intervene if not, and accept the situation if the intervention doesn’t work out.
I came up with these by doing a brief post-mortem analysis on social problems in my life. I did it like this:
Describe the situation as fairly as possible, both what happened and how it felt to me and others.
Use LW concepts to generalize the situation and form an action plan. For example, OSSEE Activity arose from applying the concept of “diminishing marginal returns” to my outings.
Format the action plan into a mnemonic, such as an acronym.
Experiment with applying the action plan mnemonic in life and see if it leads you to behave differently and proves useful.
If you measure death-badness from behind the veil of ignorance, you’d naively prioritize well-liked, famous people with large families.
Would you prioritize the young from behind the veil of ignorance?
I’m annoyed that I think so hard about small daily decisions.
Is there a simple and ideally general pattern to not spend 10 minutes doing arithmetic on the cost of making burritos at home vs. buying the equivalent at a restaurant? Or am I actually being smart somehow by spending the time to cost out that sort of thing?
Perhaps:
“Spend no more than 1 minute per $25 spent and 2% of the price to find a better product.”
This heuristic cashes out to:
Over a year of weekly $35 restaurant meals, spend about $35 and an hour and a half finding better restaurants or meals.
For $250 of monthly consumer spending, spend a total of $5 and 10 minutes per month finding a better product.
For bigger buys of around $500 (about 2x/year), spend $10 and 20 minutes on each purchase.
Buying a used car ($15,000) I’d spend $300 and 10 hours. I could use the $300 to hire somebody at $25/hour to test-drive an additional 5-10 cars, a mechanic to inspect it on the lot, a good negotiator to help me secure a lower price.
For work over the next year ($30,000), spend $600 and 20 hours.
Getting a Master’s degree ($100,000 including opportunity costs), spend 66 hours and $2,000 finding the best school.
Choosing from among STEM career options ($100,000 per year), spend about 66 hours and $600 per year exploring career decisions.
Comparing that with my own patterns, that simplifies to:
Spend much less time thinking about daily spending. You’re correctly calibrated for ~$500 buys. Spend much more time considering your biggest buys and decisions.
For some (including younger-me), the opposite advice was helpful—I’d agonize over “big” decisions, without realizing that the oft-repeated small decisions actually had a much larger impact on my life.
To account for that, I might recommend you notice cache-ability and repetition, and budget on longer timeframes. For monthly spending, there’s some portion that’s really $120X decade spending (you can optimize once, then continue to buy monthly for the next 10 years), a bunch that’s probably $12Y of annual spending, and some that’s really $Z that you have to re-consider every month.
Also, avoid the mistake of inflexible permissions. Notice when you’re spending much more (or less!) time optimizing a decision than your average, but there are lots of them that actually benefit from the extra time. And lots that additional time/money doesn’t change the marginal outcome by much, so you should spend less time on.
I wonder if your problem as a youth was in agonizing over big decisions, rather than learning a productive way to methodically think them through. I have lots of evidence that I underthink big decisions and overthink small ones. I also tend to be slow yet ultimately impulsive in making big changes, and fast yet hyper-analytical in making small changes.
Daily choices have low switching and sunk costs. Everybody’s always comparing, so one brand at a given price point tends to be about as good as another.
But big decisions aren’t just big spends. They’re typically choices that you’re likely stuck with for a long time to come. They serve as “anchors” to your life. There are often major switching and sunk costs involved. So it’s really worthwhile anchoring in the right place. Everything else will be influenced or determined by where you’re anchored.
The 1 minute/$25 + 2% of purchase price rule takes only a moment’s thought. It’s a simple but useful rule, and that’s why I like it.
There are a few items or services that are relatively inexpensive, but have high switching costs and are used enough or consequential enough to need extra thought. Examples include pets, tutors, toys for children, wedding rings, mattresses, acoustic pianos, couches, safety gear, and textbooks. A heuristic and acronym for these exceptions might be CHEAPS: “Is it a Curriculum? Is it Heavy? Is it Ergonomic? Is it Alive? Is it Precious? Is it Safety-related?”
The problem with mansplaining -
Why do men mansplain and why do people (particularly women) hate it? People sometimes struggle to articulate what mansplaining is and why they dislike it, but I’m surely not the discoverer of this argument.
Recently I was talking to a colleage during a strategy game session. He said “You are bad because you made these mistakes” and I said “yes I am bad at these aspects of the game. Alsol, you should have invested more into anti-aircraft guns”. He immediately began repeating a list of mistakes I had made, as evidence that his investment in AA was optimal. But I have seen the AA formula so I made a technical argument for strong AA in the late game. He stepped back and said “Well you were trashtalking me so I had to...”.
Then I realized that we were not explaining the game to one another or discussing the best build. We were really having a dominance fight, through vaguely technical arguments. Once you realize that men do this, you see it fairly often. Recently I said that transaction costs were built into the bitcoin code from the beginning, and a friend argued back that only third party exchanges charge transaction fees. It took me a while to prove him wrong because the content was about dominance, not about bitcoin[1].
Meanwhile I’ve been learning to flirt with women. Originally I though that you cannot disagree with a women while flirting, since it is a dominance play ala man world. But actually you can disagree in a non-conflictive way. For example, on Tinder I asked a women what she reads, and she said “I don’t think reading indicates intelligence or curiosity”. I responded by saying “I accept the argument on intelligence, but I expect curiosity and reading are correlated. Otherwise would be too surprising”. I have found this type of disagreement actually gets a longer response (signal of interest). My theory is that the qualified disagreement shows intelligence, status and social intelligence.
1 - To his credit, he eventually accepted the technical argument
This is a great observation! But, as often happens in political debates, “mansplaining” is a motte-and-bailey term, which could mean the thing you just described, or it could mean “a man tried to say something”, or anything in between, depending on who used it and in which context.
Also, it is not an exclusively male behavior, despite the name. I have no strong opinion on whether it is a mostly male behavior, because I assume that most female dominance fights happen out of my sight, for various reasons. But I am pretty sure I have seen women fighting for dominance using supposedly factual arguments a few times.
Depends on whether the specific woman finds dominance attractive. And that probably also depends on the type/degree of dominance, her mood, and how well you know each other. Yes, this “partially agree, partially disagree” strategy seems like the golden middle way between being disagreeable and boring.
I agree with all of those points.
I think many women, perhaps a majority, find a more dominant man attractive. Basically ensure any fact-based dominance display doesn’t make the other person feel stupid. Good rule for lots of interactions.
With a term like mansplaining that’s used in quite different scopes by different people it would be helpful to start with a more clear definition before going into why people do it.
True. I should rephrase my thesis “What people often mean when they say “mansplaining” is explanations which are intended to express dominance rather than to mutually arrive at better understanding”.
“Medium Engagement Activities” are the death of culture creation.
Expecting someone to show up for a ~1-hour or more event every week that helps shape your culture is great for culture creation, or requiring them to wear a dress code—large commitments are good in the early stages.
Removing trivial inconveniences to following your values and rules is great for building culture, doing things that require no or low engagement but help shape group cohesion. Design does a lot here—no commitment tools to shape culture are great during early stages.
But medium commitment tools are awful, a series of little things that take 5-50 minutes a week to work on—these things are death to early stage cultures. It’s death by a 1000 cuts of things they can’t see clear immediate benefit for, and which they can see clear immediate cost for.
I don’t know why exactly this is, and haven’t really mapped out what’s behind this intuition, it’s something about the benefits of building identity vs. the time required, it’s ushaped, the tails are a much more effective tradeoff than the middle.
With some frequency, LW gets a new user writing a post that’s sort of… in the middle of having their mind blown by the prospect of quantum immortality and MWI. I’d like to have a single post to link them to that makes a fairly succinct case for “it adds up to normality”, and I don’t have a clear sense of what to do other that link to the entire Quantum Physics sequence.
Any suggestions? Or, anyone feel like writing said post if it doesn’t exist yet?
I wrote a thing about this.
https://www.lesswrong.com/posts/6wkY2DcCnzNyJTDsw/looking-for-answers-about-quantum-immortality?commentId=b3ZLzjSYWhHsMEYRr
The wavefunctioncollapse algorithm measures whichever tile currently has the lowest entropy. GPT-3 always just measures the next token. Of course in prose those are usually the same, but I expect some qualitative improvements once we get structured data with holes such that any might have low entropy, a transformer trained to fill holes, and the resulting ability to pick which hole to fill next.
Until then, I expect those prompts/GPT protocols to perform well which happen to present the holes in your data in the order that wfc would have picked, ie ask it to show its work, don’t ask it to write the bottom line of its reasoning process first.
Long shortform short: Include the sequences in your prompt as instructions :)