In the past, also, war was one of the main instruments by which human societies were kept in touch with physical reality.
(...)
In philosophy, or religion, or ethics, or politics, two and two might make five, but when one was designing a gun or an aeroplane they had to make four.
-- George Orwell, 1984
A pithy way of summarizing the above comment:
If someone tells you his cause is so important that lying for it is justified, assume he’s lying.
It only indicates that experienced rationalists and proto-rationalists treat their beliefs in different ways. Proto-rationalists form a belief, play with it in their heads, and then do whatever they were going to do anyway - usually some variant on what everyone else does. Experienced rationalists form a belief, examine the consequences, and then act strategically to get what they want.
Alternate hypothesis: the experienced rationalists are also doing what everyone else (in their community) is doing, they just consider a different group of people their community.
Opportunity is missed by most people because it is dressed in overalls and looks like work.
Thomas Edison
Also keep in mind that it’s more important to make your beliefs as correct as possible then to make them as consistent as possible. Of course the ultimate truth is both correct and consistent; however, it’s perfectly possible to make your beliefs less correct by trying to make them more consistent. If you have two beliefs that do a decent job of modeling separate aspects of reality, it’s probably a good idea to keep both around, even if they seem to contradict each other. For example, both General Relativity and Quantum Mechanics do a good job modeling (parts of) reality despite being inconsistent and we want to keep both of them. Now think about what happens when a similar situation arises in a field, e.g., biology, psychology, your personal life, where evidence is messier then it is in physics.
The scientific process has been so corrupted by signaling and politics that outside the hard sciences, most of what is called “science” these days, especially mainstream opinion at universities, is less entangled with reality then most religions. At least the religions have been around long enough to be subject memetic selection.
Nor of course must they ever be allowed to raise Aristotle’s question: whether “democratic behaviour” means the behaviour that democracies like or the behaviour that will preserve a democracy. For if they did, it could hardly fail to occur to them that these need not be the same.
(..)
Even if they don’t read Aristotle (that would be undemocratic) you would have thought the French Revolution would have taught them that the behaviour aristocrats naturally like is not the behaviour that preserves aristocracy. They might then have applied the same principle to all forms of government.
-- Screwtape, from “Screwtape Proposes a Toast” by C. S. Lewis.
I would simply like to point out the irony of having this discussion in a thread that is hidden by default due to being below a comment currently at −9.
Well, his point only makes any sense when applied to the metaphor since a better answer to the question
“Wait, Professor… If Sisyphus had to roll the boulder up the hill over and over forever, why didn’t he just program robots to roll it for him, and then spend all his time wallowing in hedonism?”
is:
“where would Sisyphus get a robot in the middle of Hades?”
Edit: come to think of it, this also works with the metaphor for human struggle.
One day, I was playing with an “express wagon,” a little wagon with a railing around it, I noticed something about the way the ball moved. I went to my father and said, “Say, Pop, I noticed something. When I pull the wagon, the ball rolls to the back of the wagon. And when I’m pulling it along and I suddenly stop, the ball rolls to the front of the wagon. Why is that?”
“That, nobody knows,” he said. “The general principle is that things which are moving tend to keep on moving, and things which are standing still tend to stand still, unless you push them hard. This tendency is called ‘inertia,’ but nobody knows why it’s true.” Now, that’s a deep understanding. He didn’t just give me the name.
-Richard Feynman
I’ve seen enough of the QM sequence and know enough QM to see that Eliezer stopped learning quantum mechanics before getting to density matrices. As a result, the conclusions he draws from QM rely on metaphysical assumptions and seem rather arbitrary if one knows more quantum mechanics. In the comments to this post Scott Aaronson tries to explain this to Eliezer without much success.
Somehow it seems appropriate that it’s hard to track down the originator of this idea.
Smart people often think social institutions are basically arbitrary and that they can engineer better ways using their mighty brains. Because these institutions aren’t actually arbitrary, their tinkering is generally harmful and sometimes causes social dysfunction, suffering, and death on a massive scale. Less Wrong is unusually bad in this regard, and that is a serious indictment of “rationality” as practiced by LessWrongers.
A case of this especially relevant to Less Wrong is “Evangelical Polyamory”.
Agreed except for the part about Less Wrong is unusually bad in this regard. I think it’s actually doing better then most gatherings of smart people attempting to reorganize society. Keep in mind lesswrong’s equivalent 50 years ago would have been advocating Marxism.
Atheists assume that self-identified atheists are representative of non-religious people and use flattering data about self-identified atheists to draw (likely) false conclusions about the world being better without religion. The expected value of arguing for atheism is small and quite possibly negative.
Agreed.
Ceteris paribus dictatorships work better than democracies.
You’ve never lived under a dictatorship have you? I strongly disagree with the above statement and think it’s another good example of your first point.
Nerd culture is increasingly hyper-permissive and basically juvenile and stultifying. Nerds were better off when they had to struggle to meet society’s expectations for normal behavior.
True, however, the previous culture was hyper-conformist, since it was ‘designed’ to create people intelligent enough to operate machinery but conformist enough to work in an assembly line.
Fix this bug.
To see why always maximizing expected utility is clearly bad, consider an action A with a 10^-10 chance of giving you 10^100 units of utility, and a 1-10^-10 chance of losing you 10^10 units of utility. Then expected utility maximization requires you to perform A, even though it is obviously a bad idea.
I don’t think you understand what the word utility means. In particular utility is not linear in money. If the you had a 10^-10 chance of giving you $10^100, and a 1-10^-10 chance of losing you $10^10, you would be correct. That’s because you exponentially discount the value of large amounts of money. However, utility is defined to already take the exponential discounting into account.
Unfortunately, your brain is wired to exponentially discount. Even though the utility values have already taken this into account, your intuition doesn’t realize this and wants to exponentially discount again.
Another way to see what’s going on is that your intuition is getting confused by the large numbers (after all 10^100 doesn’t look much bigger then 10^10). Since you didn’t specify what units you were measuring utility in, let’s rescale them by 10^10 and see what your statement looks like:
consider an action A with a 10^-10 chance of giving you 10^90 units of utility, and a 1-10^-10 chance of losing you 1 unit of utility.
Now it should hopefully be clearer why you do indeed want to perform action A.
Subsidizing the markers of status doesn’t produce the character traits that result in that status; it undermines them.
The many worlds interpretation of Quantum Mechanics is false in the strong sense that the correct theory of everything will incorporate wave-function collapse as a natural part of itself. ~40%
I can’t help think part of the difference is that they’re your books so you can do whatever you want to them, whereas he’s your employee and is being paid to do this the “right way”.
The problem with Yvain’s reply is that he omits the main reason why lying is a bad idea. Yvain compares lying to violence. I don’t think this is a good comparison. It’s acceptable to respond to violence with violence. It’s not a good idea to respond to lies with lies.
Eliezer touched on this issue in his post here where he pointed out that one problem with lying to support a cause is that you’d better be absolutely sure that all your beliefs about the cause and what to do for it are in fact correct. However, the problem is even worse, there is a vicious cycle here since a cause that frequently lies is much more likely to acquire incorrect beliefs.
Think about it this way: suppose you believe that your cause justifies lying, so you lie about it. Your lies attract people to your cause who believe those lies. They in turn make up further lies (that they think are justified based on the lies they believe to be true). And so no until your cause’s belief system is full of falsehoods and anti-epistomology. Your cause may ultimately “win” in the sense that it’s followers acquire power, but by that point said followers may no longer care about your original goal. Even if they do, they’re likely to have so many false beliefs that what they do to accomplish it is likely to be counter-productive and probably have other unpleasant side effects.
Note that the above argument applies to lying but not to violence. Thus in some sense lying for your cause is in fact worse than committing violence for it.
-- Milton Friedman