Felix qui potuit rerum cognoscere causas.
Virgil Kurkjian
Karma: 318
You’re entirely right that taking Herbert’s views on most specific subjects isn’t helpful. He was wrong about genetics, about education, and about a lot of things besides. (Though like moridinamael, I’m also not clear on whether he personally believed in things like genetic memory, though I would be interested to see sources if you have them. I assumed that it was an element he included for fictional/allegorical purposes.) But I think he was a clever guy who spent a lot of time thinking about problems we’re interested in, even if he often got it wrong.
I think it’s a little harsh to say that I commit the error of reading-in my beliefs about the Butlerian Jihad, given that I quote Reverend Mother Gaius Helen Mohiam as saying, “Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them,” and Leto II as saying, “The target of the Jihad was a machine-attitude as much as the machines.” I’m aware that there are a lot of textual clues that the Jihad wasn’t a war against autonomous machines themselves. Though autonomous machines were certainly involved; the glossary to the original book defines the Butlerian Jihad as, “the crusade against computers, thinking machines, and conscious robots”, and the OC Bible’s commandment is, “Thou shalt not make a machine in the likeness of a human mind.”
More generally, I was using the Jihad as a metaphor to make a point about automation in general.
It’s clear that Strong AI is illegal under the prohibition of “thinking machines”, but it had always puzzled me why lesser devices — like calculators and recording devices — were included. I had passed it off as another mistake on Herbert’s part. But when I read Nabil’s comment it reminded me strongly of the Jihad, and I realized that if taken to an extreme conclusion it would lead to a proscription against almost all automation, like the one we find in Dune. Consider it a steelman of the position, if you would like.
Just because I quote Samuel Butler at the end, doesn’t mean I think the Jihad was named after him! It’s just an amusing coincidence.
Looking forward to reading your essay on the Genetics of Dune!
Interesting! I think you may be reading Dune backwards. I always thought of it as book strictly against the concept of heroes, rather than as a power fantasy.
No more terrible disaster could befall your people than for them to fall into the hands of a Hero. (Pardot Kynes)
Make no heroes, my father said. (The voice of Ghanima)
Consider it a steelman for the pro-hero position. Paul (& others) have all the attributes you describe, and even so, even with the benefit of prescience(!) they still make their lives miserable and the lives of everyone around them. Whether you find this a convincing argument against heroes is one thing, but I think that was the tack he was taking.
Herbert’s original publisher refused to publish Dune Messiah because he found that message so personally disturbing:
Campbell turned down the sequel. Now his argument was that I had created an anti-hero in Paul in the sequel. … the thing that got to Campbell was not that I had an anti-hero in this sense, but that I had destroyed one of his gods. (FH)
You’re correct that I’m not writing about “Automation” in the usual sense, but the categories were made for man. On the other hand, I am talking about writing scripts to automatically enter data into spreadsheets. My discussion was centered around the question of if there could be a reason for the Jihad to ban something so simple as well, and I think it has. If spreadsheet-scripts depend on legible signals (and they usually will) then they are part of this problem.
I then mention more literal bureaucrats a few times, as an attempt to show that I don’t mean just those things that are machine-automated. But perhaps my examples were too convincing!
In regards to constitutions, I disagree. So does James Madison. Take a look at what he has to say about “parchment barriers”:
Will it be sufficient to mark, with precision, the boundaries of these departments, in the constitution of the government, and to trust to these parchment barriers against the encroaching spirit of power? This is the security which appears to have been principally relied on by the compilers of most of the American constitutions. But experience assures us, that the efficacy of the provision has been greatly overrated; and that some more adequate defense is indispensably necessary for the more feeble, against the more powerful, members of the government.
...
The conclusion which I am warranted in drawing from these observations is, that a mere demarcation on parchment of the constitutional limits of the several departments, is not a sufficient guard against those encroachments which lead to a tyrannical concentration of all the powers of government in the same hands. (The Federalist Papers : No. 48)
Incidentally, this is what the separation of powers is all about! By forcing actual people to compete and negotiate rather than trusting them to follow rules, we can avoid all sorts of nasty things. If you were to accuse me of cribbing from the founders, you wouldn’t be far off!
In regards to the other moral from the Butlerian Jihad, you’re totally right. That’s normally the lesson I would take away too. I just figured that this audience would already be able to see that one! I tried to present something that LW might find surprising or novel.
Thanks for your thoughts!
Wow, that’s an implication I hadn’t considered! But you’re right on the money with that one.
The one danger I see here is that very simple models can often account for ~70% of the variance in a particular area. People might be tempted to automate these decisions. But the remaining 30% is often highly complex and critical! So personally I wouldn’t automate a bureaucrat until something like 95% match or 99% match.
Though I’m sure there are bureaucrats who can be modeled 100% accurately, and they should be replaced. :P
This is pretty close to my thinking too. Herbert’s proposal was something like, “We have no idea what levels of human potential are out there.” He takes this idea and describes what it might look like, based on a few possible lines of development. Possibly he thought these were the most likely avenues of development, but that still seems unclear. Either way, he happened to pick examples that were wrong in the details, but the proposal stands.
I think it’s a reasonable inference that humanity nearly went extinct, given that the impact of the Jihad was so pronounced as to affect all human culture for the next 10,000+ years. And I think it’s a reasonable inference that we banded together, given that we did manage to win.
Well, I don’t think that foot-binding is necessarily a result of Confucianism directly, and even if it is, I see even less connection to the anti-automation aspects. You could also say that Confucianism as it was practiced bears about as much relationship to what Confucius actually taught as modern Christianity bears to what Jesus actually taught.
Potentially relevant is my post Leto among the Machines, where I discuss the early stages of what you aptly call “get what you can measure”.
You’re right to read it as an in-joke, and I’m glad that you saw it that way. I think your comment actually goes on to capture the rest of what I was trying to say with that quip. The deeper level of the joke is that this community regularly mints new jargon which can get pretty weird — if we’re not careful, someone will go off the deep end and try to call a new idea something stupid like, I dunno, “Counterspells”. (I was thinking of this comment when I wrote the joke.)
On a more textual level, I came up with the term as a personal joke, but the name stuck, and now I can’t think of them as anything else. Feel free to call this idea whatever you want.
I don’t think “Fully General Counterargument” describes what I’ve done here, though. Eliezer points out on that page that ‘you are a sophisticated arguer; you have used your intelligence to trick yourself’ is an argument that you can use “when you encounter a seemingly intelligent person who says something you don’t like”. (So technically his example is not FULLY general.) But each of the Counterspells I present here are coherent arguments only when someone has made an argument based on a specific logical fallacy. If someone makes an ad hominem against you, it doesn’t make sense to pull out a Counterspell designed for Straw Men.
I agree that rhetoric can be dangerous, but I’m actually not sure how it applies in this case. Don’t these Counterspells strictly dominate, “You used X logical fallacy against me”?
It’s true that many complaints about “logical fallacies” are mistaken, but I think that one of the nice features of Counterspells is that because their forms are so tight, they can actually help you realize when you’re misapplying them to something that is a valid inference (or something that is a different fallacy than you first thought). In the process of developing this idea, more than once I have gone to use a Counterspell against something, only to find in the process that my interlocutor was saying something very different than what I had initially imagined.
I think their formulaic, fill-in-the-blanks nature forces you to engage with the material more than you might otherwise; I think they’re the opposite of an excuse not to listen. And since they only make sense in response to certain invalid inferences, I don’t think there’s much opportunity to shoot your foot off. Many of them include a request for elaboration, and in a respectful discussion where no one engages in rhetorical tricks, they will never come up.
Agreed; like many things, these are guidelines for a conversation with someone you want to have a conversation with. You’ll notice that I didn’t include Counterspells for things like Social Shaming from Varieties of Argumentative Experience or Name-Calling from How to Disagree. Alexander discusses the whole issue at length in his essay.
Ah, understood! Yeah, the term is imperfect. Call them whatever you like.
I agree that they are most useful when actually trying to change someone’s mind.
I originally thought that they would be more effective in writing than in person, but after making the list some I’ve been surprised at how quickly and naturally they can be used in conversation. It took a few seconds the first few times, but I didn’t really need to practice. YMMV.
I actually have a pretty narrow range of uses in mind, and I think some of our disagreement comes from thinking I intend these for general use. You correctly point out that Counterspells don’t make any positive progress in the debate, they just swat things down (like a… counterspell?). And yeah, that’s all they are. They don’t do much heavy lifting and you need other tools to actually change someone’s mind (most of the time).
All I’m saying is whenever you would say, “That’s invalid because [Fallacy Name]”, instead say, “[Counterspell]”. It has to exist within the framework of better rhetorical skills, and yeah, if someone is the kind of asshole who pulls them out as underhanded tricks, these won’t save them.
All of your proposals miss my point, as does the idea of counter-counterspells. They may be epistemically virtuous, but they are what Alexander would call Meta-Debate, discussion of what can be debated, who is a trusted source, and how the discussion can be held. As Alexander points out, there’s nothing wrong with Meta-Debate, and it can be useful. It’s still not part of the actual debate.
I’ve placed it in a sphinx outside the pyramid to emphasize that it’s not a bad argument for the thing, it’s just an argument about something completely different.
Ultimately they have no bearing on whether or not the topic of discussion is true or false. Certainly I could tell someone, “Your belief in a flat earth makes me not interested in trusting your thoughts on homeopathy”, and I would be right to do so. But homeopathy is still true or false regardless of this person’s other unconventional beliefs.
Even beyond that, assuming the goal of a discussion is to change your partner’s mind (I know it’s not always, but let’s assume), then these do a terrible job of that. Counterspells are designed with the assumption that you want to convince someone discussion in good faith, and designed to engage with their (incorrect) arguments directly. What you propose are all dismissals.
Even so, I don’t think they’re great meta-arguments. Certainly you can see the problem with, “Martin Luther King was a criminal, criminals are usually bad, and I have a limited amount of time to analyze every argument that comes my way.”
You seem to be thinking of a case where Counterspells would be used against honest epistemological heuristics. I agree that this is inappropriate. But in such a case you just need to tell them that you were making a call about the discussion rather than their position (this is still meta-debate), something like: “I didn’t suggest that your tone makes you wrong, but it does make me not want to engage with you.” Though frankly in such a case I can’t see why you would want to engage further, making me even more skeptical of the idea of these double counters.
The idea of a Counterspell assumes (by definition; I invented it) 1) That the original speaker made a true logical fallacy, 2) That the responder is choosing to engage and respond in good faith, and 3) That the Counterspell response is appropriate, i.e. that it really does point out why the original argument was incorrect in that it didn’t provide good evidence for some conclusion. If they fail at that, it’s not a Counterspell. Because of this, the very idea of counter-counterspells is wrongheaded. The suggestion is to make a dismissive response to an honest, correct, and good-faith attempt to engage with someone. That’s something I’m not interested in.
Another way to put it is that Counterspells are intended to move discussions away from meta-debate. Rather than calling someone out for violating norms and trying to rack up hits against their credibility, Counterspells help you to engage with the content of someone’s complaint even as you are disagreeing with it. I think that’s incredibly helpful.
(Also I can’t see why this is at all frequentist in perspective. Bayesians can believe in true states of the world as well.)
You’re entirely right! Like you said, these are sort of concrete suggestions to be used on a case-by-case basis. I don’t think a conversational strategy should be based around them, and what you describe is much more appropriate.
Sometimes, though, you’ll be talking to someone you know and trust, and notice that they introduce an isolated demand for rigor or respond to tone, and you’ll think, “I notice that’s wrong, how do I disagree in a respectful way?” This is intended to help fill the gap in such situations. One tool in the toolbox.
Ok, actually I was just going back through one of Alexander’s posts from 2014 and found a case of him using the term “counterspell” in exactly the way I use it here:
The proper counterspell to such nonsense is Reverse Causal Arrows – could it not be that states with more marijuana users are more likely to pass proposals liberalizing marijuana laws? Yes it could.
I wasn’t aware of this when I wrote the post, but apparently there is some precedent for my usage.
You say, ”...we care a lot about which interpretation of quantum mechanics is correct because it does make predictions about the thing we care about.” I agree that if different interpretations make different predictions about things we care about (or any things), then we should care a lot about the interpretation.
However, I am writing explicitly about cases where different interpretations strictly do not make different predictions. In the section you quote, I say, “This is assuming that they [the different perspectives] truly are empirically indistinguishable.”
A reasonable objection might be that different theories can never truly be empirically indistinguishable. If so, then my arguments would apply only to cases where theories seem indistinguishable for the moment.
You also say that our purpose may be “to understand what’s going on beyond what can be predicted”; the issue is that observations are the only tool we have to understand what is going on. If two theories both fit all available and predict the same potential observations, those theories seem to be perfectly equal.
Writing by an individual who isn’t very skillful doesn’t seem likely to get this much attention or ~500 signatures, even if many of them are fake. Some sort of momentum was behind this that I think is more than the effort given by an individual — certainly given that this letter appears to be anonymous! Some authors would have this clout but seeing as how we don’t know who authored this...
That is funny! I hadn’t thought about Dune in a while, but Nabil’s comment on SSC brought thoughts of the Jihad flooding back.
I agree with your critiques of unilateral action; it’s a major problem with all proposals like this (maybe a whole post on this at some point). Something that bugs me about a lot of calls to action, even relatively mundane political ones, is that they don’t make clear what I, personally, can do to further the cause.
This is why I specifically advised that people not automate anything new. Many of us are programmers or engineers; we feel positively about automation and will often want to implement it in our lives. Some of us even occupy positions of power in various organizations, or are in a position to advise people who are. I know that this idea will make me less likely to automate things in my life; I hope it will influence others similarly.
Dismantling the automation we have sounds like a much tougher coordination problem. I’m less optimistic about that one! But maybe we can not actively make it worse.
The fealty proposal was intended as a joke! I just think we could consider being more Confucian.
Exactly what Herbert believed is hard to say, but my impression has always been that he mostly agrees with the views of his “main” characters; Leto I, Paul, Hayt, Leto II, Siona, Miles Teg, etc. Regarding Feudalism, he says that it is the “natural condition of human beings…not that it is the only condition or not that it is the right condition”. I’ve found this interview pretty enlightening.
In regards to the “multi-millennial ideologically motivated political stranglehold”, I’m not sure if he thinks it’s good. But insofar as we think human extinction is bad, we have to see this system as, if not good, then at least successful.
Thanks for the feedback! :)