Update: A failed attempt at rationality testing
This post was originally a link post to
together with an instruction to read the article before proceeding, and then the following text rot13′d:
I believe this article is a nice rationality test. Did you notice that you were reading a debate over a definition and try to figure out what the purpose of the classification was? Or did you get carried away in the condemnation of the hated telecoms? If you noticed, how long did it take you?
I’m open to feedback on whether this test was worthwhile and also on whether I could have presented it better. There’s a tradeoff here where explaining the post’s value to Less Wrong undermines that value. Had I put “Rationality Test” in the title, I could have avoided the appearance of posting an inappropriate article but made the test weaker.
As you can see from the comments here, it didn’t work very well.
I’m mostly editing this now because the apparent outrage-bait link in the discussion section was a bit of a nuisance, but I’ll take the chance to list what I’ve learned:
Not many LWers are susceptible to this genre of outrage-bait. That is, they don’t have the intended gut reaction in the first place, so this didn’t test whether they overcame it.
The only commenter who admits having had said reaction immediately and effortlessly accounted for the fact that the debate was over a definition. This suggests the test was on the easy side, even for those eligible. (Unless a bunch of people failed and didn’t comment, but I doubt that)
Most commenters did not indicate finding it obvious that this was a test. The sort of misdirection I employed is quite viable.
Feedback on the idea of the test is mixed. People don’t seem to mind the concept of being misdirected, but (if I read the top comment correctly) being put through the experience of an outrage-bait link was annoying and the test didn’t offer enough value to justify that.
I understand what you were trying to do here, but the overall experience is a bit like following an Upworthy link: being driven by artificial curiosity to expend my time on something without much payoff.
I got maybe halfway through and got bored.
I wondered if there was any actual regulatory consequence to whether service was called broadband or not. I suspect there is be, even if the article never got around to it.
This is a general peeve of mine about political articles. There’s a lot of who is presumed to be on whose side, and what people feel about what, but precious little about the relevant law.
It’s likely that the definition of the term has regulatory consequences, or at least would be used for regulatory consequences, whether or not there is actual statutory authority for it.
EDIT: I got around to reading the rest. Rather infuriating discussion that always seems on the edge of identifying an actual consequence, but never quite getting there.
To clarify, have you read the rot13?
Yes.
I really like this idea, but I can’t tell whether I failed the test, I passed the test, or the article-selection for this test was bad.
I very much felt the “condemnation of the hated telecoms” (and a bit of victory-hope). I think this means I’ve failed the test.
It took no time to realize that I was reading a debate over a definition and its purpose. I think this means I’ve passed the test.
I feel like the above realization was trivial. I didn’t consciously think “I am reading a debate of definition. ” In the same way that, when I’m playing a scary game, I don’t think “I am playing a scary game.” I thought the whole point of the article was to emphasize a debate of definition, why said debate is happening, and what side “won”. I think this means that the article was a bad one for the purposes of this test (that is: using this article feels more like a test of reading comprehension than rationality).
Which is the most appropriate result?
(To reiterate, though, I think this idea is an awesome one.)
Edit: I also don’t think the article failed to give information on what the reason behind said definition-changing was:
Edit 2: Now THIS article doesn’t emphasize the point that it’s purely a matter of definition: http://arstechnica.com/business/2015/01/tons-of-att-and-verizon-customers-may-no-longer-have-broadband-tomorrow/ The article in the OP feels like “We’ve changed the definition of broadband to increase broadband access.” The above-linked article feels like “THEY’RE TAKING AWAY OUR BROADBAND!!!” Does this seem like a reasonable differentiation, or am I being biased?
I readily thought of three possibilities for consequences of the redefinition —
The definition is just used to measure “how many households have access to broadband” for political purposes, for instance in making arguments about inequality. There is no regulatory consequence. It literally is just a rhetorical dispute over definitions, nothing more.
Given the redefinition, it becomes false advertising for an ISP to claim that they offer “broadband” when their speeds don’t meet the new definition. This imposes costs; at least, ISPs have to change their marketing materials. This effectively rewards ISPs that already meet the new definition at the expense of those who don’t.
Some sort of funding is at stake.
The article seemed to be about “should legislation requiring internet access at a certain speed to be made more available require a higher speed than it currently does?” Yes, it is true that the means of increasing the speed required would be modifying a legal definition.
However, the debate over “should law X define Y to include Z” always seems to be about “should law X apply to Z” and not “does Z really count as Y”, and people saying “Z is/isn’t Y” tend to immediately follow that with “if Z is/isn’t defined as Y, that would lead to implications.” Take, for example, the piracy debate. People counting piracy as theft say things like “if piracy isn’t classified as theft, then it won’t be prosecuted, which will lead to fewer people spending money on media”, and people who say that piracy isn’t theft tend to say things like “classifying piracy and peer-to-peer file sharing as theft leads to treating these poor teenagers who just wanted to make CDs for their friends as criminals and sending them to jail”.
It looked like a debate over whether legislation should be changed by way of changing a definition, but I wouldn’t call noticing that a special rationality skill.
I was going to downvote you for posing a link to an article with an inflammatory headline. Wouldn’t say it was a good test for me, since I was prejudiced against the article from the title alone.
I did notice that they were spending the whole time debating a definition, and that the article failed to get to any consequences.
I think that existing policies are written in terms of “broadband”, perhaps such as benefits to ISPs based on how many customers have access to broadband? That would make it a debate about conditions for subsidies, minimum service requirements, and the wording of advertising.
Yeah, my conclusion is that there isn’t really enough information in the article to be sure, and I haven’t done the external research. But asking the right question is the important part.
Your rationality test is pointless, because people who are following the issue will already know that the definition is not ‘mere’; it has significant political consequences. (So people not following the issue, who successfully deduced that the argument was pointless because it was over a mere definition, would actually be wrong.)
I think these kinds of rationality tests tend to appear disrespectful of the audience, since they present the poster’s rationality as being in excess of the audience’s; and as long as the ‘correct solution’ is arguable, which it very often is, it becomes not-very-defensible to hide the purpose of the post from the readers in order to try to force them to find the ‘correct solution’ on their own.
I think pretty much only Eliezer can get away with this sort of material, since people seem to feel he’s smart enough to outthink us all.
Concluding that the definition is pointless is not the passing condition. Backing up to ask what the point of the definition is is the passing condition.
The rest of what you say is good feedback.
There no good reason to assume that all relevant information to answer that question is contained in the article.
I only skimmed the article because it has very little relevance to me. The only reason I skimmed it was because I didn’t want to ruin whatever you were trying to do. Which leads me to the next point: It was pretty clearly a test, rather than an actual article you just wanted to share. Even if the article was relevant to my interests (in which case I’d actually read it in full), I would have done so with a lot of extra caution because you clearly had an ulterior motive for posting this.
Is there a reason this is rationality related? I know a lot of us care about tech issues, but this doesn’t seem terribly on topic.
Did you read the rot13′d text? EDIT: But read the article first if you want to actually get value from this post.
No. I had not. I see why you ROT-13rd it, and that does make it seem like rationality related material. At a glance, this looked more like a general link with ROT-13 to avoid influencing us with your opinion prematurely. I’m not sure if this is a good or a bad thing, but I do think it will likely reduce how many people bother to read it.
Right, that’s the tradeoff. I considered putting “I promise this is relevant” somewhere, but that still might put people on their guard. Since I’m worried the test is too easy as it is, having a lower number of readers take a stronger test seemed like the right call.
I noticed immediately. Still, definitions do things. It doesn’t seem fair for ISPs to have to stop calling internet broadband even though it’s just as good as it was last year. I feel like what they should have done is made up a new word for even faster internet. That way, the ISPs can keep calling their old speed broadband, they have a nice way of advertizing faster speeds for people who don’t know about what the numbers mean, and if the politicians want to talk about faster internet, they have a word for it. I suspect there’s a downside I’m missing, since they didn’t do that.
The entire thing wasn’t about definitions though. They were also talking about a sort of standard speed that they want to bring everyone up to. My opinion on this is that it’s more expensive to bring fast internet to people that don’t live in cities. You can’t just call all of the benefits of living in cities rights and then expect to be able to live in the middle of nowhere and get the best of both worlds. If you really want fast internet without living in a city, pay for it yourself.
I guess I had an unfair advantage. I don’t hate the ISPs. I can’t get mindkilled if I don’t care.
I had the same unfair advantage, but I think to a lesser degree. I noticed quickly but not immediately, which is to say not quickly enough.
On the object level, I think there are possible legislative purposes that would justify the change (as instrumental to those purposes), but my strong suspicion is that the FCC is wrong. I fear that ‘broadband’ effectively means ‘above baseline speed’, which actually would make it reasonable to tell the ISPs to change what speed they advertise as ‘broadband’, but would turn the mandate to give everyone broadband into a Lake Wobegon/Kafka crossover.
If by broadband they mean “above baseline speed”, then clearly giving everyone broadband is impossible.