It’s slightly disconcerting to imagine some of the writing coming from the pen of an Anglican deacon.
MatthewW
Karma: 149
The useful advice is in the first 5000 words of the essay, most importantly in the examples of bad writing. The 100 words or so of ‘rules’ are just a summary at the end.
This kind of teaching is common in other subjects. For example, in a Go textbook it’s not rare to see a chapter containing a number of examples and a purported ‘rule’ to cover them, where the rule as stated is broken all the time in professional play. It would be a mistake to conclude that the author isn’t a strong player, or that the chapter doesn’t contain helpful advice. The ‘rule’ is just a way to describe a group of related examples.
I think it’s better to think of the ‘rules’ in Orwell’s essay more like mnemonics for what he’s said earlier, rather than instructions to be followed on their own.
I don’t find it off-putting, but it does make me feel I’m reading Lewis Carrol.
Priors don’t come into it. The expert was presenting likelihood ratios directly (though in an obscure form of words).
That isn’t what was going on in this case. The expert wasn’t presenting statistics to the jury (apparently that’s already forbidden).
The good news from this case (well, it’s news to me) is that the UK forensic science service both understands the statistics and has sensible written procedures for using them, which some of the examiners follow. But they then have to turn the likelihood ratio into a rather unhelpful form of words like ‘moderately strong scientific support’ (not to be confused with ‘moderate scientific support’, which is weaker), because bringing the likelihood ratios into court is forbidden.
(Bayes’ Theorem itself doesn’t really come into this case.)
This isn’t quite “a judge has ruled that [Bayes’ theorem] can no longer be used”, but I don’t think it’s good.
The judges decided that using a formula to calculate likelihood isn’t allowed in cases where the numbers plugged into the formula are themselves uncertain (paragraph 86), and using conservative figures apparently doesn’t help.
Paragraph 90 says that it’s already established law that Bayes’ theorem and likelihood ratios “should not be used”, but I think it means “shouldn’t be talked about in front of the jury”.
Paragraph 91 says explicitly that the court wasn’t deciding how (or whether) Bayes’ Theorem and likelihood ratios can be used in cases where the numbers plugged into the formula aren’t themselves very uncertain.
In paragraph 95, the judges decide that (when matching footprints) it’s OK for an expert to stare at the data, come up with a feeling about the strength of the evidence, and express that in words, while it’s not OK for the same expert to do a pencil-and-paper calculation and present the result in similar words.
I think part of the point is that when the expert is cross-examined, the jury will react differently if she says “this evidence is strong because I’ve got lots of experience and it feels strong to me”, rather than “this evidence is strong because I looked up all the frequencies and did the appropriate calculation”.
I do get the impression that the approach of multiplying likelihood ratios is being treated as a controversial scientific process (as if it were, say, a chemical process that purported to detect blood), and one which is already frowned upon. Eg paras 46, 108 iii).
Thanks for the link.
I think paragraphs 80 to 86 are the key paragraphs.
They’re declaring that using a formula isn’t allowed in cases where the numbers plugged into the formula are themselves uncertain.
But in this case, where there was uncertainty in the underlying data the expert tried to take a conservative figure. The judges don’t seem to think that helps, but they don’t say why. In particular, para 108 iv) seems rather wrongheaded for this reason.
(It looks like one of the main reasons they overturned the original judgement was that the arguments in court ended up leaving the jury hearing less conservative estimates of the underlying figures than the ones the expert used (paras 103 and 108). That seems like a poor advertisement for the practice of keeping explicit calculations away from the jury.)
Well, I’m in the UK, and there’s no law against using IQ-style tests for job applicants here. Is that really the case in the US? (I assume the “You’re a terrorist” bit was hyperbole.)
Employers here still often ask for apparently-irrelevant degrees. But admission to university here isn’t noticeably based on ‘generic’ tests like the SAT; it’s mostly done on the grades from subject-specific exams. So I doubt employers are treating the degrees as a proxy for SAT-style testing.
In your third speculation, I think the first and second category have got swapped round.
I don’t think there’s much need for heuristics like “rate of effectiveness change times donation must be much smaller—say, a few percent of—effectiveness itself.”
If you’re really using a Landsburg-style calculation to decide where to donate to, you’ve already estimated the effectiveness of the second-most effective charity, so you can just say that effectiveness drop must be no greater than the corresponding difference.
If it’s a belief you’ve previously thought of as obvious and left unexamined, then this is probably a useful heuristic. Otherwise, no.
You say ‘the world’, but it seems to me you’re talking about a region which is a little smaller.
Yes, several of these models look like they’re likely to run into trouble of the Goodhart’s law type (“Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes”).
Principal component analysis of UK political views, from a few years back: http://politicalsurvey2005.com/themap.pdf
I think trolley problems suffer from a different type of oversimplification.
Suppose in your system of ethics the correct action in this sort of situation depends on why the various different people got tied to the various bits of track, or on why ‘you’ ended up being in the situation where you get to control the direction of the trolley.
In that case, the trolley problem has abstracted away the information you need (and would normally have in the real world) to choose the right action.
(Or if you have a formulation which explicitly mentions the ‘mad philosopher’ and you take that bit seriously, then the question becomes an odd corner case rather than a simplifying thought experiment.)
I wonder whether the ‘right answers’ are what the subject of the photograph was actually feeling, what an expert intended the photograph to represent, or what most people respond.
I think it’s quite normal that if someone is acknowledged by their peers to be among the very best at what they do, they won’t waste much time with status games.
There’s an exception if doing what they do requires publicity to bring in sales or votes.
It would become a mind game: you’d have to explicitly model how you think Omega is making the decision.
The problem you’re facing is to maximise P(Omega rewards you|all your behaviour that Omega can observe). In the classical problem you can substitute the actual choice of one-boxing or two-boxing for the ‘all your behaviour’ part, because Omega is always right. But in the ‘imperfect Omega’ case you can’t.
To get to the conclusion that against a 60% Omega you’re better off to one-box, I think you have to put in a strong independence assumption: that the probability of Omega getting it wrong is independent of the ways of thinking that the player is using to make her choice.
I think that’s really the original problem in disguise (it’s a 100% Omega who rolls dice and sometimes decides to reward two-boxing instead of one-boxing). The analysis if all you know is that Omega is right 60% of the time would look different.
More generally, the words for the non-metric units are often much more convenient than the words for the metric ones. I think this effect is much stronger than any difference in convenience of the actual sizes of the units.
I think it’s the main reason why many of the the non-metric units are still more popular for everyday use than the metric ones in the UK, even though we’ve all learned metric at school for the last forty years or so.