A few random thoughts on statistics and terrorism

In the world of statistics and operations, people usually talk of two kinds of error – omission and commission. For simplicity, they are referred to as “type 1” and “type 2” errors. I can never remember which is which, but after a little bit of googling, I can tell you that type 1 error is the error where a correct hypothesis is rejected, while type 2 is one where we fail to reject an incorrect hypothesis.

The most common example for this is one of a quality control department. Suppose you are in the business of checking the quality of widgets. There are two kinds of errors you can make – you can classify bad widgets as “good”, or you can classify a good widget as “bad”. Which of this is type 1 and type 2 depends upon how you frame the hypothesis. However, let’s not get into those details – they don’t matter. All that matters is that you understand the two ways you can err – which is not hard to understand at all.

Elementary statistics says that one can’t simultaneously minimize both type 1 and type 2 errors in the same process. This again – I think – is fairly intuitive. If you keep your criteria for success too high, you may hardly classify bad widgets as good. However, the chance that you classify a good widget as bad increases. Similarly, if you loosen your criteria, you will end up rejecting less good widgets, but will pay for it by accepting more bad widgets. Simple, right?

Ok I suppose you you might have figured out where I’m trying to lead you with regards to the fight on terrorism. The reason different groups have vastly different views on how terrorism should be countered lies in what they are trying to minimize. Let’s rephrase the widget problem. Everyone is either a terrorist (T) or a non-terrorist (NT). Now, the job of the police is to identify and put in jail all of T, while ensuring that no NT is put behind bars.

The problem arises because there is no clear test which can classify people into T and NT. There are a few tests that people apply, but they can only give some idea as to whether the testee is a T or a NT. The real battle between different groups lies in drawing the line for this test – which can bring some kind of objectivity into the classification into T and NT.

So on one hand, you have the human rights people whose main objective is that no NT should be classified as T. To achieve this end, they advocate a “decision line” where the chances of a NT being classified as T are minimized. In fact, going by Salil Tripathi’s view that human rights people need to be unreasonable in their demands, if the human rights guys have their way, the line should be set such that no one is classified as T.

On the other hand, you have the pro-security forces, whose sole objective is to reduce the chances of a terrorist attack. Which means that they want a line where no T is classified as NT. Hence, they will advocate a line based on which the chances of a T getting classified as NT is minimal. The side effect of this is that a number of NTs end up getting classified as T.

Now that we have figured out the conflict between human rights and pro-security people, there is another angle to this story – one that is far removed from statistics. The issue is that people who have been classified as T are more likely to have a face than those classified as NT. It’s common to read reports such as “Binayak Sen arrested on grounds of terrorism”, but one never gets to read reports that say “Salil Tripathi not arrested because of lack of evidence that he’s a terrorist”. I’m not able to exactly point out what kind of bias this is, but it is important to note that a NT being classified as T is not the same as T being classified as NT. This asymmetry in footage gives further leeway for human rights people to gain a better position of strength.

Then there is the issue that most Ts are muslim. This has automatically communalised the whole issue. If you are seen as a hardline pro-security guy, you automatically get labeled as “anti-Muslim” and communal. Actuallly, a simple application of Bayes’s Theorem shows that in India, the probability that a random Muslim is a terrorist is significantly higher than the probability that a random non-Muslim is a terrorist. What we need to note here is that though the former probability is quite low, it is still an order of magnitude higher than the latter.

The challenge for the pro-security people is to effectively challenge the unreasonable demands of the human rights people, while at the same time try not to appear to be anti-Muslim. What we need to accept is that we can never have a perfect T/NT filter. And that the “confidence line” that we draw to classify people as T and NT is socially optimal. We will need to balance, on one hand, the loss of lives and property in case of a terrorist attack, and on the other hand, the inconvenience that the average citizen will face in case the line is drawn too “tight”. We will need to evaluate costs of each, estimate probabilities and then draw the line. Even in one such scenario, we need to remember that there can never be a perfect filter. Recognizing this deficiency, I think, is also a major part of the solution to this problem.

I don’t know who made this statement – it was one of Madness, Disease and Ugliness. One of them said “most Muslims are not terrorist but most terrorists are Muslim”.

19 thoughts on “A few random thoughts on statistics and terrorism”

  1. I have long held that instead of saying Type 1 error and Type 2 error, we should say Type A Error and Type B error. To minimise confusion, of course.

    But yeah, this is a great point. Only I think it also applies to law enforcement in general, and not just terrorism in particular. On the one hand you have guys like American law-and-order conservatives, whose (stated) aim is to make sure no criminal goes unpunished. These are the people who pilloried Michael Dukakis for pardoning that guy who went on to rape and murder. On the other hand you have people like the characters on The Practice, whose (stated) motto is “Better to let 10 guilty men go free than let one innocent man go to jail”.

    The problem is that nobody sees this as a Type 1/Type 2 problem. Type 1 people therefore regularly accuse Type 2 people of loving terrorists/other criminals, while Type 2 people accuse Type 1 people of being motivated by hatred of blacks/Muslims/whatever. And so it continues.

    1. how does calling it type A and type B help? because of the association with alpha and beta?

      and i agree with you that most people don’t look at this as a type1/type2 problem. I think apart from irony deficiency that one of the pandey kids will soon write about, there is also a profound deficiency in statistical methods. and this doesn’t apply only to india.

  2. Precisely. And the T/NT filter in is obviously going to be much worse in India than in US because the police force is ill paid and corrupt. And the liberals naively say things like “Make the police force more efficient”. And laws like POTA do have a large conviction rate.

    Secondly, there is this issue that terrorism is much more serious than law and order problems – because law and order problems involve a few individuals trying to accomplish personal gains while terrorism has its origin in/affilitation with an entire network designed to destroy the country – it is a very large scale enterprise.

    1. Yeah but making the police force more efficient will have a beneficial impact even if it does not have a completely beneficial impact. And in terms of things like making the police force less susceptible to political control, it would actually be better in places like Maha/ AP where supposedly the CMs have told the police to go slow.

    1. no
      i wasn’t there at the time it was said. i joined you guys only much later that night, after you’d seen pyaar ke side effects and talked about sunk costs.

      but yeah – it’s very likely that disease would’ve said that.

  3. My gripe with the situation is that the “confidence line” varies from fairly accommodating in times of peace to extremely restrictive immediately after a terrorist attack (when the security people are all worked up), which is hardly socially optimal.

    Immediately after an attack, in addition to fearing for our lives, we need to trudge through rampant security checks. The probability of catching terrorists (with fixed effort) is lower immediately after an attack than it is in times of peace, as they tend to lie low. As I see it, the confidence line ought to remain lax immediately after a series of attack (at least where the attacks occurred) and tighten up when all is well, a counter-intuitive but hopefully more workable notion. This causes the average NT lesser worry in the long run.

    The book “Little Brother” by Cory Doctorow is a fantastic (and scary) illustration of paranoia run rampant after a terrorist attack. Very engaging read.

    1. i agree with you that the temporal movement of the confidence line is flawed. and i think it’s too painful to figure out how to move the line depending upon when the attacks have happened. i think it should just remain in one place

  4. It was Thomas Friedman who first made that statement about Muslims and terrorists in an NYT article sometime in 2004-2005.

  5. Oh, for sure. What I meant was that it isn’t realistic to expect India to have a non-corrupt and efficient police force like in the West. Hence the necessity for more stringent anti-terror laws. i.e., the “existing laws are sufficient if the law and order equipment becomes efficient” argument doesn’t hold much ground.

  6. >>how does calling it type A and type B help?

    This is a joke. I realise I should have started paragraph 2 with “but seriously” instead of “but yeah” to make this more obvious. Apologies. I am curious about one thing though. How are alpha and beta relevant here?

  7. Unquestionably imagine that that you stated. Your favorite reason appeared to be on the net the easiest thing to have in mind of. I say to you, I certainly get irked whilst other people consider issues that they plainly do not realize about. You managed to hit the nail upon the top and also defined out the whole thing with no need side effect , people can take a signal. Will likely be back to get more. Thank you

Leave a Reply to KunalCancel reply