Tag Archives: probability

Religion and Probability

If only people were better at mathematics in general and probability in particular, we may not have had religion

Last month I was showing my mother-in-law the video of the meteor that fell in Russia causing much havoc, and soon the conversation drifted to why the meteor fell where it did. “It is simple mathematics that the meteor fell in Russia”, I declared, trying to show off my knowledge of geography and probability, arguing that Russia’s large landmass made it the most probable country for the meteor to fall in. My mother-in-law, however, wasn’t convinced. “It’s all god’s choice”, she said.

Recently I realized the fallacy in my argument. While it was probabilistically most likely that the meteor would fall in Russia than in any other country, there was no good scientific reason to explain why it fell at the exact place it did. It could have just as likely fallen in any other place. It was just a matter of chance that it fell where it did.

Falling meteors are not the only events in life that happen with a certain degree of randomness. There are way too many things that are beyond our control which happen when they happen and the way they happen for no good reason. And the kicker is that it all just doesn’t average out. Think about the meteor itself for example. A meteor falling is such a rare event that it is unlikely to happen (at least with this kind of impact) again in most people’s lifetimes. This can be quite confounding for most people.

Every time I’ve studied probability (be it in school or engineering college or business school), I’ve noticed that most people have much trouble understanding it. I might be generalizing based on my cohort but I don’t think it would be too much of a stretch to say that probability is not the easiest of subjects to grasp for most people. Which is a real tragedy given the amount of randomness that is a fixture in everyone’s lives.

Because of the randomness inherent in everyone’s lives, and because most of these random events don’t really average out in people’s lifetimes, people find the need to call upon an external entity to explain these events. And once the existence of one such entity is established, it is only natural to attribute every random event to the actions of this entity.

And then there is the oldest mistake in statistics – assuming that if two events happen simultaneously or one after another, one of the events is the cause for the other. (I’m writing this post while watching football) Back in 2008-09, the last time Liverpool FC presented a good challenge for the English Premier League, I noticed a pattern over a month where Liverpool won all the games that I happened to watch live (on TV) and either drew or lost the others. Being rather superstitious, I immediately came to the conclusion that my watching a game actually led to a Liverpool victory. And every time that didn’t happen (that 2-2 draw at Hull comes to mind) I would try to rationalize that by attributing it to a factor I had hitherto left out of “my model” (like I was seated on the wrong chair or that my phone was ringing when a goal went in or something).

So you have a number of events which happen the way they happen randomly, and for no particular reason. Then, you have pairs of events that for random reasons happen in conjunction with one another, and the human mind that doesn’t like un-explainable events quickly draws a conclusion that one led to the other. And then when the pattern breaks, the model gets extended in random directions.

Randomness leads you to believe in an external entity who is possibly choreographing the world. When enough of you believe in one such entity, you come up with a name for the entity, for example “God”. Then people come up with their own ways of appeasing this “God”, in the hope that it will lead to “God” choreographing events in their favour. Certain ways of appeasement happen simultaneously with events favourable to the people who appeased. These ways of appeasement are then recognized as legitimate methods to appease “God”. And everyone starts following them.

Of course, the experiment is not repeatable – for the results were purely random. So people carry out activities to appease “God” and yet experience events that are unfavourable to them. This is where model extension kicks in. Over time, certain ways of model extension have proved to be more convincing than others, the most common one (at least in India) being ‘”God” is doing this to me because he/she wants to test me”. Sometimes these model extensions also fail to convince. However, the person has so much faith in the model (it has after all been handed over to him/her by his/her ancestors, and a wrong model could definitely not have propagated?) that he/she is not willing to question the model, and tries instead to further extend it in another random direction.

In different parts of the world, different methods of appeasement to “God” happened in conjunction with events favourable to the appeasers, and so this led to different religions. Some people whose appeasements were correlated with favourable events had greater political power (or negotiation skills) than others, so the methods of appeasement favoured by the former grew dominant in that particular society. Over time, mostly due to political and military superiority, some of these methods of appeasement grew disproportionately, and others lost their way. And we had what are now known as “major religions”. I don’t need to continue this story.

So going back, it all once again boils down to the median man’s poor understanding of concepts of probability and randomness, and the desire to explain all possible events. Had human understanding of probability and randomness been superior, it is possible that religion didn’t exist at all!

The day I learnt to stop worrying and learnt to protect myself

For at least six years, from early 2006 to early 2012 I “suffered” from what medical practitioners term as “anxiety”. It was “co-morbid” with my depression, and I think it was there from much before 2006. I would frequently think about random events, and and wonder what would happen if things happened in a certain way. I would think of “negative black swan” events, events with low probability but which would have a significant negative impact on my life.

While considering various possibilities and preparing for them is a good thing, the way I handled them were anything but good. Somewhere in my system was wired the thought that simply worrying about an event would prevent it from happening. I once got fired from one job. Every day during my next two jobs, I would worry if I would get fired. If I got an uncharitable email from my boss, I would worry if he would fire me. If my blackberry failed to sync one morning I would worry that it was because I had already been fired. Needless to say, I got fired from both these jobs also, for varying reasons.

I used to be a risk-taker. And it so happened that for a prolonged period in my life, a lot of risks paid off. And then for another rather prolonged period, none of them did (Mandelbrot beautifully calls this phenomenon the Joseph effect). The initial period of successful risk-taking probably led me to take more risk than was prudent. The latter period of failure led me to cut down on risks to an unsustainable level. I would be paranoid about any risks I had left myself exposed to. This however doesn’t mean that the risks didn’t materialize.

It was in January of last year that I started medication for my anxiety and depression. For a few days there was no effect. Then, suddenly I seemed to hit a point of inflexion and my anxious days were far behind. While I do credit Venlafaxine Hexachloride I think one event in this period did more than anything else to get me out of my anxiety.

I was riding my Royal Enfield Classic 500 across the country roads of Rajasthan, as part of the Royal Enfield Tour of Rajasthan. The first five days of the tour had gone rather well. Riding across the rather well-made Pradhan Mantri Gram Sadak Yojana (PMGSY) roads set across beautiful landscapes had already helped clear out my mind a fair bit. It gave me the time and space to think without getting distracted. I would make up stories as I rode, and at the end of each day I would write a 500 word essay in my diary. All the riding gear meant that the wind never really got into my hair or my face, but the experience was stunning nevertheless. For a long time in life, I wanted to “be accelerated”. Ride at well-at-a-faster-rate, pulling no stops. And so I rode. On the way to Jaisalmer on a rather empty highway, I even hit 120 kmph, which I had never imagined I would hit on my bike. And I rode fearlessly, the acceleration meaning that my mind didn’t have much space for negative thoughts. Things were already so much better. Until I hit a cow.

Sometimes I rationalize saying I hadn’t consumed my daily quota of Venlafaxine Hexachloride that morning. Sometimes I rationalize that I was doing three things at the same time – one more than the number of activities I can normally successfully carry out simultaneously. There are times when I replay the scene in my head and wonder how things would have been had I done things differently. And I sometimes wonder why the first time I ever suffered a fracture had to happen in the middle of nowhere far off from home.

It had been a wonderful morning. We had left the camp at Sam early, stopping for fuel at Jaisalmer, and then at this wonderful dhaba at Devikot, where we had the most awesome samosa-bajjis (massive chilis were first coated with a layer of potato curry – the one they put in samosa – and then in batter and deep fried). For the first time that day I had the camera out of its bag, hanging around my neck. I would frequently stop to take photos, of black camels and fields and flowers and patterns in the cloud. The last photo I took was of Manjunath (from my tour group) riding past a herd of black camels.

I function best when I do two things at a time. That morning I got over confident and did three. I was riding on a road 10 feet wide at 80 kilometres per hour. I was singing – though I’ve forgotten what I was singing. And I was thinking about something. My processor went nuts. While things were steady state on the road there was no problem. There was a problem, however, when I saw a bit too late that there was a massive herd of massive cows blocking my path further down the road.

There was no time to brake. I instead decided to overtake the herd by moving to the right extreme of the road (the cows were all walking on the road in the same direction as me). To my misfortune, one of the cows decided to move right at the same time, and I hit her flush in the backside. The next thing I remember is of me lying sprawled on the side of the road about five metres from where my bike was fallen. There was no sign of the cow. The bike was oozing petrol but I wasn’t able to get up to lift it up – presently others in my tour group who were a few hundred metres behind reached the scene and picked up my bike. And I don’t know what state of mind I was in but my first thought after I picked myself up was to check on my camera!

The camera wasn’t alright – it required significant repairs after I got back home, but I was! I had broken my fifth metacarpal, which I later realized was a consequence of the impact of the bike hitting the cow. There were some gashes on my bicep where the protective padding of my riding jacket had pressed against my skin. I still have a problem with a ligament in my left thumb, again a consequence of the impact. And that was it.

I had had an accident while traveling at 80 kmph. I had fallen a few metres away from the point of impact (I don’t know if I did a somersault while I fell, though). I fell flush on my shoulder with my head hitting the ground shortly. It was a rather hard fall on the side of the road where the ground was uneven. And there was absolutely no injury because of the fall (all the injury was due to impact)!

It was the protection. No amount of worry would have prevented that accident. Perhaps I was a bit more careless than I should have been but that is no reason for there not being an accident. When you are riding on a two wheeler at a reasonable pace on country roads, irrespective of how careful you are there is always a chance that you may fall. The probability of a fall can never go to zero.

What I had done instead was to protect myself from the consequences of the fall. Each and every piece of protective equipment I wore that day took some impact – helmet, riding jacket, riding gloves, knee guard, shoes.. Without any one of these pieces, there is a chance I might have ended up with serious injury. There was a cost I paid – both monetary and by means of discomfort caused by wearing such heavy gear – but it had paid off.

Black swans exist. However, worrying about them will not ease them. Those events cannot be prevented. What you need to do, however, is to hedge against the consequences of those events. There was always a finite possibility that I would fall. All I did was to protect myself against the consequences of that!

Despite contrary advice from the doctor, I decided to ride on and finish the tour, struggling to wear my riding glove over my swollen right hand – stopping midway would have had a significant adverse impact on my mental state which had just begun to improve. I’ve stopped worrying after that. Yes, there are times when I see a chance of some negative black swan event happening. I don’t worry about that any more, though. I only think of how I can hedge against its consequences.

Pot and cocaine

Methylphenidate, the drug I take to contain my ADHD, is supposed to be similar to cocaine. Overdosing on Methylphenidate, I’m told, produces the same effects on the mind that snorting cocaine would, because of which it is a tightly controlled drug. It is available only in two pharmacies in Bangalore, and they stamp your prescription with a “drugs issued” stamp before giving you the drugs.

Extrapolating, and referring to the model in my post on pot and ADHD, snorting cocaine increases the probability that two consecutive thoughts are connected, and that there is more coherence in your thought. However, going back to the same post, which was written in a pot-induced state of mind, pot actually pushes you in the other direction, and makes your thoughts less connected.

So essentially, pot and cocaine are extremely dissimilar drugs in the sense that they act in opposite directions! One increases the connectedness in your train of thought, while the other decreases it!

I’ve never imbibed cocaine, so this is not first-hand info, but I’ve noticed that alcohol when taken in heavy doses (which I never reach since I’m the designated driver most of the time) acts in the same direction of cocaine/methylphenidate – it increases the coherence in your thoughts. Now you know why junkies in your college would claim that the kind of “high” that pot gives is very different from the kind of high that alcohol gives.

US MBA Admissions

B-schools based in the US use a unique self-selecting mechanism to filter out applicants who might be a bad fit for a management job. This they achieve by making the application process more complicated, but in a way that the kind of people they hope to attract find it simple.

Let me explain. Like most other graduate programs in the US, B-schools also require applicants to get a set of letters of recommendation. Unlike other programs, though, these are not simple letters of recommendation. Rather than the recommender simply writing out one essay where he/she extols the virtue of the candidate he/she is recommending and requests the university to grant admission, here he/sh has to answer a bunch of questions that the university is asking for. These questions might range from the mundane sounding (I’m told there’s a catch, though) “How do you know the applicant?” to some high-sounding stuff like “What is your opinion of the leadership qualities of the applicant? How can that be improved?”. World limit for all questions put together comes to 1500 words.

So now, if someone comes to you asking for a recommendation, unless you are really invested in their careers you will not want to put the enthu of putting so much effort. If you like the candidate, you might be willing to put in some time into it, but you are likely to wholeheartedly produce four good essays for each school the applicant is applying (note that no two schools ask the same question) only if you feel really invested in the applicant’s career, the probability of which is really low.

By having such a complicated system of soliciting recommendations, the schools ensure that all candidates fall into one of two categories. Either they should have done so well in one of their jobs that their boss or client feels invested enough to spend a few hours of their time writing recommendations, or they should have the necessary people management skills to go to bosses and clients and professors to get them to write the recommendations. Of course, irrespective of how good your people management skills are , it is unlikely to get someone to spend so many hours on your recommendation letters. Still, the minimum you require is to convince them that you will write the recommendation yourself and they should rubber stamp it. No big deal, that.

This way, all applicants to US B-schools are people who have a knack of getting things done. The age at which application happens (mid to late twenties) also minimizes parental participation in the effort. Apart from the self-selection and filtration, the amount of time and effort required for application also helps weed out frivolous candidates (remember those that “wrote CAT just as a backup”?).

Slow deaths and sudden deaths

My parents both died slow deaths. My father spent the last three months of his life in hospital, of which the last month was in intensive care on ventilator support. He had been rendered immobile, and when the ventilator tube and food pipe went in, there was absolutely no way in which he could communicate to us during the brief times we were allowed to meet him.

My mother’s was a different story, but on a shorter time scale. She spent her last month in hospital, with the last ten days in intensive care and on ventilator, again what I think was fairly painful existence for her, living in a fairly isolated and airconditioned room, not being able to communicate with anyone, with all sorts of tubes and measuring devices stuck all over the body.

In hindsight, I regret my decision to allow them to be put on ventilator. I feel guilty for having extended their lives in a way which was both painful to them and where there was little meaning, for they lived cut off, and unable to communicate (and in both cases, had I thought rationally, I would’ve known that there was little chance the time on ventilator would allow them to recover). The only upside to this was that it gave me time to prepare. That it gave me time to prepare for their impending passing,

People who attended either of my parents’ funerals might have been surprised, a bit shocked even, to see that I was quite composed and in control of things. I wouldn’t be wrong if a number of them thought I was a heartless emotionless wretch. The reason I behaved thus was because it was only an incremental change as far as my mental preparedness was concerned. Till the day prior to both my parents’ deaths, I knew that the chances that they would recover and get back home was minimal. Delta. Epsilon. The death, normally a “discrete event” had only pushed this chance to zero, not a big change in probability.

I was thinking about all this two nights back when my grandfather-in-law passed away, once again after a prolonged illness (he refused to be admitted to hospital or be put on life support so in a way he was spared of time on ventilator), but his condition had deteriorated steadily enough for us to know that he would be gone soon. Several family members reacted quite badly, but several others were quite brave and acted bravely. The slow death was the reason for this, I thought.

There are too many factors that affect death, and no one can choose either the time or mode or pace of dying, but I have been thinking if slow deaths are better than sudden deaths or vice versa. The upside of a sudden death is that there is little suffering on the part of the dyer, but the discrete nature of the change (probability that the person would be no more the next day would jump suddenly from close to zero to one) would imply a huge shock for family members and friends, which they would take considerable time and effort to come out of.

A slow death, on the other hand, is extremely painful for the dyer, while it gives time to the family members to come to terms with the reality. Here, too, of course there is usually one big discrete step involved (like that Monday night when in the matter of less than an hour, my mother went from happily chatting with me to gasping for breath so uncontrollably that they had to immediately wheel her to intensive care and a ventilator; or that Thursday morning when my father suddenly realized he had lost all the power in his legs and couldn’t stand on his own), so it is more like a time-shifting of pain (for relatives/friends) rather than the pain being amortized over a number of days.

Once again, there are no clear answers to this question about which mode of death is better, but ever since I saw my father spend his last three months in hospital I’ve believed that sudden deaths are superior. I’ve found myself reacting to other people’s sudden deaths saying “good for them they went without suffering”. Again, no one really has control about how or when they’ll die. It’s only a question about what to hope for in life.

Card Games

So the other day, while playing rummy with the members of the in-law family, I figured why I suck so much at some card games despite having played them quite regularly when I was a kid. Back then, in family gatherings, it was common for the host to come up with a couple of packs of cards, and we would play either rummy or this game called donkey (some kind of variation of hearts is how I’ll describe it for those that don’t know it). Given how regularly we played it, I should have become rather good at either of them, which unfortunately is not the case.

Bridge was the first card game that I learnt “formally”, in the hostel blocks of IIT Madras. Soon after being explained the rules of the game, I was taught conventions, both in bidding and play. I was taught the math, the probabilities of various distributions and to make intelligent guesses. While I quickly became decently good at bridge, it didn’t help my game in any of the other card games that I’d learnt.

So while playing recently, I realized that I know little about the science of rummy. And then I realized the reason for it – we used to play with incomplete decks. The problem with old family-held packs of cards with which no “formal” games are played is that cards tend to go missing over the course of time (especially if there are kids around), and no one really bothers to check. And when you play with incomplete packs of cards, all the beautiful math and rules of probability go out of the window. And if you have learnt playing with such a pack of cards, it is unlikely you’d have figured out much math also.

Last night, while playing rummy with the wife, I tried my best to use math, to keep a careful note of discarded cards, the joker (for example, if seven of hearts had turned up as the joker card, that meant a six of hearts in hand was of less use than otherwise (we were playing with only one pack) ), mathematical probabilities of which cards are still available based on discards and stuff. Then, it turned out that there was too much luck involved in the distribution of cards, and I started missing the duplicate bridge games that we used to play back in IIT.

The wife has shown an inclination to learn bridge, and I’m trying to teach her. We’re also trying to learn poker (we’d bought this nice poker set in Sri Lanka last year but it remains unused since neither of us can play the game). Yeah, becoming really good at these card games is one of the aims of my “project thirty”.

Jobs and courtship

Jobs, unlike romantic relationships, don’t come with a courtship period. You basically go for a bunch of interviews and at the end of it both parties (you and the employer) have to decide whether it is going to be a good fit. Neither party has complete information – you don’t know what a typical day at the job is like, and your employer doesn’t know much about your working style. And so both of you are taking a risk. And there is a significant probability that you are actually a misfit and the “relationship” can go bad.

For the company it doesn’t matter so much if the odd job goes bad. They’ll usually have their recruitment algorithm such that the probability of a misfit employee is so low it won’t affect their attrition numbers. From the point of view of the employees, though, it can get tough. Every misfit you go through has to be explained at the next interview. You have a lot of misfits, and you’re deemed to be an unfaithful guy (like being called a “much-married man”). And makes it so tough for you to get another job that you are more likely to stumble into one where you’re a misfit once again!

Unfortunately, it is not practical for companies to hire interns. I mean, it is a successful recruitment strategy at the college-students level but not too many people are willing to get into the uncertainty of a non-going-concern job in the middle of their careers. This risk-aversion means that a lot of people have no option but to soldier on despite being gross misfits.

And then there are those that keep “divorcing” in an attempt to fit in, until they are deemed unemployable.

PS: In this regard, recruitments are like arranged marriage. You make a decision based on a handful of interviews in simulated conditions without actually getting to know each other. And speaking of arranged marriage, I reprise this post of mine from six years ago.

Managing stud work

I begin this post with an apology. About two years back I’d promised that I won’t write any more on Studs and Fighters on this blog, and I’ll save all that for my forthcoming book. Unfortunately, since then I’ve managed not more than one page of my book, and that too has been in the last couple of weeks. I realize that by not writing about studs and fighters here, I’m losing that perspective of thought entirely, because of which I’ve not been able to write my book.

So, Chom (a friend) raised an important point during a discussion earlier today. He said that people who are studs, after they become “managers” (in which case their job is solely to manage other people. Think of someone like a partner in a consulting firm), start angling for more fighter work for their team.  That they seem to forget all their studness, and assume that all the people they manage are fighters.

I had argued earlier that once the partner of a consulting firm stops doing day-to-day work, the quality of work at the firm suffers. This post is an extension of that. So what Chom says inherently makes sense. Here’s why.

Stud work is risky. There is a good probability that it may not be completed. So when your target changes from the “total impact of work done” to “number of pieces of work successfully completed” the whole equation changes. You are not looking for those “big wins” from your team, any more. What you need from your team is a high rate of delivery, and a large number of projects that are completed. If you get big wins, that is just a bonus. But all you care for now is the number of wins.

So you start taking on more fighter work, and letting go of stud work. After all, it is now rational for you to do that. Your own working style can sit aside.

Alco Haalu

Does anyone know why the colloquial name for liquor in Kannada is “oil” (eNNe) while the corresponding word in Tamil is “water” (thaNNi)?

Is there some kind of a caste/class origin to it, with me being biased given that most Tamilians I know are upper caste/class, and that there is a different colloquial word that is in vogue among other classes? Because “eNNe” has more of a working-class feel to it (the name, that is), and one that has been appropriated by all sections of society.

While on the topic, I learn that the Gult word for alcohol is medicine (mandu)!! Fantastic!

What is the colloquial name for alcohol in your language, and what does it mean? Put it down in the comments here.

PS: and does anyone know why alcohol bottles are sold in black polythene covers? Never seen these things being used elsewhere so if you see a black polythene cover you know there’s a good probability it’ll contain a bottle of alcohol

Making guest list

So last night I sat down to do the presumably fun task of preparing my wedding guest list. This was just the first cut, where I just put down the names of people I want to invite in an excel sheet. In the second cut, I’ll parse the sheet and figure out how each person on the list should be invited – personally, or on phone, or email, and so on.

So there can be two kinds of error while making such a list – errors of omission and errors of commission. The probability of error or commission is quite low. After all there aren’t too many people who you explicitly don’t want at your wedding. And if there exist any such people, you will remember that only too well while putting their names down in the invite sheet.

Errors of omission is what I’m concerned about. There have been times in the past when people have gotten married, and I haven’t had a clue. It’s a different matter about whether I’d've gone or not, but I know that there is scope for hurt feelings if certain people are left out of the list. So one must be careful.

The problem is that I’m currently not in touch with a lot of these people. I would’ve been good enough friends with them at some point of time in life that I’d want to invite them to my wedding. But the fact that I haven’t kept in touch means I may not remember their existence, but when eventually they see my wedding pics on facebook it might result in a kinda hurtful “congrats” message.

The other question I must ask is that if I’m prone to forgetting about someone’s existence, if they are worth being invited at all. That I’ve forgotten about them means that obviously they are quite low in my list of people I want at my wedding. So am I generally paining myself by trying to remember people who I wouldn’t normally remember?

So far the easiest list I’ve made is from my batch of people from IIMB. We have a google doc with everyone’s names and personal details. So one parse through that meant I wouldn’t forget anyone’s existence. Much peace ensued. The problem is similar lists don’t exist for my other social networks. Anyway I’ll try my best.

Tangentially, another issue is about how “forcefully” I invite certain people who don’t live in Bangalore and have to fly down for my wedding. For a variety of reasons I happened to bunk their weddings, and now it’s a little embarrassing to insist that they be there.

PS: This old post of mine, I think, is pertinent.