Indian Americans and the Selection Bias

There is this one chart from the Economist that has been doing its rounds over the interwebs over the last few days:

Basically it shows that Indian Americans are much more accomplished academically and professionally compared to other immigrants. And there are many theories floating around as to why Indians are so successful.

The answer, however, is rather simple – selection bias. Migrating from India to the US was an extremely difficult task till the 1960s – there were some quotas that the US had for immigration under which the Indians had nothing. And when Indians did finally start migrating in the 1960s, it was mostly for education.

Most people who migrated from India to the US even in the 1960s and 70s did so to go to graduate school. And this meant that they already had 16 years of education in India, which either meant an engineering or medical degree, or a masters in one of the other fields. So basically most Indians migrating to the US were highly accomplished already.

And considering the kind of foreign exchange controls imposed by the Indian government, the only Indians who could afford to go to the US for an education were those that received a fellowship or support from their universities. Thus increasing the seelection bias! (Now that I’ve mentioned foreign exchange controls, you should listen to this song, which was apparently meant to parody such policies)

Yes, you had the odd Patel without much education who made it to open a “Potel” (Patel run Motel), but that is probably the reason that the Indian bubble in the above chart is not farther out!

So that Indians have done better than other migrating communities in the US is not about innate Indian intelligence, or innate Indian ability to work hard, or because the Americans took in the Indians much better than other nationality. It is simple selection bias, based on tight immigration controls and tight emigration controls and stupid foreign exchange policy on the part of Indian government (which, at one point of time, only allowed citizens to take out eight dollars from the country).

To illustrate this point, look at the country that is “second” (quotes since we are looking at two dimensions here, so second is subjective) in this list – Iran.

Axes of diversity

Companies and educational institutions, especially those that have a global footprint and a reputation to protect, make a big deal about diversity policies. It is almost impossible to sit through a recruitment or admissions talk by one such entity without a mention to their diversity policies, which they are proud of.

And they have good reasons to have a diverse workforce. It has been shown, for example, that diversity leads to better decision-making and overall better performance. Having a diverse workforce brings together people with different backgrounds, and since backgrounds influence opinion, a more diverse team is more likely to have more diversity of opinion which results in better decision making. And so forth.

The problem, however, is that it is not easy to simultaneously achieve diversity on all possible axes. Let’s say that we have defined a number of axes, and are looking to recruit an incoming MBA class. If we want diversity on each of these axes, selection of each candidate is going to rule out a large number of other candidates and we will need a really large pool to choose from. In other words, it is akin to the eight queens problem (where you have to place eight queens on a chessboard such that no two of them are on the same row, column or diagonal). For those of you not familiar with chess, think of it like a Sudoku puzzle.

Since the pool of candidates large enough to achieve diversity on all axes is simply not feasible, firms and schools choose to prioritise certain axes over others, and seek to achieve diversity in these chosen axes. And since they can arbitrarily choose axes that they can prioritise, the incentive is to pick out those axes where diversity is most visible.

And so when you go to a global organisation or school that preaches diversity, you will notice that they indeed have a very diverse workforce/student body in terms of gender, race, and nationality, which are fairly visible dimensions. Beyond this, the choice of dimensions to impose diversity on is a matter of discretion. So you have organisations which seek diversity in sexual orientation. Others seek diversity in age profile. Yet others in educational backgrounds. And so forth.

The result of prioritising more “visible” dimensions to ensure diversity is that organisations end up becoming horribly similar in the “sacrificed dimensions”. Check out this excerpt from Peter Thiel’s Zero to One, for example, on the founding members of paypal:

The early PayPal team worked well together because we were all the same kind of nerd. We all loved science fiction: Cryptonomicon was required reading, and we preferred the capitalist Star Wars to the communist Star Trek

Now, remember that this was a fairly diverse team when it came to ethnicity, nationality and sexuality. But in a less visible dimension, the team was not diverse at all. And Thiel mentions it in his book as if it’s a good thing that they all thought so similarly.

On a similar note, I once worked for an organisation that made great shakes of its diversity policy, and the organisation was pretty diverse in terms pretty much every visible axis of diversity. And the seminars (some compulsory) they organised helped me significantly broaden my outlook on issues such as race or sexual orientation. But when it came to work, the (fairly large) team was horribly similar. Quoting from an earlier blogpost (a bit ranty, I admit):

First, a large number of guys building models come from similar backgrounds, so they think similarly. Because so many people think similarly, the rest train themselves to think similarly (or else get nudged out, by whatever means). So you have massive organizations full of massively talented brilliant minds which all think similarly! Who is to ask the uncomfortable questions?

So essentially because you had a large organisation of people from basically similar educational backgrounds (masters and PhDs in similar subjects), their way of thinking became dominant, and others were forced to conform, leading to groupthink, which might have potentially led to mishaps (but didn’t, at least not in my time).

And what of the Ivy League schools that again pride themselves on (visible forms of) diversity? Here is an excerpt from William Deresiewicz’s excellent 2008 essay:

Elite schools pride themselves on their diversity, but that diversity is almost entirely a matter of ethnicity and race. With respect to class, these schools are largely—indeed increasingly—homogeneous. Visit any elite campus in our great nation and you can thrill to the heartwarming spectacle of the children of white businesspeople and professionals studying and playing alongside the children of black, Asian, and Latino businesspeople and professionals. At the same time, because these schools tend to cultivate liberal attitudes, they leave their students in the paradoxical position of wanting to advocate on behalf of the working class while being unable to hold a simple conversation with anyone in it.

So the next time you want to make your organisation diverse, think of which axes you want diversity on. If you are public-minded and want to brag about your diversity, the obvious way to go would be to be diverse on visible axes, but that leaves other issues. On the other hand you could put together a team of people that look the same but think different!

It’s entirely up to you!


Ganesha, wine and vodka

I know the wife has been intending to blog about this for a while now, but in this big bad blogosphere, intent counts for nothing, and given that she hasn’t written so far, I should go ahead and write this blog post. The basic funda is that Ganesha idols in “traditional” Indian culture, wine in European culture and smirnoff plain vodka in “modren” (sic) Indian culture are all similar.

So two days back I got invited to a “bring your own liquor” party. Now, there were other attendees who mentioned they were bringing stuff that I knew I was interested in drinking, like Desmondji Agave and Amrut Two Indies Rum. From that perspective, I knew that I wouldn’t be drinking whatever I carried. Yet, not carrying anything would make me look like a cheap guy, and this is one circle where I want to preserve my reputation. So what did I do? I picked up a bottle of Smirnoff plain vodka, simply because it is the most “fungible” drink. I’ll explain later.

Similarly, when you go for a function in India and don’t know what to gift, and are “too traditional” to gift gift cards, and think it’s not appropriate to give cash, you give a Ganesha idol. So for example after our wedding we had tonnes of Ganesha idols at home (similarly after our housewarming last year). Why did people gift Ganeshas? Because it is the most “fungible”. Again I’ll expect later.

And the wife reliably informs me that in Spain, when you have to go for a party but don’t know what to take, you take a bottle of wine. I don’t know about the fungibility of wine, but the fact that it is universally drunk, can be shared widely and is seen as a classy symbol makes it a popular choice of gift. So what connects these three?

So what connects? Fungibility of course. Economists have long argued that the best gift is cash, for the recipient can utilise that cash to buy the item that gives her maximum utility. Any non-cash gift decreases utility from the maximum that can be achieved by giving cash. This is a different discussion and I’ll not touch upon that now.

When you are going to a party, you can’t take along cash, so since the top choice is not available you take the “second best” option. What is the “second best” option in this case? Something that is close to cash, or something whose general utility is so high that the recipient values it as much as she would value the equivalent amount of cash. Of course you don’t assume that the recipient will sell your gift for cash, so you gift something that is a “safe option”, that you think they will have the least chance of rejecting.

So why did I take vodka? It is a universally popular, colourless odourless tasteless liquid, and I estimated that there was a good probability that the demand for that is going to be high. So even if I don’t drink what I carried, I posited, someone else will, and that will help me deliver maximal utility to the party.

With wine in Spain, you know everyone drinks and appreciates it, and there is a chance that it might be opened at the party itself. Even if it isn’t, wine in a sealed bottle doesn’t “depreciate”, and the host can then pass on some of the unused bottles at a party  that she attends! And soon there will be the virtuous wine circle. So essentially wine doesn’t disappoint, and is put to good use.

And it is exactly the same story with Ganesha idols. Like wine, it has intrinsic value. Who doesn’t like idols of a cute elephant-headed God? Maybe people who already have too many such idols? But then Ganesha idols don’t depreciate either, so all you need to do is to keep it in a safe place and pull it out the next time you’re going to a function! And thus the virtuous circle of Ganeshas will continue!

As it happened, at the end of Tuesday’s party, the bottles of Desmondji and Amrut Two Indies were empty. The Smirnoff I took remained unopened, as did another similar bottle which was possibly brought by another safe player. But I’m not concerned. I’m sure the hosts will consume it in due course, and even if they don’t, it will come of good use when they go to a party next!

Queueing up for boarding

I’m writing this from Barcelona airport, waiting for my flight to Doha, as I return to Bangalore today. A pre boarding announcement was made some minutes back but boarding is yet to commence, and this is what the airport looks like now.


As you can see it’s a fairly long line. And boarding hasn’t even begun. I used to believe that this phenomenon of queueing up for boarding is a uniquely Indian phenomenon, but over two trips to Europe over the last  months I’ve disabused myself of this notion.

In the last six months I’ve taken seven flights within Europe and for each of them there has been a long boarding queue, mostly before boarding has begun. In a couple of cases I’ve participated, and for good reason. On one occasion I chose not to participate and regretted it. But there have been occasions when I’ve chosen not to participate and haven’t regretted. I have no plans to participate in the queueing today. For an international flight it’s not rational. Let me explain.

Within Europe most low cost carriers charge for any checked in baggage. As a consequence, people carry on large pieces of luggage. As a consequence of this, there is severe shortage of luggage rack space within the flight and so if you don’t board early there’s a good chance that your baggage will have be carried in the hold, resulting in unnecessary delays after landing.

Thus, pricing of low cost carriers where they anally charge for luggage results in suboptimal boarding process, and significant discomfort.

In any case, Europeans are thus used to queueing up for boarding, for that can guarantee them a relatively smooth flight experience. And my theory is that this carries on to international  too.

But why is this irrational for international flights? Because most international flights (Qatar for sure) have reasonably generous check in baggage limits, because of which people don’t carry on massive pieces of luggage. The per capita availability of rack space, from my unscientific observations, also seems higher in wide body flights. Hence it matters less whether you board first or last.

Finally the queue didn’t matter today since Qatar decided to use the rather idiotic zone wise boarding system on the flight today. I’ve boarded. And had to place my bag one seat away. Not that I mind.

See you from the dark side

NRIs and the double narrative bias

By definition journalism suffers from the narrative bias. In other words, in most conditions, only the spectacular is newsworthy. To take a popular example, “dog bites man” will never make news because of its sheer predictability – it simply doesn’t add any information content.

As a consequence, journalism “suffers” from what I call the “spectacular bias”. A spectacular event is much more likely to be reported compared to an unspectacular one. This has several implications.
Firstly it leads to distorted and suboptimal choices. For example, following the two fires in Volvo buses in 2013-14 people stopped traveling by buses of that particular make. This was irrational because even after those accidents Volvo buses continued to be safer than buses of any other make. Yet, the fact that Volvo buses had been involved in the accidents were. Ade the focus of reports and that led to irrational responses. Related to this, I usually ask in lectures I take if anyone has seen a headline that says “Ashok Leyland bus catches fire,people die” and if not, if it means that Ashok Leyland buses never catch fire.
Now that it is established that journalism suffers from the narrative bias and spectacular bias, let’s take it one level further – what about people who get their news exclusively from social media? Let us assume that news that is shared widely on social media is a subset of news that is reported in the mainstream media (it is a reasonable assumption that anything that trends on social media will get immediately picked up and reported by the mainstream media).
What kind of news will these people (who get news exclusively from mainstream media) consume? If a news item makes it big in the social media then it implies that the news item has something about it that is spectacular, and something that is spectacular relative to anything else that is reported. Now considering that news itself is a collection of spectacular stories, what this implies is that what gets shared on social media is spectacular when compared to other spectacular stories, or these stories are doubly spectacular!
Considering that news itself can cause significant irrational decisions among people, imagine the kind of impact that consumption of news solely via social media can have! Without going into merits of the news, we can safely argue that it leads to irrational decisions and opinions.
Now let us consider one such class of people who mostly consume news via social media (we are making a leap of faith here). Temporarily going into anecdata territory, let me quote examples of possibly irrational behaviour by NRIs here. First there is this relative who left India about a decade back. About a year or so back he started this Facebook community called “Bangalore – Water Issues and Solutions” . None of his actions or statements from earlier had indicated that he has any interest in this topic.
Then there is my wife who has been living abroad for the last seven months. Glancing at political status messages on her Facebook feed you see the Uber rape case, Leslie Udwin’s documentary “India’s daughter” and the case of the death of IAS officer DK Ravi. Clearly spectacular among the spectacular. Or take my own case – I’ve been out of India for more than a week now, and lacking good traditional sources of getting news from back home (websites are too cumbersome, I’m relying on social media too).
The consequent “double narrative bias” (since what you consume from social media undergoes two levels of narrative bias) means that NRIs, lacking good sources of news from back home, are likely to have a warped view of the happenings in India. This implies that their idea of India is likely to be largely shaped by this bias and unlikely to be representative of what’s actually happening (I’m by no means giving a clean chit to resident Indians here, since several of those too suffer from this double narrative bias. But proportions are smaller than that for NRIs).
From this point of view the decision by the current government to extend online voting rights to NRIs needs to be called to question – since there is good reason to believe that their idea of India suffers from two levels of narrative bias (NRIs are currently not barred from voting but they need to travel to India to cast their votes. A change in this rule has been proposed).
This concept also helps us understand why political views of NRIs is likely to be much more polarised than that of the general Indian population (as exhibits note both the campaign to deny current Prime Minister Narendra Modi a visa to visit the U.S., and the reception he received when he ultimately went there). Their view is shaped by double narrative bias which leads to suboptimal opinions.
This double narrative bias also presents a good business opportunity – for media houses to target the diaspora. While most Indian publications have websites these are just repositories of news, and only news that has been widely shared gets mileage. This ends up reinforcing the double narrative bias.
What we need instead is a daily quick bite of news that can be consumed easily. This might lead to several NRIs to subscribe, so that they can get a quick understanding of what’s happening back home. (The economists espresso is a good template to follow for this one). And can serve to eliminate at least one level of narrative bias!

On tea being served before a talk

Later this evening I’m planning to go for this talk on Temples of the Badami Chalukyas, being held at the Max Mueller Bhavan in Indiranagar.

The text of the invite (not present in the picture above) says that “tea will be served before the talk”. Now you might think that it’s no big deal – but in my opinion that’s one marker that sets apart what can be a high-quality fulfilling event for the audience member than one that is merely good.

There are two reasons that people go to talks like this one – one is for the talk itself. For example, the topic of today’s talk looks extremely promising and exciting, and inherent interest in the topic itself is likely to spur audience participation. The other reason people like to go for such talks is that they are good places to meet with other like-minded people, and that is where the tea before the talk comes into the picture.

At the Pratap Bhanu Mehta lecture at IISc two weekends back, there was no tea being served prior to the talk. As a consequence, everyone who arrived walked straight into the hall and took their seats. When I arrived there there seemed to be no conversation whatsoever taking place, and so I went and quietly took my seat. Looking around, however, I noticed a number of acquaintances, and people I wanted to get to connect to (people who could connect me to these people were also in the audience). However, there was no chance of going up and talking to them and indulging in what some people uncharitably call as “networking”. And after the event was over everyone was in a hurry to get home and there was no chance to talk!

This is where tea before the talk comes into the picture. When tea is being served, people usually stand around the service area (not to be confused with Cervezaria) and mill around talking. It’s a great occasion to catch up with old acquaintances who happen to be there, make new acquaintances (that both of you have come for the same (usually esoteric) lecture indicates that you have some common interests) and generally talk to people. And meeting interesting people (new or old) at an event is always a good thing and attendees go home much more satisfied than they would had they only consumed the lecture!

Hence it is of paramount importance that tea (or coffee or milk or water or beer) be served before the talk, for it gives an opportunity for people to talk to each other, to network and to get more out of other attendees than they would from the talk itself. And if you are the antisocial type who doesn’t want to meet other attendees, you can quietly go take your seat while others are having tea – they won’t even notice you!

Market-making in on-demand markets

I’ve written a post on LinkedIn about the need for market-making in on-demand markets. I argue that for a market to be on-demand for one side, you require the other side to be able to provide liquidity. This liquidity comes at a cost and the side needs to get compensated for it. Driver incentive schemes at Ola/Uber and two-part electricity tariffs are examples of such incentives.

An excerpt:

In a platform business (or “two sided market”, or a market where the owner of the marketplace is not a participant), however, the owner of the market cannot provide liquidity himself since he is not a participant. Thus, in order to maintain it “on demand”, he should be able to incentivise a set of participants who are willing to provide liquidity in the market. And in return for such liquidity provided, these providers need to be paid a fee in exchange for the liquidity thus provided.

Read the whole thing! :)