Tag: exit polls

Opinion polling in India and the US

(Relative) old-time readers of this blog might recall that in 2013-14 I wrote a column called “Election Metrics” for Mint, where I used data to analyse elections and everything else related to that. This being the election where Narendra Modi suddenly emerged as a spectacular winner, the hype was high. And I think a lot of people did read my writing during that time.

In any case, somewhere during that time, my editor called me “Nate Silver of India”.

I followed that up with an article on why “there can be no Nate Silver in India” (now they seem to have put it behind a sort of limited paywall). In that, I wrote about the polling systems in India and in the US, and about how India is so behind the US when it comes to opinion polling.

Basically, India has fewer opinion polls. Many more political parties. A far more diverse electorate. Less disclosure when it comes to opinion polls. A parliamentary system. And so on and so forth.

Now, seven years later, as we are close to a US presidential election, I’m not sure the American opinion polls are as great as I made them out to be. Sure, all the above still apply. And when these poll results are put in the hands of a skilled analyst like Nate Silver, it is possible to make high quality forecasts based on that.

However, the reporting of these polls in the mainstream media, based on my limited sampling, is possibly not of much higher quality than what we see in India.

Basically I don’t understand why analysts abroad make such a big deal of “vote share” when what really matters is the “seat share”.

Like in 2016, Hillary Clinton won more votes than Donald Trump, but Trump won the election because he got “more seats” (if you think about it, the US presidential elections is like a first past the post parliamentary election with MASSIVE constituencies (California giving you 55 seats, etc.) ).

And by looking at the news (and social media), it seems like a lot of Americans just didn’t seem to get it. People alleged that Trump “stole the election” (while all he did was optimise based on the rules of the game). They started questioning the rules. They seemingly forgot the rules themselves in the process.

I think this has to do with the way opinion polls are reported in the US. Check out this graphic, for example, versions of which have been floating around on mainstream and social media for a few months now.

This shows voting intention. It shows what proportion of people surveyed have said they will vote for one of the two candidates (this is across polls. The reason this graph looks so “continuous” is that there are so many polls in the US). However, this shows vote share, and that might have nothing to do with seat share.

The problem with a lot (or most) opinion polls in India is that they give seat share predictions without bothering to mention what the vote share prediction is. Most don’t talk about sample sizes. This makes it incredibly hard to trust these polls.

The US polls (and media reports of those) have the opposite problem – they try to forecast vote share without trying to forecast how many “seats” they will translate to. “Biden has an 8 percentage point lead over Trump” says nothing. What I’m looking for is something like “as things stand, Biden is likely to get 20 (+/- 15) more electoral college votes than Trump”. Because electoral college votes is what this election is about. The vote share (or “popular vote”, as they call it in the US (perhaps giving it a bit more legitimacy than it deserves) ), for the purpose of the ultimate result, doesn’t matter.

In the Indian context, I had written this piece on how to convert votes to seats (again paywalled, it seems like). There, I had put some pictures (based on state-wise data from general elections in India before 2014).

An image from my article for Mint in 2014 on converting votes to seats. Look at the bottom left graph

What I had found is that in a two-cornered contest, small differences in vote share could make a massive difference in the number of seats won. This is precisely the situation that they have in the US – a two cornered contest. And that means opinion polls predicting vote shares only should be taken with some salt.

Time for bragging

So the Karnataka polls are done and dusted. The Congress will form the next government here and hopefully they won’t mess up. This post, however, is not about that. This is to stake claim on some personal bragging rights.

1. Back in March, after the results of the Urban Local Body polls came out, I had predicted a victory for the Congress in the assembly elections.

2. Then, a couple of weeks back, I used the logic that people like to vote for the winner, and this winner-chasing will result in a self-fulfilling prophecy that will lead to a comfortable Congress victory.

These two predictions were on the “Resident Quant” blog that I run for the Takshashila Institution. It was a classic prediction strategy – put out your predictions in a slightly obscure place, so that you can quickly bury it in case it doesn’t turn out to be right, but showcase it in case you are indeed correct! After that, however, things went slightly wrong (or right?). Looking at my election coverage Mint asked me to start writing for them.

As it happened I didn’t venture to make further predictions till the elections, apart from building a DIY model where people could input swings in favour of or against parties, and get a seat projection. Watching the exit polls on Sunday, however, compelled me to plug in the exit poll numbers into my DIY model, and come up with my own prediction. I quickly wrote up a short piece.

3. As it happened, Mint decided to publish my predictions on its front page, and now I had nowhere to hide. I had taken a more extreme position compared to most other pollsters. While they had taken care to include some numbers that didn’t mean an absolute majority in the range the predicted for the Congress (so as to shield themselves in that eventuality), I found my model compelling enough to predict an outright victory for the Congress. “A comfortable majority of at least 125 seats”, I wrote.

I had a fairly stressful day today, as the counting took place. Initial times were good, as the early leads went according to my predictions. Even when the BJP had more leads than the Congress, I knew those were in seats that I had anyway tipped them to win, so I felt smug. Things started going bad, however, when the wins of the independents started coming out. The model I had used was unable to take care of them, so I had completely left them out of my analysis. And now I was staring at the possibility that the Congress may not even hit the magic figure of 113 (for an absolute majority), let alone reach my prediction of 125. I prepared myself to eat the humble pie.

Things started turning then, however. It turned out that counting had begun late in the hyderabad karnataka seats – a region that the Congress virtually swept. As I left my seat to get myself some lunch, the Congress number tipped past 113. And soon it was at 119. And then five minutes again back at 113. And so it continued to see-saw for a while, as I sat at the edge of my office chair which I had transplanted to in front of my television.

And then it ticked up again, and stayed at 119 for a while. And soon it was ticking past 120. All results have now been declared, with the Congress clocking up 121 seats. It falls short of the majority I had predicted, but it is a comfortable majority nevertheless. I know I got the BJP number horribly wrong, but so did most other pollsters, for nobody expected them to get only 20% of the popular vote. I also admit to have missed the surge in Independents and “Others”.

Nevertheless, I think I’ve consistently got the results of the elections broadly right, and so I can stake claim to some bragging rights. Do you think I’m being unreasonable?