How to avoid Murray Gell-Mann Amnesia

I’ve written about Murray Gell-Mann Amnesia here a couple of times. The first time was when I discovered it in The Economist. Another time was when I likened it to the Vodnoy Paradox, where people recommend deregulation in all sectors except their own.

I’ve been thinking a lot about Murray Gell-Mann Amnesia for a while now because I recently bought a (cheap – ?25 a week) subscription to the New York times). And they take the level of Sominism in their India coverage (no surprise since Somini Sengupta used to be their correspondent) another launch.

In fact, when I was mulling buying a subscription, I was explicitly warned about their India coverage.

And as I’ve read them for the last month and a half, this remains true. Their coverage of India is really shitty. It’s no different for many other global “liberal” newspapers such as the Guardian, or the Washington Post, or the Atlantic. The most baffling thing is that all these worthies is that they frequently employ writers of fiction as their vehicles of choice to interpret India for their readers (recently even the FT fell for this, asking the excellent-writer-but-insanely-political Arundhati Roy to write for them about India).

In any case, I’ve been wondering why this is the case. Why is it that these newspapers do such a shoddy job of covering India (or possibly any other emerging market) (I’m not saying they do a great job of covering their home markets either, since these newspapers have all become rather political, but at least there is some good coverage)?

My hypothesis about this is that they do a shitty job of covering India because they don’t care about the Indian reader, who contributes a microscopic minority of their revenues. That they can offer their zero-marginal-cost product for half of what Indian newspapers charge Indians for print subscriptions suggests that Indian readers don’t contribute significantly to their revenues.

Instead, what they have is large numbers of paying subscribers in their home markets who are (rightly) their primary audience. And because the people who are paying them and the people they are writing about are disjoint, there is no need to be authentic in their coverage. They can simply offer their readership the sort of slant and opinions they want without ever being held accountable.

It is similar in the case of Murray Gell-Mann. The science reporting can afford to be bad because scientists who really care about the research form only a tiny part of the subscriber base of the newspaper, and they possibly couldn’t care about holding the papers to account.

Now you can argue that each and every person is a “minority of one”, and so newspaper coverage ought to be uniformly shitty about all subjects. Except that some groups of readers are more similar to each other than they are to others, and such groups are likely to be “better taken care of” by the newspapers than all the other readership.

I don’t really know how this can be solved. For each newspaper, there will always be groups of core readership who might hold them to account, but there will be nobody holding them to account on vast sections of their coverage.

The only thing I can think of is the Times of India model – apart from being mass-market advertising funded, they have the habit of “putting ordinary people in the newspaper” through their tabloid supplements such as Bangalore Times (this was stated to me by someone who used  to work with the group). When you put ordinary people in the paper, these ordinary people will be more invested, and you better not write shit about them.

Yet another pinnacle

During our IIMB days, Kodhi and I used to measure our lives in pinnacles. Pinnacles could come through various ways. Getting a hug from a sought-after person of the opposite gender usually qualified. Getting featured in newspapers also worked. Sometimes even a compliment from a professor would be enough of a “pinnacle”.

In any case, in college days pinnacles keep coming your way, and you can live your life from one pinnacle to another. Once you graduate, that suddenly stops. Positive feedback of any kind in your work life is rare. If you “manage to do well in social life”, it might work, but there is really nobody else to show off your pinnacle to. It is really hard to adjust to this sudden paucity of positive feedback in life, and this usually leads to what people call as “quarter life crisis”.

Shortly after I had resolved my quarter life crisis, and “done well in social life”, I decided to change track. I quit full time employment, and as you all might know, have been pursuing a sort of “portfolio life” in the last eight odd years. And this means doing several things apart from the thing that contributes to most of my income.

One upside of this kind of life (lack of steady cash flow is the big downside) is that you keep getting pinnacles. Publishing my book was a pinnacle, for example. Getting invited to write regularly for Mint was another. Becoming a bit of a social media star (nothing like yesterday) in the run up to the 2014 general elections was yet another. And there were the kicks about being invited to teach at IIMB. And all that.

It had been a while since I had one such pinnacle. Perhaps the last one was in 2018, during the Karnataka Assembly Elections, when I had my first shot at television punditry, when I appeared on News9, and waxed eloquent about sample sizes and survey techniques.

In any case, that was nothing compared to the sort of pinnacle that I’ve got following my tweetstorm from yesterday. This email came to the inbox of NED Talks (I didn’t know that email ID was public) this afternoon:

Kind Attn: Karthik Shashidhar, Founder NED TALK

Dear Sir,

Greetings from Republic TV!

I’m <redacted> , a Mumbai based News Coordinator with Republic TV. Republic TV is India’s first and only Independent News Venture headed by Mr. Arnab Goswami.

Sir,  we would like to get in touch with you for our show on Corona virus reality on Rahul Gandhi Claim that Lock down is temporary measures and not the solution to defeat virus and need more testing to be done in the country.

Sir,It will be our pleasure to have you join us on our channel at 9PM.

We, at Republic TV,  believe that your command over the issue will add depth and perspective to our discussions and help mould popular discourse.

As it happened, I was unable to accept this invitation. However, I’m documenting this here to record this absolute pinnacle of life. The next time I feel shitty about myself, or feel a sort of imposter syndrome, I can look at this invite and think that I’ve truly arrived in life.

PS: Just look at the number of times I’ve been called “Sir” in that email. That alone should constitute a pinnacle.

The Two Overton Windows

If you want to appear intelligent when discussing something about public policy, you could do worse than uttering the phrase “Overton Window”. The Overton Window, “invented” by one Joseph Overton, suggests that there is a “range of policies acceptable to political mainstream”.

And so you frequently have political commentators talking about the Overton Window “shifting” whenever a new political idea (or person) comes to the fore. This was bandied about much when Modi became Prime Minister of India, or when Trump became President of the US, or when Jeremy Corbyn became the Labour Party leader.

While “shifting Overton window” is something you come across rather often in policy discourse, my argument is that with the rise of subscription media, the Overton window is not shifting as much as it is “splitting”. In other words, we now have not one but two Overton Windows.

Without loss of generality, let us call them the “Jamie Overton Window” and the “Craig Overton Window”. Since both the twins are right arm fast bowlers, it doesn’t matter which brother is associated with which Overton Window.

So how did we get here, and what does it mean for us?

We started with the classic Overton Window. Let’s assume that all politics can be reduced to one axis (if we do a Principal Component Analysis of political views, the principal axis is certain to account for a large share of the variance, so this is not a bad assumption). So the Overton Window can be referred to by a line which the shifts.

As long as the world was “ruled by mainstream media”, this Overton Window kept moving back and forth, expanding and contracting, but it remained united. And then with the start of subscription ad-free media (maybe a decade or decade and half ago), the Overton Window started expanding.

The “left media” (that’s a convenient term isn’t it?) started admitting stuff that was left to the then Overton Window. The “right media” started admitting stuff that was to the right of the then Overton Window. And so over time, the Overton Window started expanding. And things can’t get into the media Overton Window unless they’re part of the mainstream political Overton Window.

The thing is that as the media became subscription-heavy and hence biased, political ideas that were once on the fringe now got a voice. And so the Overton Window got larger and larger.

Until a point when it got so unwieldy that it split, giving rise to Jamie and Craig. The image on the right is an approximate illustration of what happened.

And once the Overton Window split, there was no looking back. They started moving away from each other well-at-a-faster-rate. The Jamies could not come to terms with the policies of the Craigs, and vice versa. Political analysts and commentators started getting associated with the Jamie and Craig camps.

For a while, a few commentators continued to write for both sides, but the extreme fringes, which were getting more and more extreme, started overreacting. “How can we have someone who has written 10 articles for Craigs write for us”, the Jamies asked. “Most of our commentators are Craigs, so we might as well become a Craig newspaper”, the other side reasoned.

And that’s where mainstream media is going. The Overton Window has split down the middle. Crossing this gap is considered a crime worse than crossing the floor in Parliament.

Sadly, it is not just media that is getting Jamie and Craig. Mainstream politics reflects this as well, and so across countries we get political opponents who just cannot talk to each other, since everything one says is outside the Overton Window of the other.

Maybe the only way this can end is by going across axes, or inventing a new axis even. With the current spectrum politics, there is no hope of the two Overton Windows coming to meet.

 

Footage

So after a fifteen year gap, I was in the Times of India yesterday, writing about the joys of working from home (I’d shared the clipping yesterday, sharing it again). The interesting thing is that this piece got me the kind of attention that I very rarely got with my six  years with the HT Media family (Mint and Hindustan Times).

The main reason, I guess, that this got far more footage, was that it came in a newspaper with a really high circulation. ToI is by far the number one English newspaper in India. While HT may be number two, we don’t even know how much of a number two it is, since it seemingly didn’t participate in the last Indian Readership Survey.

Moreover, ToI is read widely by people in my network. While the same might be true of Mint (at least until its distribution in Bangalore went kaput), it was surely not the case with HT. I didn’t know anyone who read the paper, and since my articles mostly never appeared online, they seemed to go into a black hole.

Another reason why my article got noticed so widely was the positioning in the paper – it was part of ToI’s massively extended “page one” (it came on the back of the front page, which was full of advertisements). So anyone who picked up the paper would have seen this in the first “real page of news” (though this page was filled with analysis of working from home).

On top of all this, I think my mugshot accompanying the article made a lot of difference. While the title of the article itself might have been missed by a few, my photo popping out of there (it helps I have the same photo on my Twitter, Facebook, LinkedIn and WhatsApp – thanks Anuroop) ensured that anyone who paid remote attention to my face would end up reading the article, and that helped me get further reach among my existing network.

ToI is going to pay me a nominal amount for this article, far less than what Mint or HT used to pay me per piece (then again, this one is completely non-technical), but I don’t seem to mind it at all. That it’s given me much more reach among my network means that I’m satisfied with ToI’s nominal payment.

Thinking about it, if we think of newspapers as three-sided markets connecting writers, readers and advertisers, it is possible that others who write for ToI do so for below market prices as well, for it has an incredibly large reach among “people like us”. And that sets the size-related network effects (“flywheel” as silicon valley types like to call it) in action among the writer side as well -you don’t write for money along, and if it can be sort of guaranteed that a larger number of people will read what you write, you will be willing to take lower payment.

In any case, this ToI thingy was a one-off (the last time I’d written for them was way back in 2005, when I was a student – it’s incredible I’ve given this post the same title as that one. I guess I haven’t grown up). But I may not mind doing more of such stuff for them. The more obscure the paper, though, the higher I’ll be inclined to charge! Oh, and henceforth, I’ll insist my mugshot goes with everything I write, even if that lowers my monetary fees.

 

News, Subscription, Advertising, and Bias

Dibyendu Mishra and Joyojeet Pal of the University of Michigan have some very interesting research out on the political bias of Indian news publications. Rather than do complicated gymnastics such as NLP, they’ve simply looked at the share of articles from each news publication that is retweeted by BJP and non-BJP publications, to draw out a measure of their bias (see link above for methodology).

They have made a nice scatter plot (the other axis is how “popular” these news outlets are in terms of the number of articles retweeted), and looking left to right, you can see the understood (by politicians) bias of various Indian news publications. As Helmet pointed out on Twitter, the most “centrist” news outlets seem to be the Times of India and the Economic Times, both from the Bennett, Coleman and Company group, who people crib about for “being too commercial” and “having too many advertisements”.

This reminds me of another piece of analysis that was in the news a few months ago, about how subscription-driven online news has led to news outlets being politically polarising. For example, Zach Goldberg did some analysis of frequency of words/phrases in the New York Times that are associated with the extreme left.

Note the inflexion point sometime in 2012 or so, around the same time when the NY Times put up its paywall.

David Rozado has a more comprehensive picture (check out his nifty tool here).

The idea is this – when newspapers depended on advertising for most of their funding, they needed to be centrist. Taking political sides meant that large mass-market advertisers wouldn’t want to advertise in this newspaper, and the paper would thus lose revenues. Hence, for the longest time, whatever the quality of the reporting and writing was, news outlets strove to be reasonably politically unbiased – taking sides would mean a loss of money.

Once digital took off, and it became clear that digital advertising wouldn’t really sustain the papers, they started putting their content behind paywalls. And subscription revenues meant two things – news outlets weren’t as beholden to advertisers as they used to be, and it was easier to get paying subscribers if you had a strong ideology. Moreover, online you can provide targeted advertising (rather than mass-market), so you can get away with being biased. And so with the coming of paywalls, newspapers started becoming far more political as the New York Times graph above indicates.

In India, there haven’t been too many publications behind paywalls, but media is evidently getting more and more polarised over time. Papers and channels are branding themselves (implicitly) as being pro or against a particular political party, and that is driving their viewership.

While these media outlets are good for fanbois (and fangirls) of particular ideologies, the ideological bent has meant that it has become harder to get objective news.

And that’s where money, and advertising, comes in.

The positioning of ToI and ET in the middle of the Indian media ideological graph is interesting because they belong to a group that is brazen about commercialisation and revenues (from advertising). And in terms of news objectivity, that’s a good thing. Since ToI and ET are highly money minded, they want to get as much advertising as possible, and in order to attract mass marketers, they need to not be biased.

Taking a political stand means pissing off people belonging to the opposite political persuasion, and that means less readership, which means less advertising revenues. And so if you read the editorials of these newspapers (I read ET everyday), you see that they maintain a careful balance of not appearing too biased in favour or against any party. And you see them raking in the advertisers while more biased (and “ideological”) competitors are forced to request for donations, or put up paywalls restricting their readership.

Putting it another way, there is no surprise that ToI and ET are not biased in their news, and are retweeted by politicians of all persuasions. It is the classic money-driven media model, and that is the one that is capable of providing the most objective news.

Tautological Claims

Sometimes the media can’t easily reason on what led to something that they consider to be negative. In such cases they resort to tautologies. One version of this was seen in the late 2000s, during the Global Financial Crisis. The crisis “was caused by greed”, claimed many a story. “It is because of the greed of a handful of bankers that we have to suffer”, they said.

Fast forward ten to twelve years later, and the global financial crisis is behind us (though many economies aren’t yet doing as well as they were before that crisis). The big problem that a lot of people are facing is addiction – to their smartphones, to apps, to social media, and so on. Once again, media at large seems to have been unable to reason effectively on why this addiction is happening. And so once again, they are resulting in “tautologies”.

“Apps are engineered so that you engage more with them”, they say. If you ask the product manager in charge of the app, you will find out that his metric is to increase user engagement, and make sure people spend more time on the app. “Apps use psychological tools to make you spend more time on them”, the outlets write, as if that is a bad thing.

However, if you are an overstretched product manager hard-pressed to increase engagement, there is no surprise that you would use every possible method – logical and psychological, to do so. And if that means relying on psychological research that talks about how to increase addiction, so be it!

It is tautological that social media companies “want to increase engagement” or “want to increase the amount of time people spend on the platforms”, and that they will try to achieve these goals. So when media agencies talk about these goals as something to be scared about, it’s like they’re bullshitting – there’s absolutely no information that is being added in such headlines.

It is similar to how a decade and a bit ago the same media decided to blame a fundamental human tendency – greed – for the financial crisis.

What Makes The Athletic Great

In recent times I’ve bought subscriptions to two online media outlets – The Ken and The Athletic. I’d subscribed to the Ken a year ago, and was happy enough with the hit rate of their pieces (I’d find one in two pieces insightful) that I extended my subscription for three years earlier this year.

And since I did that extension, the product has been disappointing. They lost half their team to The Morning Context, a breakaway (and similar) outlet. They decided to expand in South East Asia, and since I have little interest in articles about that reason (at least not enough to pay for the writing), that automatically means less content that interest me. In some senses their quality is slipping. All this together means that I find less than one in five articles in The Ken compelling, and with the frequency of their publication (one article every weekday) I’m pretty disappointed.

Maybe it has to do with Marie Kondo’s popularity, or interest in behavioural economics research about the paradox of choice, but organisations are starting to make minimalism and limitations in inventory a virtue. The Ken started with the aim of “exactly one long form article every day”.

Having less choice, and being minimalistic, is good when this limited choice fits the appetite of the customer. However, if the choice isn’t particularly relevant, then minimalism becomes a bug rather than a feature – the customer doesn’t find what she is looking for and goes on to another outlet.

In that sense, I quite like the model of The Athletic, which I bought a year-long subscription to a year back. The Athletic’s model is just the opposite – massively high volumes with a highly curated personal feed. And maybe they’ve got their curation right, in terms of getting customers to click on the right kind of tags at the time of sign up, but so far I’ve found at least two useful articles on their site every single day since I turned up. And that’s insane value for money!

And that is despite me being interested in exactly one out of the nine sports that The Athletic covers (it’s mostly US-centric, and I don’t follow American sport at all. However I guess I’ll find it useful when I have to follow any controversy in American sport). And I’m interested in a subset of that – I follow one league (English Premier League) and games played by a handful of clubs in that league.

If I compare The Athletic to Netflix (both subscription-driven media outlets with large volumes of content), where the former scores is in its discoverability.

Maybe sport is easier compared to movies/tv shows in order to understand someone’s interests. Maybe it is that The Athletic, right up front, asked me to identify which sports, leagues, authors and teams I’m interested in (Netflix never made an attempt to do that). Maybe it is that The Athletic, with loads of fresh content every single day, is able to serve my preferences far easier than Netflix.

In any case, reading the Athletic makes me think that if I were to run a media outlet some day, I would want to follow that kind of a model – produce lots of content, so that lots of people will be interested in buying subscriptions, and then hope to use superior algorithms to make sure that people can see what they want and not have to cut through too much noise in order to do so!

The Business Standard is innumerate

I guess there is not that much information in the headline here – claiming that a bunch of journalists and editors are innumerate is like saying that the sky is blue. You would be hard-pressed to find journalists and editors who can actually parse numbers, though I must mention that I’ve been lucky enough to work with a few editors who actually understand arithmetic!

So what happened today? Basically in today’s front page, BS journalists (one Vinay Umarji in particular) and editors have displayed an utter lack of understanding on how relative grading and percentiles work. The context is CAT results, which came out yesterday.

(I’ve put a scan since the online version is behind a paywall).

There is information in saying that “number of candidates scoring 100 percentile is lowest in six years”, and the information I take out of that is that the number of test takers this year is the lowest in six years.

And for four of those six years, the numbers were inflated, since double the number of people who were supposed to get 100 percentile actually got 100 percentile. Since CAT percentiles are given to two decimal places, you get 100 percentile if you are in the top 0.005% of all candidates who took the exam. Or – if your “percentile” is higher than 99.995, it gets rounded up to 100.

For three years in the middle, the CAT administrators (usually they’re Quantitative Methods professors at IIMs), for whatever reason, rounded up everyone who got a percentile higher than 99.990 to 100. I’d written about that in my article for Mint three years back.

Coming back, CAT is an exam that follows relative grading. All that someone  has got “100 percentile” means is that they are within the top 0.005% of all candidates who wrote the exam. So if more candidates write the exam, more people will get “100 percentile”. In my time, for example (CAT 2003-4) some 1.3 lakh people had written the exam, so 7 of us got “100 percentile”. Nowadays the number of test takers has gone up, so more people get that score.

And then I found the rest of the article funny in a way as well, trying to do some sort of sociological analysis of the backgrounds of the people who had scored highly in the exam.

PS: The graph doesn’t give out much information (and I don’t know why the 2019 data point is missing there), but I guess it’s been put in there to make the journalists and editors seem more numerate than they are.

 

Video Geographic Monopolies

There is one quirk about video which we don’t face with print – some content is simply impossible to access legally in some parts of the world.

I’m specifically talking about BBC’s Match Of The Day, their end of day highlights package covering the English Premier League. It was one show that I watched unfailingly during my time in London, both for the match highlights, and for the quality of the discussion featuring Gary Lineker, Alan Shearer, Ian Wright et al.

Now I find that the show is simply not available in India – some youtube channels illegally offer the show (before they are taken down, I guess), but without the bits that show pictures of the game (which they are not allowed to show). And that makes for rather painful watching, knowing that you’re watching something substandard.

This is not the case with something like text – as long as I’m willing to pay, I’m able to access content produced anywhere in the world. I can sit here in Bangalore and buy a subscription to the New York Times, and access all its content. Audio is also similar – I can sit here and subscribe to any international podcast, and be able to access the content.

Video doesn’t work that way. The problem is with the way rights are sold – the Star network, for example, has a monopoly on showing pictures from the Premier League in India (having paid a substantial amount for it). And part of their arrangement means that nobody else is allowed to broadcast this material in India. A consequence of this is that we are stuck with whatever (mostly crappy) analysis Star decides to provide around its games. Stuff that is unwatchable.

There is a lot of great sport content online, but the video part is constrained by the inability to show pictures. Check out analysis by Tifo Football, for example – it’s absolutely top class. However, for most games, they have to rely on stock images and block diagrams since they can’t show the pictures which someone has a monopoly on. And that makes the analysis less rich (the Athletic, which I have a subscription to, “solves” this in an interesting way – by using screenshots of the TV footage of the game as part of their text analysis).

I wonder if there is a way out of this. Some leagues such as the NBA have shown some enlightened thinking on this – while they are anal about copyright of their live feed, they don’t care about copyrights on recorded footage. This means that anyone can use footage from historical NBA games as part of their analysis. Better analysis means more people interested in the sport, which means more people watching the live feed, which makes more money for the league (read this excellent interview of NBA Commissioner Adam Silver).

I’m also beginning to think if there is a regulatory antitrust response to this issue. Video distribution (especially of live content) is a natural monopoly, so it doesn’t make sense to have competing broadcasters. However, I wonder if there is any regulation possible for historical feeds that makes them more tradable (with the rights holders getting appropriately compensated without much transaction costs)!

One can only hope..

Evolution of sports broadcasting

I had a pleasant surprise yesterday morning when I was watching the highlights of Liverpool’s 4-0 victory at Leicester. The picture quality suddenly looked better. The production aesthetics in the first few seconds (before coverage of the actual match began) looked “American”. I doubted myself for a minute if this was actually English football I was watching.

And then I remembered that the pictures for this  game came from Amazon Prime. The streaming service had got rights to broadcast two full rounds of Premier League games this season, making a small chink in the duopoly of Sky Sports and BT Sport.

Traditional media wasn’t too impressed by it. Streaming necessarily meant a small delay in broadcast, and that made it less exciting for some viewers. The Guardian predictably made a noise about the “corporate takeover” with Amazon’s entry. From all the reports I read (mostly across the Guardian and the Athletic), commentators seemed intent on picking holes in Amazon’s performance.

That said, the new broadcaster also brought a fresh production aesthetic. While there were the inevitable teething problems (I must confess I didn’t watch these games live – being midweek evening games, they were very late night in India), Amazon for sure brought some new ideas into the broadcast.

Just like Fox Sports had done when it had done a big launch into NFL broadcasting in the early 90s. Read this oral history of that episode. It’s rather fantastic. Among the “innovations” that Fox Sports brought into American broadcasting (based on its sports broadcast in Australia, primarily) was this box at the corner showing the time and the live score. The thing wasn’t initially well received, but is now a fixture.

For evolution to happen, you need sex. And that means mixing things up, in ways they weren’t mixed before. If we were all the children of a super-god and a super-goddess, we would all be pretty much the same since the amount of “innovation” that could happen would be limited. And things would be boring, and static. Complex forms such as human beings could have never happened.

It is similar in business, and sports broadcasting, as well. When you have the same channels covering the same sports, they get into well-set local optima, and nothing new is tried. There is no necessity for improvement in that sense.

When new players comes in, preferably from another market, however, they see the need to differentiate themselves, and bring in ideas from their former market. And this leads to a crossover of ideas. In their efforts to stand out and make an impact, they might also bring in some ideas never seen anywhere – “mutations” in the evolutionary sense.

A lot of them don’t make sense and they die out. Others score unexpected hits and catch on. And that way, this memetic evolution leads to better business.

The great thing about memetic evolution is that while bad ideas come along much more often than good ideas, they get discarded fairly quickly, while the good ideas live on. And that leads to overall better products.

Right now in India we have a duopoly in sports broadcasting, controlled by the Star family and the Sony family. I’ve ranted several times about how the latter is absolutely atrocious and does nothing to improve the game. Hopefully a new player getting rights of some sport here will shake things up and bring in fresh ideas. Even if some of the ideas turn out to be bad, there will be plenty of good ideas.

Check out the highlights of the Leicester-Liverpool game, and you’ll get an idea.