news

Risk and data

A while back a group of <a large number of scientists> wrote an open letter to the Prime Minister demanding greater data sharing with them. I must say that the letter is written in academic language and the effort to understand it was too much, but in the interest of fairness I’ll put a screenshot that was posted on twitter here.

I don’t know about this clinical and academic data. However, the holding back of one kind of data, in my opinion, has massively (and negatively) impacted people’s mental health and risk calculations.

This is data on mortality and risk. The kind of questions that I expect government data to have answered was:

If I get covid-19 (now in the second wave), what is the likelihood that I will die?
If my oxygen level drops to 90 (>= 94 is “normal”), what is the likelihood that I will die?
If I go to hospital, what is the likelihood I will die?
If I go to ICU what is the likelihood I will die?
What is the likelihood of a teenager who contracts the virus (and is otherwise in good health) dying of the virus?

And so on. Simple risk-based questions whose answers can help people calibrate their lives and take calculated enough risks to get on with it without putting themselves and their loved ones at risk.

Instead, what we find from official sources are nothing but aggregates. Total numbers of people infected, dead, recovered and so on. And it is impossible to infer answers to the “risk questions” based no that.

And who fill in the gaps? Media of course.

I must have discussed “spectacularness bias” on this blog several times before. Basically the idea is that for something to be news, it needs to carry information. And an event carries information if it occurs despite having a low prior probability (or not occurring despite a high prior probability). As I put it in my lectures, “‘dog bites man’ is not news. ‘man bits dog’ is news”.

So when we rely on media reports to fill in our gaps in our risk systems, we end up taking all the wrong kinds of lessons. We learn that one seventeen year old boy died of covid despite being otherwise healthy. In the absence of other information, we assume that teenagers are under grave risk from the disease.

Similarly, cases of children looking for ICU beds get forwarded far more than cases of old people looking for ICU beds. In the absence of risk information, we assume that the situation must be grave among children.

Old people dying from covid goes unreported (unless the person was famous in some way or the other), since the information content in that is low. Young people dying gets amplified.

Based on all the reports that we see in the papers and other media (including social media), we get an entirely warped sense of what the risk profile of the disease is. And panic. When we panic, our health gets worse.

Oh, and I haven’t even spoken about bad risk reporting in the media. I saw a report in the Times of India this morning (unable to find a link to it) that said that “young are facing higher mortality in this wave”. Basically the story said that people under 60 account for a far higher proportion of deaths in the second wave than in the first.

Now there are two problems with that story.

A large proportion of over 60s in India are vaccinated, so mortality is likely to be lower in this cohort.
What we need is the likelihood of a person under 60 dying upon contracting covid. NOT the proportion of deaths accounted for by under 60s. This is the classic “averaging along the wrong axis” that they unleash upon you in the first test of any statistics course.

Anyway, so what kind of data would have helped?

Age profile of people testing positive, preferably state wise (any finer will be noise)
Age profile of people dying of covid-19, again state wise

I’m sure the government collects this data. Just that they’re not used to releasing this kind fo data, so we’re not getting it. And so we have to rely on the media and its spectacularness bias to get our information. And so we panic.

PS: By no means am I stating that covid-19 is not a risk. All I am stating is that the information we have been given doesn’t help us make good risk decisions

Apolitical fake news

For the last 4-5 years, the ills of “political fake news” have been well documented – documented well enough that I don’t even need to link to them (I think). However, there is another kind of fake news that doesn’t get the sort of (negative) attention it deserves – unbiased or apolitical fake news.

Before we describe such news, a couple of frameworks. Firstly, there are two kinds of media publications – periodicals and perennial. Periodicals deliver news at a certain periodicity – daily or weekly or monthly or whatever. Their job is to tell the reader what happened in the world (or the subset of the world that the publication focuses on) since the previous edition. Examples of periodicals include newspapers and magazines and the 9 o’clock (or whenever) news on Doordarshan.

The other side is perennials, which are “always on”. When some news breaks, their mandate is to break it to their audience as quickly as possible. When there is no breaking news, they need to make up something, or analyse, or have talk shows and shouting matches, or whatever. Examples of perennial publications include 24×7 TV channels and twitter.

The second framework is something I’ve written about a fair bit – on finite and infinite games. This was introduced by the late NYU philosopher James Carse. The basic concept is that the objective of a finite game is to win. There is a particular end point. In an infinite game, there is no concept of “winning”. The objective is to just continue playing. I think it’s a rather profound theory, and has consequences in lots of facets of life.

Including media. My argument is that periodicals play a finite game and perennials play an infinite game.

The objective of a periodical is to make each issue good enough that the reader/viewer continues the subscription until the next issue. This might, at face value, appear like an infinite game, but from the point of view of a single edition, it is a finite game. If the reader/viewer continues subscription (however you define it) till the next issue, you have “won”.

It is different with perennials because there is no discrete “next edition”. The next edition of a next edition is the next minute. And that makes the “game” mentioned in the earlier paragraph hard to play. Instead, running a perennial media house is like playing an infinite game, where your objective is to make sure that the viewer/reader “continues to play the game”, or continues to watch without switching channels or diverting attention.

In other words, the objective of a perennial media house (like a 24×7 media channel, or twitter) is to make sure users stay on the platform. Which is good.

Except that, over a period of time, some of these media houses have figured out that one surefire way of retaining viewership and viewer interest is by stoking viewer anxiety. When a viewer is anxious about something, they want to get as much information as possible about the thing they are anxious about, and continue to hunt for information. This means that they are going to continue to hang around the channel (or social media platform) in the hope of resolving their anxieties. Which means that these channels or platforms “win” the infinite game of retaining audience attention.

And how do these channels create anxiety? By creating outrage. By creating sensationalism. By resorting to fake news, of the kind that is certain to cause anxiety among viewers, in the hope that they will continue to watch (and consume the intervening ads).

I clearly remember the Kaveri riots in Bangalore in 2016 (the week my daughter was born), when Kannada 24×7 news channels took to showing the riots and arson live on TV. And giving reports in a rather sensational voice on how the riots were only going to increase and things are going to get worse. This wasn’t “fake” per se, but sensational and anxiety causing (we kept the TV on one whole afternoon wondering if it was safe to go to the obstetrician’s clinic (300m away from home) ).

And the Kannada 24×7 channels were at it again in 2020 during the covid-19 induced lockdown. One day (in May) suddenly one of them claimed that “all of Bangalore would get sealed down because of increasing cases”. It turned out that two small neighbourhoods were “sealed down” because of a high density of cases there. The rumours of “seal down” were clearly fake news, that clearly created anxiety among the viewers.

I’m only quoting one such instance from this period, but news channels kept at this business of fostering anxiety by saying things that weren’t true (I don’t normally watch these channels, but kept getting informed about these fake “news” by elderly relatives who as a rule keep watching news all the time).

What I’m disappointed by is that this kind of fake news gets no attention at all, compared to the more political sort of fake news which is easy to see through for someone with an iota of brain cells. Then again, the platforms that give footage to the ills of political fake news (twitter, some whatsapp groups, etc) are also perennial news sources themselves and so it doesn’t make sense to call out people of their own ilk.

News and the Cornish coastline

Following news at more frequent intervals means there is more negative news, and thus a greater chance of getting triggered.

How much news exists in the world? Is there enough news to fit a daily newspaper? Is there enough news to fit a daily that is focussed on a city? Is there enough news for All India Radio to cover in its three (?) news broadcasts a day? And what about 24/7 news channels? Do they have enough news?

The answer to this question is simple – within reasonable limits, irrespective of how frequently you want to report the news, there will be in some way or the other sufficient news to report.

In some ways, this is like that famous question in Chaos Theory about the length of the Cornish coast. The answer is – it depends upon the ruler.

The length of the famously jagged Cornish coast depends upon the length of the ruler you use to measure it. The smaller the length of the ruler that you use, the more the indentations in the coastline matter, and thus the longer the coast. There is a limit there, of course, if I remember, the “fractal dimension” of the Cornish coast is 1.33 or something.

It is similar with news. The amount of news that is there to report is a function of how frequently you want to report it. A good analogy here is with the stock markets.

As regular readers of this blog might well know, stock price movement, upto some approximation, follow a random walk. This means that the “distance” covered by the stock ticker during the day is far higher than the “displacement”.

So a stock might gain 10% in the first minute of trading, and then lose 5% in the next hour, and then lose all its gains by lunchtime, and then go up and down and round and round and then end up pretty much where it started off at the beginning of the day.

If you are now reporting the news of this stock market with a “one day ruler” (say for your business daily), the market did nothing that day. However, if you have been watching its movement on CNBC (or any other real time news channel), there was a lot to report.

All news is this way. When you follow it at frequent intervals (through 24/7 TV, or through Twitter, for example), there appears to be a lot more news than there actually is when you follow it using a daily newspaper. And given that any piece of negative news is likely to cause anxiety, following news at more frequent intervals exposes you to far more negative news (think the stock market example again), and thus causes far more anxiety.

Not following the news at all (as I did for a while when I was in undergrad) sometimes means that you’ve missed out on all that has happened in the world, and might find it hard to cope with life. And so there is a tradeoff. This involves using a (time) ruler of an appropriate length.

Use too short a ruler and you cause yourself unnecessary anxiety and find yourself getting triggered all the time. Use too long a ruler and you find that you miss out on stuff that might have been necessary for you to know.

The frequency I’ve settled down upon is daily. I get three newspapers delivered to my door each morning, and that is how I’m informed about the world.

It’s interesting that back when the New York Times was a dead-tree periodical, it had a tagline that went “all the news that’s fit to print”. Now that it’s gone online, got a paywall and had to get into real time news, it’s become an outrage machine.

I wake up early on weekdays nowadays, so go the first two hours of the day without really knowing what is happening in the world. As you might know, I’m on a social media break, so that source of news is cut off. And it’s only around 7 am by when a copy of the Business Standard gets delivered to my door.

Until last month, a copy of the Deccan Herald would arrive at home as well, but I stopped it after I found it to be largely useless. A lot of stories in that newspaper were written as they might have been 20 or 30 years ago. There was little distinction between reporting and analysis and opinion. A lot of news couldn’t be simply consumed without the accompanying (and sometimes patronising) opinion.

The Business Standard, which I started reading in 2005, is still a very good paper. The editorials continue to be first-rate (though their quality had dipped in the 2011-14 period). The analysis pieces and columns cover a variety of topics that simply don’t make it to social media (since they aren’t really “sensational”). And the newspaper is “crisp” and quickly tells you what’s going on in India.

For two years, when I lived in London, I lived without a daily newspaper, and it was a struggle. Online newspapers have simply not been able to provide the same kind of product as offline newspapers. And the reason is that online newspapers are “flat” – all the contextualising and prioritising that a dead-tree paper can do is completely absent in the online version.

In a dead-tree newspaper, you know how important a piece of news is based on the page it appears, the size of the headline, the size of the column and so on. Based on where it appears, you know if it is news or analysis or opinion. In case it is opinion, you can easily see who has written it before you “click through” (start reading it). You can easily how big a piece is (and how much of your time it will take) before deciding to invest time in it.

All this is absent from an online newspaper. Check out, for example, the homepage of the Business Standard, that I so fulsomely praised earlier in this post.

It is impossible to know what’s the important stuff here. If I have only five minutes to read, I don’t know what to focus on. I don’t know which of this is opinion and which is news. Before I click through, I don’t know how big a piece is or who has written it or if it has been syndicated.

Unless the link has come from a qualified source (such as Twitter) I don’t know much about it, and so don’t know how to consume it. This might explain to you why a lot of online news sources are losing revenues to the likes of Google or Facebook – the latter do the important job of putting the news in context!

Finally, I’m glad I now consume news only once a day (from the physical paper). Sometimes, what is news intra-day would have ceased to be news by nightfall. So when you consume news at a reasonable interval (such as daily), what comes to you is “qualified” real stuff. A piece of news should have been important enough for a day to make it to the next day’s newspapers. And once a day is also a reasonable interval to get to know of what is happening in the world.

Algorithmic curation

When I got my first smartphone (a Samsung Galaxy Note 2) in 2013, one of the first apps I installed on it was Flipboard. I’d seen the app while checking out some phones at either the Apple or Samsung retail outlets close to my home, and it seemed like a rather interesting idea.

For a long time, Flipboard was my go-to app to check the day’s news, as it conveniently categorised news into “tech”, “business” and “sport” and learnt about my preferences and fed me stuff I wanted. And then after some update, it suddenly stopped working – somehow it started serving too much stuff I didn’t want to read about, and when I tuned (by “following” and “unfollowing” topics) my feed, it progressively got worse.

I stopped using it some 2 years back, but out of curiosity started using it again recently. While it did throw up some nice articles, there is too much unwanted stuff in the app. More precisely, there’s a lot of “clickbaity” stuff (“10 things about Narendra Modi you would never want to know” and the like) in my feed, meaning I have to wade through a lot of such articles to find the occasional good ones.

(Aside: I dedicate about half a chapter to this phenomenon in my book. The technical term is “congestion”. I talk about it in the context of markets in relationships and real estate)

Flipboard is not the only one. I use this app called Pocket to bookmark long articles and read later. A couple of years back, Pocket started giving “recommendations” based on what I’d read and liked. Initially it was good, and mostly curated from what my “friends” on Pocket recommended. Now, increasingly I’m getting clickbaity stuff again.

I stopped using Facebook a long time before they recently redesigned their newsfeed (to give more weight to friends’ stuff than third party news), but I suspect that one of the reasons they made the change was the same – the feed was getting overwhelmed with clickbaity stuff, which people liked but didn’t really read.

Basically, there seems to be a widespread problem in a lot of automatically curated news feeds. To put it another way, the clickbaity websites seem to have done too well in terms of gaming whatever algorithms the likes of Facebook, Flipboard and Pocket use to build their automated recommendations.

And more worryingly, with all these curators starting to do badly around the same time (ok this is my empirical observation. Given few data points I might be wrong), it suggests that all automated curation algorithms use a very similar algorithm! And that can’t be a good thing.

The Economist and the Murray Gell-Mann Amnesia effect

I’ve been a subscriber to the Economist for the last couple of years and quite enjoy reading that newspaper. There are weeks when I don’t manage to go through the week’s edition, but there are certain weeks when it forms a large part of my reading. I quite like the paper, and I subscribe to the daily “Espresso” issue on my mobile phone.

I have only one problem – a lot of their writing about India is biased, and filled with Sominisms. I mean their business pieces are pretty good, like this profile of Welspun. But their political coverage is generally biased by their correspondents’ dislike for Prime Minister Narendra Modi, and hence not particularly objective.

I had recently carried a copy of the newspaper to read on my way to a quiz, where I met Baada. Baada was surprised that I read the newspaper, since it was so obviously biased in its coverage of Indian politics. “I agree that India coverage is biased”, I said, “but its worldwide coverage is really good. Hence I read it”.

Recently I came across the “Murray Gell-mann Amnesia effect“. It is named after the physicist, and claims that you will trust the rest of a newspaper even though you know that its coverage of your domain is shit. And thinking about it, I’m wondering if I should continue trusting The Economist.

Currently, I believe that the Economist’s coverage of Indian politics is shit, but I continue to read the newspaper for its other coverage. But what if everyone believes that the Economist’s coverage of their domain is inadequate? If that is the case, does it still make it a good paper? Should I use the fact of the Economist’s coverage of Indian business being better as a mitigating factor?

The problem is that there is no other paper that gives a nice concise view of what is happening in the world (FT is too voluminous given its frequency), and that makes the Economist good. But if their coverage is biased by their correspondents’ views in every country, it is not that trustworthy any more.

The only option I can think of is to continue reading the newspaper, but to “add salt to taste”. Every time I read a political story set in some country, I should keep in mind that the correspondent might be biased, and adjust my views accordingly. That way, I can consume the paper’s curation and analysis while not getting influenced by its inherent biases.