Known stories and trading time

One of the most fascinating concepts I’ve ever come across is that of “trading time”. I first came across it in Benoit Mandelbrot’s The (Mis)Behaviour of Markets, which is possibly the only non-textbook and non-children’s book that I’ve read at least four times.

The concept of “trading time” is simple – if you look at activity on a market, it is not distributed evenly over time. There are times when nothing happens, and then there are times when “everything happens”. For example, 2020 has been an incredibly eventful year when it comes to world events. Not every year is eventful like this.

A year or so after I first read this book, I took a job where I had to look at intra-day trading in American equities markets. And I saw “trading time” happening in person – the volume of trade in the market was massive in the first and last hour, and the middle part of the day, unless there was some event happening, was rather quiet.

Trading time applies in a lot of other contexts as well. In some movies, a lot of action happens in certain times of the movie where nothing happens in other times. When I work, I end up doing a lot of work in some small windows, and nothing most of the time. Children have “growth spurts”, both physical and mental.

I was thinking about this topic when I was reading SL Bhyrappa’s Parva. Unfortunately I find it time-consuming to read more than a newspaper headline or signboard of Kannada, so I read it in translation.

However, the book is so good that I have resolved to read the original (how much ever time it takes) before the end of this year.

It is a sort of retelling of the Mahabharata, but it doesn’t tell the whole story in a linear manner. The book is structured largely around a set of monologues, largely set around journeys. So there is Bhima going into the forest to seek out his son Ghatotkacha to help him in the great war. Around the same time, Arjuna goes to Dwaraka. Just before the war begins, Bhishma goes out in search of Vyasa. Each of these journeys associated with extra long flashbacks, and philosophical musings.

In other words, what Bhyrappa does is to seek out tiny stories within the great epic, and then drill down massively into those stories. Some of these journey-monologues run into nearly a hundred pages (in translation). The rest of the story is largely glossed over or given only a passing mention to.

Bhyrappa basically gives “trading time treatment” to the Mahabharata. It helps that the overall story is rather well known, so readers can be expected to easily fill in any gaps. While the epic itself is great, there are parts where “a lot happens”, and parts where “nothing happens”. What is interesting about Parva is that Bhyrappa picks out unintuitive parts to explore in massive depth, and he simply glosses over the parts which most other retellings give a lot of footage to.

And this is what makes the story rather fascinating.

I can now think of retellings of books, or remakes of movies, where the story remains the same, but “trading time is inverted”. Activities that were originally given a lot of footage get glossed over, but those that were originally ignored get explored in depth.


Book Recommendations for Children

On Saturday, the daughter and I went book-shopping to Blossom, and came back with a bunch of books that the wife described as “mostly useless”. I put it down to my lack of judgment on what is a good children’s book.

That is a serious issue – how do you really know what is a good children’s book? And what is a book that is appropriate for the child’s age? I tried the usual things like googling for “best books for three year olds”, but the intersection of those lists and what was there at Blossom wasn’t great.

For starters – we’ve got the basics . Eric Carle’s The Very Hungry Caterpillar. Julia Donaldson’s The Gruffalo. Judith Kerr’s The Tiger Who Came To Tea. A bunch of brilliant books the wife picked up at a bookstore in Oxford which were recommended by a kindly lady she bumped into at the store who has kids older than ours.

However, in the interest of getting the daughter to handle books more (she can’t read yet, just about learning the letters (or “sounds” as she calls them) ), we want to get more books. And it was with this noble intention that we ended up at Blossom (which is where I go to for my physical books) on Saturday.

I tried a couple of heuristics. One was to buy more books from authors you have read and liked. Julia Donaldson, for example, is rather prolific, as is Eric Carle. One book by each was part of the “useless bunch” that we got on Saturday.

The other heuristic I followed was to seat the child on a chair, and then pick out books one by one from the shelf and see which one she got more interested in. And then ask her if she wanted the book, and let her decide what she wants (we ended up with more “useless books” this way).

For my own physical book shopping nowadays, I rely on Goodreads. I got this idea from Whaatra Woreshtmax, whom I’d accompanied to Bookworm (down the road from Blossom) a few months back. He walked around the store with his Goodreads app open, scanning the barcodes in the app and checking for ratings. Anything with an average rating over 4.15 went into his basket (he reads prolifically so he can be more liberal with his choices).

I don’t scan barcodes, and I check on Goodreads only if I have an initial sense of whether the book is going to be of my liking. And since I understand my preferences may not match “the crowd”‘s, I have a lower cutoff – incidentally set at 3.96 which happens to be the current average rating of my book on Goodreads.

Now I don’t know if people rate children’s books on Goodreads the same way as they do adults’, and if I should rely on them. The number of factors that affect whether a book is good or not for children is much longer (I think) than for adults’ books.

So what heuristics do you follow to buy books for your children? Let the children decide? Go for known authors? Goodreads? Anything else?

Big Data and Fast Frugal Trees

In his excellent podcast episode with EconTalk’s Russ Roberts, psychologist Gerd Gigerenzer introduces the concept of “fast and frugal trees“. When someone needs to make decisions quickly, Gigerenzer says, they don’t take into account a large number of factors, but instead rely on a small set of thumb rules.

The podcast itself is based on Gigerenzer’s 2009 book Gut Feelings. Based on how awesome the podcast was, I read the book, but found that it didn’t offer too much more than what the podcast itself had to offer.

Coming back to fast and frugal trees..

In recent times, ever since “big data” became a “thing” in the early 2010s, it is popular for companies to tout the complexity of their decision algorithms, and machine learning systems. An easy way for companies to display this complexity is to talk about the number of variables they take into account while making a decision.

For example, you can have “fin-tech” lenders who claim to use “thousands of data points” on their prospective customers’ histories to determine whether to give out a loan. A similar number of data points is used to evaluate resumes and determine if a candidate should be called for an interview.

With cheap data storage and compute power, it has become rather fashionable to “use all the data available” and build complex machine learning models (which aren’t that complex to build) for decisions that were earlier made by humans. The problem with this is that this can sometimes result in over-fitting (system learning something that it shouldn’t be learning) which can lead to disastrous predictive power.

In his podcast, Gigerenzer talks about fast and frugal trees, and says that humans in general don’t use too many data points to make their decisions. Instead, for each decision, they build a quick “fast and frugal tree” and make their decision based on their gut feelings about a small number of data points. What data points to use is determined primarily based on their experience (not cow-like experience), and can vary by person and situation.

The advantage of fast and frugal trees is that the model is simple, and so has little scope for overfitting. Moreover, as the name describes, the decision process is rather “fast”, and you don’t have to collect all possible data points before you make a decision. The problem with productionising the fast and frugal tree, however, is that each user’s decision making process is different, and about how we can learn that decision making process to make the most optimal decisions at a personalised level.

How you can learn someone’s decision-making process (when you’ve assumed it’s a fast and frugal tree) is not trivial, but if you can figure it out, then you can build significantly superior recommender systems.

If you’re Netflix, for example, you might figure that someone makes their movie choices based only on age of movie and its IMDB score. So their screen is customised to show just these two parameters. Someone else might be making their decisions based on who the lead actors are, and they need to be shown that information along with the recommendations.

Another book I read recently was Todd Rose’s The End of Average. The book makes the powerful point that nobody really is average, especially when you’re looking a large number of dimensions, so designing for average means you’re designing for nobody.

I imagine that is one reason why a lot of recommender systems (Netflix or Amazon or Tinder) fail is that they model for the average, building one massive machine learning system, rather than learning each person’s fast and frugal tree.

The latter isn’t easy, but if it can be done, it can result in a significantly superior user experience!


Over the last 4-5 days I kinda immersed myself in finishing Rory Sutherland’s excellent book Alchemy.

It all started with a podcast, with Sutherland being the guest on Russ Roberts’ EconTalk last week. I’d barely listened to half the podcast when I knew that I wanted more of Sutherland, and so immediately bought the book on Kindle. The same evening, I finished my previous book and started reading this.

Sometimes I get a bit concerned that I’m agreeing with an author too much. What made this book “interesting” is that Sutherland is an ad-man and a marketer, and keeps talking down on data and economics, and plays up intuition and “feeling”. In other words, at least as far as professional career and leanings go, he is possibly as far from me as it gets. Yet, I found myself silently nodding in agreement as I went through the book.

If I have to summarise the book in one line I would say, “most decisions are made intuitively or based on feeling. Data and logic are mainly used to rationalise decisions rather than making them”.

And if you think about it, it’s mostly true. For example, you don’t use physics to calculate how much to press down on your car accelerator while driving – you do it essentially by trial and error and using your intuition to gauge the feedback. Similarly, a ball player doesn’t need to know any kinematics or projectile motion to know how to throw or hit or catch a ball.

The other thing that Sutherland repeatedly alludes to is that we tend to try and optimise things that are easy to measure or optimise. Financials are a good example of that. This decade, with the “big data revolution” being followed by the rise of “data science”, the amount of data available to make decisions has been endless, meaning that more and more decisions are being made using data.

The trouble, of course, is availability bias, or what I call as the “keys-under-lamppost bias”. We tend to optimise and make decisions on things that are easily measurable (this set of course is now much larger than it was a decade ago), and now that we know we are making use of more objective stuff, we have irrational confidence in our decisions.

Sutherland talks about barbell strategies, ergodicity, why big data leads to bullshit, why it is important to look for solutions beyond the scope of the immediate domain and the Dunning-Kruger effect. He makes statements such as “I would rather run a business with no mathematicians than with second-rate mathematicians“, which exactly mirrors my opinion of the “data science industry”.

There is absolutely no doubt why I liked the book.

Thinking again, while I said that professionally Sutherland seems as far from me as possible, it’s possibly not so true. While I do use a fair bit of data and economic analysis as part of my consulting work, I find that I make most of my decisions finally on intuition. Data is there to guide me, but the decision-making is always an intuitive process.

In late 2017, when I briefly worked in an ill-fated job in “data science”, I’d made a document about the benefits of combining data analysis with human insight. And if I think about my work, my least favourite work is where I’ve done work with data to help clients make “logical decision” (as Sutherland puts it).

The work I’ve enjoyed the most has been where I’ve used the data and presented it in ways in which my clients and I have noticed patterns, rationalised them and then taken a (intuitive) leap of faith into what the right course of action may be.

And this also means that over time I’ve been moving away from work that involves building models (the output is too “precise” to interest me), and take on more “strategic” stuff where there is a fair amount of intuition riding on top of the data.

Back to the book, I’m so impressed with it that in case I was still living in London, I would have pestered Sutherland to meet me, and then tried to convince him to let me work for him. Even if at the top level it seems like his work and mine are diametrically opposite..

I leave you with my highlights and notes from the book, and this tweet.

Here’s my book, in case you are interested.


Serials and movies

Yesterday I finished reading Gita Krishnankutty’s English translation of MT Vasudevan Nair’s Randamoozham. It’s the story of the Mahabharata told from Bhima’s perspective.

This wasn’t the first time that I was reading a translation of this magnificent book. A few years ago, journalist Prem Panicker had created a series on his blog where he would put up translations of bits of this book daily. I remember quite liking that, and a lot of people raving about it.

Prem’s version of the book was far longer than the version that I finished yesterday (Gita Krishnankutty’s version is 380 pages long, which comes to around 70000 words or less. Prem’s is 120,000 words long). It was also far more passionate. Rather than directly translating the novel, Prem took liberties in adding his own inputs.

It’s been over a decade since I read Prem’s version, but from what I remember, the parts of the story where Bhima mourns Ghatotkacha’s death, for example, are far more well sketched out in that version. It is similar with the parts which show Bhima’s frustration with Yudhishthira’s leadership.

Thinking about it, though, one reason why Prem was able to go into such detail was that he presented his book in a serialised format. Every day he would put out the translation of a few pages’ worth of a book, and the translation would come out to be the length of a long form article (the kind of articles that Prem became a specialist in writing during his time at Rediff).

When you’re reading it in book form, in which you read the whole thing together, reading in such detail may not work so well since that might make the book unnecessarily thick, and people might put NED midway. Give the inputs in small doses, however, and people will be happy to consume the greater detail. In that sense, Prem’s and Gita Krishnankutty’s translations are both excellent, and both very well suited for the formats they came out in.

It is a similar story with movies and serials. Movies have a 2-2.5 hour length because that’s how much typically people can consume at a time without putting NED. Serials, on the other hand, because they are consumed bit by bit at a time, can go much longer in aggregate (sometimes unnecessarily long).

Netflix releasing all episodes of a series at the same time, however, is changing this dynamic. Sacred Games apart, I’ve been unable to get through any Netflix fiction series because of their sheer length. Because binge-watching has become a thing (thanks to Netflix putting out an entire season at once), the entire season comes to resemble a movie. So a season with 8 one-hour episodes effectively becomes a 8-hour movie. And unless it’s extremely well made, or has sufficient stuff going on through the 8 hours, it becomes incredibly hard to sit through!


Dreamers and Dignity

If I’d picked up Snigdha Poonam’s Dreamers before I had read Chris Arnade’s Dignity, I might have liked it better. As it happened, having read Dignity, I found Dreamers to be unnecessarily judgmental and prescriptive, and was unable to read it beyond the first two chapters. It is now there on my goodreads page, as a book that I “finished” and gave one star.

Dignity is a book I highly recommend. Chris Arnade, a former investment banker with a PhD in astrophysics wanders around and hangs around in what he calls as “back row America”, and chronicles people’s lives there. The entire book is simply a set of chronicles, garnished with beautiful photos he has taken of his interviewees.

While Arnade makes no secrets of his own political leaning, he doesn’t let that affect his book. Rather, he keeps his own politics to the minimum and lets his interviews do the talking, literally. There are no policy prescriptions in the book, and the reader is simply presented a set of lives and asked to draw her own conclusions. And that means that even if you don’t agree with the politics of the author (I certainly don’t), the book is an incredibly compelling read.

I picked up Snigdha Poonam’s Dreamers about a month or so after I’d finished Dignity. The premise is sort of similar – except that given that India has recently had far higher growth than the US, the “back row Indians” can be classified as “dreamers” who are seeking a better life. And in this book, Poonam chronicles the stories of some of these dreamers, and what they are doing to get themselves a better life.

Poonam is clear about her politics as well (“my family has always voted for the Congress Party”), but what makes her book different from Arnade’s is that she lets her politics take over her narrative. While telling the story of Moin Khan, who runs a spoken English class in Ranchi, she doesn’t hesitate to make snide remarks about either the teacher or any of his students.

Rather than letting her characters talk, Poonam talks on their behalf and overlays her politics to pretty much everything she is talking about. “This is how you are expected to get ahead in Modi’s India” is a refrain through the book.

And even leaving the politics aside, what made me uncomfortable with Dreamers is that the author seems to talk down to the interviewees. The tone throughout the parts of the book that I read is one of moral superiority and smugness of being part of “front row India”.

Maybe if I had read Dreamers before I read Dignity, I would have appreciated it for what it is, and for the stories that it told. I might have discarded the politics and the tone and just enjoyed the stories (I see the book has got 4 stars on Goodreads from a lot of my friends).

Having read Dignity, however, I perhaps had this image in my head of how these stories can be told well. And that meant that I was simply unable to look beyond the overt politics and smug tone in Dreamers. And that meant I abandoned it midway, and gave it a low rating.

Margaret Atwood doesn’t escape my fate

My book released exactly two years ago (if you haven’t read it yet, you can buy it here). Rather, it was supposed to release two years ago, on 6th of September 2017. As it happened, people who had pre-ordered the book got deliveries a few days early. Amazon had messed up with the release date.

I remember getting in touch with Amazon Customer Care. They didn’t seem to care. I spoke to friends and relatives who worked there, and they suggested a “Jeff B escalation” (an email sent to Jeff Bezos – apparently he reads them). There was no response to that either. And so my book came out in a trickle, being sent to people as they ordered them, rather than with a bang.

I’m possibly feeling a sense of schadenfreude that it’s not just first-time authors like me who got screwed over like this by Amazon in terms of early release of the book. I am in illustrious company – Canadian author Margaret Atwood suffered the same fate this week.

Amazon, the biggest book vendor in the United States, recently started shipping preorders of Margaret Atwood’s book Testaments. The problem, notably, is that Atwood’s book is not supposed to launch until Tuesday, September 10. Amazon is violating the embargo that all sellers of the book have agreed to. And its indie bookselling rivals are pissed.

In my case, Amazon had exclusive sales on the book – thanks to using a small first-time publisher, we didn’t have the network to go wider and get the book into more stores. In that sense, apart from me, there was possibly nobody pissed off at the early release of the book.

Then again, this early release of pre-ordered books was an endemic problem to Amazon, and a high-profile leak such as this one was bound to happen some time or the other. Hopefully this will lead to the retailer to put enough measures in place to prevent this kind of thing from happening again (mainstream publishers have strong relationships with bookshops, so they are likely to put pressure on Amazon).

In any case, I’m glad to have such good company!

PS: If you haven’t listened to Atwood’s conversation with Tyler Cowen, you should do so soon. It’s fantastic (and I say this as someone who hasn’t read any of her works)

Gamification and finite and infinite games

Ok here I’m integrating a few concepts that I learnt via Venkatesh Guru Rao. The first is that of Finite and Infinite games, a classic if hard to read book written by philosopher James Carse (which I initially discovered thanks to his Breaking Smart Season 1 compilation). The second is of “playflow”, which again I discovered through a recent edition of his newsletter.

A lot of companies try to “gamify” the experiences for their employees in order to make work more fun, and to possibly make them more efficient.

For example, sales organisations offer complicated incentives (one of my historically favourite work assignments has been to help a large client optimise these incentives). These incentives are offered at multiple “slabs”, and used to drive multiple objectives (customer acquisition, retention, cross-sell, etc.). And by offering employees incentives for achieving some combination of these objectives, the experience is being “gamified”. It’s like the employee is gaining points by achieving each of these objectives, and the points together lead to some “reward”.

This is just one example. There are several other ways in which organisations try to gamify the experience for their employees. All of them involve some sort of award of “points” for things that people do, and then a combination of points leading to some “reward”.

The problem with gamification is that the games organisations design are usually finite games. “Sell 10 more widgets in the next month”. “Limit your emails to a maximum of 200 words in the next fifteen days”. “Visit at least one client each day”. And so on.

Running an organisation, however, is an infinite game. At the basic level, the objective of an organisation is to remain a going concern, and keep on running. Growth and dividends and shareholder returns are secondary to that – if the organisation is not a going concern, none of that matters.

And there is the contradiction – the organisation is fundamentally playing an infinite game. The employees, thanks to the gamified experience, are playing finite games. And they aren’t always compatible.

Of course, there are situations where finite games can be designed in a way that their objectives align with the objectives of the overarching infinite game. This, however, is not always possible. Hence, gamification is not always a good strategy for organisations.

Organisations have figured out the solution to this, of course. There is a simple way to make employees play the same infinite game as the organisation – by offering employees equity in the company. Except that employees have the option of converting that to a finite game by selling the said equity.

Whoever said incentive alignment is an easy task..


Marginalised communities and success

Yesterday I was listening to this podcast where Tyler Cowen interviews Neal Stephenson, who is perhaps the only Science Fiction author whose books I’ve read. Cowen talks about the characters in Stephenson’s The Baroque Cycle, a masterful 3000-page work which I polished off in a month in 2014.

The key part of the conversation for me is this:

COWEN: Given your focus on the Puritans and the Baroque Cycle, do you think Christianity was a fundamental driver of the Industrial Revolution and the Scientific Revolution, and that’s why it occurred in northwestern Europe? Or not?

STEPHENSON: One of the things that comes up in the books you’re talking about is the existence of a certain kind of out-communities that were weirdly overrepresented among people who created new economic systems, opened up new trade routes, and so on.

I’m talking about Huguenots, who were the Protestants in France who suffered a lot of oppression. I’m talking about the Puritans in England, who were not part of the established church and so also came in for a lot of oppression. Armenians, Jews, Parsis, various other minority communities that, precisely because of their outsider minority status, were forced to form long-range networks and go about things in an unconventional, innovative way.

So when we think about communities such as Jews or Parsis, and think about their outsized contribution to business or culture, it is this point that Stephenson makes that we should keep in mind. Because Jews and Parsis and Armenians were outsiders, they were “forced to form long-range networks”.

In most cases, for most people of these communities, these long-range networks and unconventional way of doing things didn’t pay off, and they ended up being worse off compared to comparable people from the majority communities in wherever they lived.

However, in the few cases where these long-range networks and innovative ways of doing things succeeded, they succeeded spectacularly. And these incidents are cases that we have in mind when we think about the spectacular success or outsized contributions of these communities.

Another way to think of this is – denied “normal life”, people from marginalised communities were forced to take on much more risk in life. The expected value of this risk might have been negative, but this higher risk meant that these communities had a much better “upper tail” than the majority communities that suppressed and oppressed them.

Given that in terms of long-term contributions and impact and public visibility it is only the tails of the distribution that matter (mediocrity doesn’t make news), we think of these communities as having been extraordinary, and wonder if they have “better genes” and so on.

It’s a simple case of risk, and oppression. This, of course, is no justification for oppressing swathes of people and forcing them to take more risks than necessary. People need to decide on their own risk preferences.

Gruffaloes and Finite Games

One story that my daughter knows well, rather too well, is the story of the Gruffalo. This is a story of a mouse told in two parts.

In the first part, the mouse fools a fox, an owl and a snake from eating him by convincing them that he’s having lunch, tea and dinner respectively with a supposedly imaginary creature named “Gruffalo”. And when they each ask him what the Gruffalo is like, he makes up stuff fantastically (terrible teeth in terrible jaws, turned out paws, etc.).

Except that midway through the story there is a kahaani mein twist, and the mouse actually encounters the gruffalo. In the second part of the story, the mouse tells the gruffalo that he is going to have lunch, tea and dinner with the fox, owl and snake, and prevents the gruffalo from eating him. And the mouse lives another day.

It is evidently a nice story, and the rhyme means that the daughter had mugged up the entire story enough when she was barely two years old that she could “read” it when shown the book (she can’t read a word yet). However, I don’t like it because I don’t like the plot.

One of the most influential books I’ve read is James Carse’s Finite and Infinite Games. Finite Games are artificial games where we play to “win”. There is a defined finish, and there is a set of tasks that we need to achieve that constitutes “victory”. Most real-life games are on the other hand are “infinite games” where the objective is to simply ensure that the game simply goes on.

From the point of stories, the best stories are ones which represent finite games, where there is a clear objective, and the story ends in “victory” or “lack of victory” (in the case of a tragedy). The Good, The Bad and the Ugly has the finite aim of finding the treasure buried in the graveyard. Ganeshana Maduve has the finite aim of YG Rao marrying “Shruti”. Gangs of Wasseypur has the finite aim of the Khan family taking revenge on Ramadhir Singh. Odyssey has the finite aim of Odysseus returning home to Penelope. And so forth.

Putting it another way, finite games make for nice stories, since stories are themselves finite, with a beginning and an end. A story that represents an infinite game is necessarily left incomplete, and you don’t know what happens just outside the slice of action that the story covers. So infinite games, which is how life is lived, make for lousy stories.

And the gruffalo story is an infinite game, since the “game” that the mouse is playing in the story is survival – by definition an infinite game. There is no “victory” by being alive at the end of the day the story covers – like there is no she-mouse to marry, or a baby mouse to see for the first time, or a party to go to. It is just another day in the life of the mouse, and the events of the day are unlikely to be that much more spectacular than the days not covered by the story.

That is what makes the gruffalo story so unsatisfying. Yes, the mouse played off the fox, owl and snake against the gruffalo to ensure his survival, but what about the next day? Would he have to invent another creature to ensure his survival? Would the predators buy the same story another time?

I don’t know, and so the story rings hollow. But the rhyme is good, and so my daughter loves the story!