Making Zero Rating work without disruption

The Net Neutrality debate in India has seen a large number of analogies being raised, in order to help people understand and frame the debate. Internet services have been variously compared to television, postal services, highways, markets and what not. Things got so bad that that at some point in time people had to collectively denounce all analogies, for they were simply taking away from the debate.

One of the analogies that were being drawn in an argument in favour of Zero Rating was to compare it to e-commerce companies that provide free shipping, for example, or the deep discounts provided by services such as Uber or Ola. If you ban zero rating, other legitimate activities such as free shipping will be next, critics of net neutrality argued, arguing that there would be no end to this. The counter-argument is that free shipping doesn’t disrupt the basic structure of the market while zero rating does. Here is a way in which zero rating can be made to work without disrupting the market.

And it is a rather simple one – cash transfers. Rather than an e-commerce company subsidising your browsing of their website directly (by paying the telecom provider to make your access free), they can instead refund your costs of browsing their sites in terms of a discount. Going back into the analogy space, this is similar to malls that charge you heavily for parking but then offset your parking fees against any purchase you make in the mall.

So Flipkart, for example, can estimate the amount of bandwidth a particular user would have spent in browsing their app (not hard to track at all, especially if the user uses the app), and any purchase on their site can be appropriately discounted to that extent (and maybe a little more to cover for browsing that didn’t lead to a purchase).

This works in several ways. In the current proposed model of Zero Rating, the e-commerce company doesn’t know how many users will access it, using each ISP, so there is uncertainty in the amount that they have to pay the ISPs for such access. By moving to a user-wise subsidy model, they know exactly what users are using how much, and this enables them to target the subsidies much better. Another way in which it helps the retailer is that it doesn’t waste money spending on bandwidth for people who only browse the website without buying (of course, if they wish to, they can subsidise such usage also, but since it can be so obviously gamed, they won’t do it).

More importantly, what such a system ensures is that the internet is not broken. You might recall my earlier post on this topic that zero rating results in “walled gardens” that leads to a broken internet which reduces the overall value of the internet. With a cash transfer scheme (rather than direct subsidy), such distortions are avoided, and the internet remains “free” (of any barriers, not free of cost) and maximum value of the internetwork is realised.

So as described above it is well possible for e-commerce players to subsidise users’ browsing of their apps without distorting the internet, and without using zero rating. And as shown above, doing so is in their interest.

PS: This post also came out of the same discussions from which my earlier post on 2ab had come out.

How Long Tail affects pricing

My late mother never shopped for fruits and vegetables in the Gandhi Bazaar market. She found that the market was in general consistently overpriced, and if we look at the items that she would buy, it is still the case. For “normal” stuff, you are better off going to nearby “downmarkets” like the one at NR Colony, or even Jayanagar Fourth Block.

So why is the Gandhi Bazaar market overpriced? The answer lies in the long tail. In the book of the same name, Chris Anderson talks about products that are not the most popular, but which has a niche demand. In that he talks about companies such as Amazon or Netflix which are successful not because they do a better job of selling the “bestsellers” but because they are able to service well the “long tail” – items that are not found elsewhere thanks to the high cost of selling.

In other words, it is a liquidity story. If the neighbourhood kirana, for example, wants to sell olives, his costs are going to be high as the rate at which he sells olive bottles is going to be so low that his inventory costs are going to increase, and the risks of ageing and spoilage of inventory also goes up. And he has to spend that much more manpower and effort in managing this extra item, so he decides to not sell this item at all (he will have to charge such a high premium to sell such goods that it doesn’t make sense for the customer to buy it).

Yesterday I bought an “imam pasand” mango in Gandhi Bazaar. Now, this is not one of the “standard” mango varieties that are available in Bangalore. In fact, I had never in my life eaten this variety of mango until yesterday, for the simple reason that it is not generally available in Bangalore. The fruit stall in Gandhi Bazaar, however, stocked it. A neighbouring fruit stall was where I used to source the Dashehri mangoes (common in North India but rare in Bangalore) a couple of mango seasons back. Avocados, which are generally hard to find in “traditional” retailers in Bangalore were also available in every fruit stall in Gandhi Bazaar, as were other not-so-common fruits.

So why did my mother find Gandhi Bazaar expensive? The answer is that the fruit sellers at Gandhi Bazaar stock the “long tail” because of which their general costs of inventory are high compared to competitors who don’t. Thanks to the range, they will have a large number of customers who come to them to buy specifically these “long tail” items. And while they are at it (buying the long tail items), they also end up buying some “normal” items. Customers who come seeking the long tail are usually those that are willing to pay a premium, and thus the shops in Gandhi Bazaar are able to charge a premium for the non long tail items also.


Thus, if you purely look at rates of “common” items, Gandhi Bazaar, a market which offers the “long tail” will always be more expensive than other markets. Anecdotally, along with the Imam Pasand yesterday, I also bought a kilo of “vanilla” Raspuri mangoes, at the rate of Rs. 100 per kg. At the shop down the road, Raspuri was available for Rs. 90 per kg. The shop down the road, however, doesn’t stock Imam Pasand, which means that the price of Imam Pasand in that shop is infinity.

So if you are only looking to buy Raspuri, you are better off going to the shop down the road. If you either want only Imam Pasand, or both Imam Pasand and Raspuri, though, you should go to Gandhi Bazaar! In other words, the “range” that the fruit seller in Gandhi Bazaar offers implies that he can get away without discounting. Theoretically speaking, though, we can say that the fruit seller in Gandhi Bazaar actually discounts on the long tail items by the sheer act of stocking them (thus dropping their price from infinity to a finite number), and he is using this discount to sell his “normal” goods at “full price”. Ruminate on it, while I go off to devour a mango!


The Cooling Effect of Bangalore Rains

So it is “well known” that whenever it rains heavily in Bangalore, the city cools down like crazy. However, all these days, thanks to dodgy data from the Met department, it’s just been an (multiple) anecdotal observation, and not really backed by data.

However, thanks to the efforts of Pavan and Saurabh and the Yuktix team, we have “citizen weather monitoring centres” in several places across Bangalore. These are simple devices that have been installed on terraces or gardens of people, and they contain a rain gauge, a hygrometer, a thermometer and a wind gauge (or whatever it is that measures wind). And they have an embedded SIM card and transmit data every few minutes to the central server (for all you VCs, this is both “cloud-based” and “Internet of things”, so fund them already!).

The web interface isn’t great yet, and the data download is a bit dodgy, but hey, it works for now and we have actual data to show the weather conditions in Bangalore. And there are several stations all over the city (all installed by volunteers who have paid to have one such device in their homes. If you are interested, you can get one, too. Contact Pavan for this), so we can actually test popular hypothesis like how it can rain in one part of Bangalore and not in the other, etc.

Anyway, given the dodgy interface I’m unable plug a weather widget here (how cool would that have been?) so I’ve to shamelessly take screenshots and paste it. This one shows the temperature as measured by the device in Pavan’s house in 4th T Block (the station closest to my home) in the last one week:

Screen Shot 2015-04-24 at 9.49.10 am


Notice the nice sawtooth pattern of Bangalroe summer temperature. Temperatures rise steadily till about 2:30 pm and then fall steadily (but at a lower rate) till about 5:30 am. It is rather steady and repetitive as the graph shows. And then look at what happened yesterday! A steep plunge between 4:30 and 6:30 pm yesterday, and remember that the hailstorm started around 6!

I’ve noticed this on other days also (again by looking at Pavan’s data), and the same pattern holds. The hypothesis that rains do have an instant effect on the city’s temperature definitely holds!

For more interactivity with the data, you can check out Pavan’s station. Or whichever station that is closest to where you are! if there is none close to where you are, maybe it’s time for you to set up one such station!

Ganesha, wine and vodka

I know the wife has been intending to blog about this for a while now, but in this big bad blogosphere, intent counts for nothing, and given that she hasn’t written so far, I should go ahead and write this blog post. The basic funda is that Ganesha idols in “traditional” Indian culture, wine in European culture and smirnoff plain vodka in “modren” (sic) Indian culture are all similar.

So two days back I got invited to a “bring your own liquor” party. Now, there were other attendees who mentioned they were bringing stuff that I knew I was interested in drinking, like Desmondji Agave and Amrut Two Indies Rum. From that perspective, I knew that I wouldn’t be drinking whatever I carried. Yet, not carrying anything would make me look like a cheap guy, and this is one circle where I want to preserve my reputation. So what did I do? I picked up a bottle of Smirnoff plain vodka, simply because it is the most “fungible” drink. I’ll explain later.

Similarly, when you go for a function in India and don’t know what to gift, and are “too traditional” to gift gift cards, and think it’s not appropriate to give cash, you give a Ganesha idol. So for example after our wedding we had tonnes of Ganesha idols at home (similarly after our housewarming last year). Why did people gift Ganeshas? Because it is the most “fungible”. Again I’ll expect later.

And the wife reliably informs me that in Spain, when you have to go for a party but don’t know what to take, you take a bottle of wine. I don’t know about the fungibility of wine, but the fact that it is universally drunk, can be shared widely and is seen as a classy symbol makes it a popular choice of gift. So what connects these three?

So what connects? Fungibility of course. Economists have long argued that the best gift is cash, for the recipient can utilise that cash to buy the item that gives her maximum utility. Any non-cash gift decreases utility from the maximum that can be achieved by giving cash. This is a different discussion and I’ll not touch upon that now.

When you are going to a party, you can’t take along cash, so since the top choice is not available you take the “second best” option. What is the “second best” option in this case? Something that is close to cash, or something whose general utility is so high that the recipient values it as much as she would value the equivalent amount of cash. Of course you don’t assume that the recipient will sell your gift for cash, so you gift something that is a “safe option”, that you think they will have the least chance of rejecting.

So why did I take vodka? It is a universally popular, colourless odourless tasteless liquid, and I estimated that there was a good probability that the demand for that is going to be high. So even if I don’t drink what I carried, I posited, someone else will, and that will help me deliver maximal utility to the party.

With wine in Spain, you know everyone drinks and appreciates it, and there is a chance that it might be opened at the party itself. Even if it isn’t, wine in a sealed bottle doesn’t “depreciate”, and the host can then pass on some of the unused bottles at a party  that she attends! And soon there will be the virtuous wine circle. So essentially wine doesn’t disappoint, and is put to good use.

And it is exactly the same story with Ganesha idols. Like wine, it has intrinsic value. Who doesn’t like idols of a cute elephant-headed God? Maybe people who already have too many such idols? But then Ganesha idols don’t depreciate either, so all you need to do is to keep it in a safe place and pull it out the next time you’re going to a function! And thus the virtuous circle of Ganeshas will continue!

As it happened, at the end of Tuesday’s party, the bottles of Desmondji and Amrut Two Indies were empty. The Smirnoff I took remained unopened, as did another similar bottle which was possibly brought by another safe player. But I’m not concerned. I’m sure the hosts will consume it in due course, and even if they don’t, it will come of good use when they go to a party next!

Correlation hits finance again; Flash crash edition

So the curious case of the “flash crash” of 2010 is allegedly solved. A suspect has been named, and arrested. No it’s not one of the usual suspects. This is a guy named Navinder Singh Sarao, day-trading from his home near London. Yes, you saw that right. One guy caused the flash crash it seems. Read this excellent Matt Levine (wish I could write like him!!) piece on this.

Maybe it’s a biased sample that I follow on twitter, but the general discourse there is that this guy has probably been framed. Yes, he did “spoof”, that is, place orders on the markets without intending to execute them, and Levine explains in this post as to why that is a problem (not a very straightforward piece, this, but very insightful if you can “get it”). But in the earlier link you can see that Sarao actually switched off his (spoofing) algorithm before the flash crash happened. So it goes.

I had written glowingly about the “correlation term” in this post last week. But we had seen earlier that it was underestimation of correlation (between homeowners defaulting on mortgages) that led to the 2008 financial crisis. Based on my reading of the story so far, the flash crash was also caused due to some kind of correlation, but of a different kind – correlation between interacting algorithms. Or perhaps I should call this one “resonance”, since it was some kind of freaky frequency match that led to disastrous effects.

So as the more perceptive of you might already know, a large part of the volume of trading in most financial markets is now done by algorithms, without a human touch. On the upside, these result in highly efficient markets (sometimes too efficient, as Levine writes in this piece), and high price discovery. On the downside, this results in a large number of extremely dumb (for that is what algorithms are!) traders overcrowding the market, leading sometimes to extraordinarily dumb decisions by the market itself (like the flash crash). So high frequency trading is controversial, and there is this massive discourse that it is actually harmful to markets, liquidity be damned (Disclosure: I worked for a year at a high frequency trading hedge fund in India).

The thing with spoofing (placing orders and withdrawing them just before they can be executed, to move markets in a specific way) is that it doesn’t work on humans. At least not nowadays, given electronic markets and a high degree of algorithmic trading. The reason is that time scales don’t match – the time scale at which an order is placed, and then removed is much smaller than the time in which a human can react. And humans are usually intelligent enough to see through some of the more common spoofs.

In other words, spoofing is exclusively in place to trick “dumb” trading algorithms to move in a particular manner, which the spoofing algorithm spots and profits from. In other words, spoofing algorithms are some kind of “second degree algorithms” in that they try to profit by tricking other algorithms. And you have anti-spoofing algorithms that are in place to profit from snooping algorithms. And it is all pretty normal stuff.

So what caused that flash crash was a coming together of algorithms that shouldn’t have come together. A set of spoofs were designed to fool a particular class of algorithms, but instead ended up trapping another class of algorithms, which were possibly dumber and over-reacted to the spoof. This over-reaction triggered other algorithms which were possibly in place to detect quick market movements, and trade on the “momentum” to make a profit, and this led to a further drop in prices. And that triggered off a self-fulfilling thing among algorithms, and before the human overlords could react, the markets had crashed some 10%.


So the problem with algorithmic trading is that it creates dumb traders, and new traders are designed to take advantage of these dumb traders. Add a few more meta layers to this and you have more complexity than in a CDO Squared. As with most complex systems, such complexity is quite okay in normal times, as these multilayered algorithms usually cancel each other out and make each other more efficient. But once in a while, like on May 6th 2010, their frequencies resonate, and the system blows up.

Back when I was a HFT trader (that’s an AER redundancy), I was told that SEBI (the Securities and Exchange Board of India) mandated that our algorithms be “audited”, so that they are not malicious and don’t cause trouble. While I appreciate the principle behind the regulation, the problem is that most algorithms are harmless for the most parts, and by themselves, definitely harmless. It is only when they come together with other algorithms, and in very rare cases (like this one), that they create havoc, and a statutory audit can in no way pick this up.

The SEBI regulation is good in spirit but practically wholly useless!

Startup equity and the ultimatum game

The Ultimatum Game is a fairly commonly used game to study people’s behaviour, cooperation, social capital, etc. Participants are divided into pairs, and one half of the pair is given a sum of money, say Rs. 100. The objective of this player (let’s call her A) is to divide this money between herself and her partner for the game (whom we shall call B). There are no rules in terms of how A can divide the money, except that both sums need to be non-negative and add up to the total (Rs. 100 here).

After A has decided the division, B has an option to either accept or reject it. If B accepts the division, then both players get the amounts as per the division. If B rejects the division, both players get nothing.

Now, classical economics dictates that as long as B gets any amount that is strictly greater than zero, she should accept it, for she is strictly better off in such a circumstance than if she rejects it (by the amount that A has offered her). Yet, several studies have found that B often rejects the offer. This is to do with a sense of “unfairness”, that A has been unfair to her. Sociologists have found that certain societies are much more likely to accept an “unfair division” than others. And so forth.

The analogy isn’t perfect, but the way co-foundes of a startup split equity can be likened to a kind of an ultimatum game. Let’s say that there are two people with complementary and reasonably unique skills (the latter condition implies that such people are not easily replaceable), who are looking to get together to start a business. Right up front, there is the issue of who gets how much equity in the venture.

The thing with equity divisions between co-founders is that there is usually not much room for negotiation – if you end up negotiating too hard, it creates unnecessary bad blood up front between the founders which can affect the performance of the company, so you would want to get done with the negotiations as soon as possible. It should also be kept in mind that if one of the two parties is unhappy about his ownership, it can affect company performance later on.

So how do the founders decide the equity split in this light? Initially there will be feelers they send to each other on how much they are expecting. After that let us say that one of the founders (call him the proposer) proposes an equity division. Now it is up to the other founder (call him the acceptor) to either accept or reject this division. Considering that too much negotiation is not ideal, and that the proposer’s offer is an indication of his approximate demand, we can assume that there will be no further negotiation. If the acceptor doesn’t accept the division that the proposer has proposed, based on the above (wholly reasonable) conditions we can assume that the deal has fallen through.

So now it is clear how this is like an ultimatum game. We have a total sum of equity (100% – this is the very founding of the company, so we can assume that equity for venture investors, ESOPs, etc. will come later), which the proposer needs to split between himself and the acceptor, and in a way that the acceptor is happy with the offer that he has got. If the acceptor accepts, the company gets formed and the respective parties get their respective equity shares (of course both parties will then have to put in significant work to make that equity share worth something – this is where this “game” differs from the ultimatum game). If the acceptor rejects, however, the company doesn’t get formed (we had assumed that neither founder is perfectly replaceable, so whatever either of them starts is something completely different).

Some pairs of founders simply decide to split equally (the “fairest”) to avoid the deal falling through. The more replaceable a founder or commoditised his skill set is, the less he can be offered (demand-supply). But there are not too many such rules in place. Finally it all boils down to a rather hard behavioural problem!

Thinking about it, can we model pre-nuptial agreements also as ultimatum games? Think about it!

The problem with Indian agriculture, and government

The problem with the Indian agriculture sector is that the government takes a very “cash view” of the sector while what is required is a “derivative view”. 

So Congress VP Rahul Gandhi railed on in a rally about how the current Narendra Modi government is anti-farmer, and pointed out at the land acquisition amendment bill and the lack of raising of “minimum support price” as key points of failure. Gandhi was joined at the rally by a large number of farmers, who reports say were primarily very pissed off about the failure of their rabi crops thanks to unseasonal rains in the last month and a bit.

If the government were to take Gandhi’s criticism seriously, what are they expected to do? Not amend the land acquisition act, or amend it in a different way? Perhaps, and we will not address that in this post, since it is “out of syllabus”. Increase the Minimum Support Price (MSP)? They might do that, but it will do nothing to solve the problem.

As I had pointed out in this post written after a field trip to a farm, what policymakers need understand is that farming is fundamentally a business, and like any other business, there is risk. In fact, given the number of sources of uncertainty that exist, it can be argued that farming is a much riskier business than a lot of other “conventional” businesses.

So there is the risk of high prices of inputs, there is risk of bad weather, there is risk of a glut in supply that leads to low prices, there is a risk that the crop wasn’t harvested at the right time, there is a risk that elephants trampled the field, or there is a risk that there might be a new strain of bugs that might destroy the crops. And so forth. And given that most farmers in India are “small”, with limited land holdings, it needs to be kept in mind that they don’t have diversification as a (otherwise rather straightforward) tool to mitigate their risks.

And when the farmers face so many risks, what does the government do? Help them mitigate at max one or two of it. One of them is the “minimum support price” which is basically a put option written by the government, for free, in favour of the farmers. All it entails is that the farmer  is assured of a minimum price for his wares if market prices are too low at the time of harvest. In other words, it helps the farmer hedge against price risk.

What other interventions do Indian governments do in farming? There are straightforward subsidies, all of the input variety. So farmers get subsidised seeds, subsidised fertilisers, subsidised (or in several cases, free) electricity, occasional subsidies in irrigation, subsidised loans (“priority sector lending” rules), and occasionally, when shit hits the fan, a loan waiver.

Barring the last one, it is easy to see that the rest are all essentially input subsidies, making it cheaper for the farmer to produce his produce (I’m proud of that figure of speech here, and I don’t know what it’s called in English). Even loan waivers, while they happen when market conditions are really bad, are usually arbitrary political decisions, and never targeted, meaning that there are always significant errors, of both omission and commission.

So if you ask the question of whether the government, through all these interventions, make the business of farming easier, it should be clear that an answer is no, for while it makes inputs cheaper and helps farmers hedge against price risk, it doesn’t help at all in mitigation of any other risks. Instead, what the government is essentially doing is by paying the farmers a premium (subsidised inputs, free options) and expecting them to take care of the risks by themselves. In other words, small “poor” farmers, who are least capable of handling and managing risk, are the ones who are handling the risk, and at best the government is just providing them a premium!

The current government has done well so far in terms of recognising risk management as a tool for overall wellbeing. For example, the Jan Dhan Yojana accounts (low-cost bank accounts for the hitherto unbanked) come inbuilt with a (albeit small) life insurance cover. In his budget speech earlier this year, the Finance Minister mentioned a plan to introduce universal insurance against accidental death. Now it is time the government recognises the merits of this policy, and extends it to other sectors, notably agriculture.

What we need is a move away from “one delta” cash subsidies and a move towards better risk management. The current agricultural policies of successive governments basically ensure that the farmer makes more when times are good (lower inputs costs, free put options (MSP) with high strike price), and makes nothing when times are bad. Rudimentary utility theory teaches us that the value of a rupee when times are good is much lower than the value of a rupee when times are bad. And for the government, it doesn’t really matter as to when it spends this money, since its economic cycle is largely uncorrelated with farmers’ economic cycles. So why waste money by spending it at a time of low marginal utility as opposed to spending it at a time of high marginal utility?

In other words, the government should move towards an institutionalised system of comprehensive crop insurance. Given the small landholdings, transaction costs of such insurance is going to be high, and the government should help develop this market by providing subsidies. And this subsidy can be easily funded – remember that the government is already paying some sort of a premium to farmers so that they manage their own risk, and part of this can go towards helping farmers manage their risk better.

It is not going to be politically simple, for the opposition (like Rahul Gandhi) will rail that the government is taking money away from farmers. But with the right kind of messaging, and subsidies for insurance, it can be done.