Category: randomness

The Trouble With Analyst Reports

The only time I watch CNBC is in the morning when I’m at the gym. For reasons not known to me, my floor in office lacks televisions (every other floor has them) and the last thing I want to do when I’m home is to watch TV, that too a business channel, hence the reservation for the gym. I don’t recollect what programme I was watching but there were some important looking people (they were in suits) talking and on the screen “Target 1200” flashed (TVs in my gym are muted).

Based on some past pattern recognition, I realized that the guy in the suit was peddling the said stock (he was a research analyst) and asking people to buy it. According to him, the stock price would reach 1200 (I have no clue what company this is and how much it trades for now). However, there were two important pieces of information he didn’t give me, because of which I’ll probably never take advice from him or someone else of his ilk.

Firstly, he doesn’t tell me when the stock price will reach 1200. For example, if it is 1150 today, and it is expected to reach 1200 in 12 years, I’d probably be better off putting my money in the bank, and watching it grow risk-free. Even if the current price were lower, I would want a date by which the stock is supposed to reach the target price. Good finance implies tenure matching, so I should invest accordingly. If the stock is expected to give good returns in a year, then I should put only that money into it which I would want to invest for around that much time. And so forth.

Then he doesn’t tell me how long it will stay at 1200. I’m not an active investor. I might check prices of stocks that I own maybe once in a week (I currently don’t own any stock). So it’s of no use to me if the price hits 1200 some time during some intraday trade. i would want the price to remain at 1200 or higher for a longer period so that I can get out.

Thirdly and most importantly, he doesn’t tell me anything about volatility. He doesn’t give me any statistics. He doesn’t tell me if 1200 is the expected value of the stock, or the median, or the maximum, or minimum, at whatever point of time (we’ve discussed this time bit before). He doesn’t tell me what are the chances that I’ll get that 1200 that he professes. He doesn’t tell me what I can expect out of the stock if things don’t go well. And as a quant, I refuse to touch anything that doesn’t come attached with a distribution.

Life in general becomes so much better when you realize and recognize volatility (maybe I’ll save that for another discourse). It helps you set your expectations accordingly; it helps you plan for situations you may not have thought of; most importantly it allows you to recognize the value of options (not talking about financial options here; talking of everyday life situations). And so forth.

So that is yet another reason I don’t generally watch business TV. I have absolutely no use for their stock prediction and tips. And I think you too need to take these tips and predictions with a bit of salt. And not spend a fortune buying expensive reports. Just use your head. Use common sense. Recognize volatility. And risk. And you’ll do well.

Kabaddi and Jesus Navas

I’ve always talked about the Kabaddi style of solving a problem. In Kabaddi, when you are defending, six out of the seven players in the team form a chain in order to encircle the attacker. The seventh defender, however, strikes it alone, in a different direction, trying to draw the attacker into a position where he can be effectively surrounded.

Now there is a footballing analogy to this – the Jesus Navas style. Those of you who watched either Spain’s game with Honduras or the second half of their loss to Switzerland would’ve noticed that Spain effectively followed two lines of attack. The first was the traditional way – attack down the middle in a series of slow passes and build-up. Five of Spain’s front six players would get involved in this attack down the centre, almost rendering their game one-dimensional. And then there was Navas.

I haven’t confirmed this stat but in the game and half that he has played Navas has completed more crosses than anyone else in the tournament. He would strike it on his own down the Spanish right flank, hug the touchline, beat the full back and put in crosses. Minute in and minute out. Sometimes with a little help from full back Sergio Ramos, but mostly alone. It was fantastic to watch.

What this ended up doing was to divert the attention of the opposing defenders to cover Navas. If everyone were to have been attacking down the centre, the defending team could’ve just parked their bus in front of their centre and prevented any scoring. Spain letting free this one guy to take a different route meant that the opposition needed to cover that also leading to insufficient cover in the centre (it is another matter that Spain failed to score against Switzerland. But they did get so many more chances after Navas came on).

I’ve always been fascinated by such strategies at work, in business. You have a bunch of guys who try to attack the problem front-on, in the conventional way, working together, passing to each other frequently. And then there is this one guy who has been left out of this clique who attacks the problem “from the flank”. In his own way, without fear of failure. He knows that he is only an auxiliary solver, that he has nothing to lose (Navas lost his place in the XI after the Honduras match but I don’t think he had expected to ever play at all), and he can just go for it. The option value of letting one guy in the team loose in order to search for alternate solutions while everyone else is building up down the middle is immense, I think.

This is similar to Nassim Taleb’s “barbell investment strategy”. Acccording to that, he parks some 90% of his assets in ultra-risk government securities. They don’t give spectacular returns but his money is safe. And the rest of the 10% he uses to punt by buying stuff like out-of-the-money options. If they expire worthlessly, he hasn’t lost much of his wealth. The optionality (here, literally) of that additional 10% is, however, immense, and there is potential for spectacular returns from this strategy. with losses being capped.

Diminishing Value of a Red Card

Often when we see players being sent off AND penalty kick being awarded in the event of an illegal stop of a goal-bound ball, Baada and I have thought that the punishment is too harsh. That for stopping one goal, the team effectively gives away the goal (conversion rate of penalties is high) and also loses a player (sometimes the goalie) for the rest of the game.

Now, after last night’s strategic hand ball by Luis Suarez, people are complaining that the punishment is not enough. Though it was a split-second instinctive decision by Suarez to handball, even if he were to replay the incident in his head and analyze the costs and benefits, I’m sure he would’ve done what he did. This clearly contradicts what I mentioned in the first paragraph.

The main issue here is with the value of a red card  at various stages of a game. The red card has intrinsic value – of being suspended for the next game. In addition to this, the red card leaves the team one short for the rest of the game, and so it is clear that the later a red card is given out, the lesser the disadvantage it causes the team because they’ve to play for lesser time with a man short.

What makes Suarez’s decision more logical is the time value of a one-goal lead. The lesser the time left in the game, the more the value of the one-goal lead since there is lesser time for which it needs to be protected. And in this case, the handball occurred on what might have been the last “kick” in the game, and so the value of the one-goal lead was really high.

The earlier this incident had occurred in the match, the less would’ve been Suarez’s incentive to handball – more time to win back the conceded goal and more time to play a man short if redcarded. At the time when it actually occurred, Suarez would’ve been a fool to NOT handball. The payoffs were heavily loaded in favour of handballing and he did it.

People on twitter are suggesting that rules be changed, that the goal should’ve been awarded anyway instead of the penalty and stuff, but considering that the same punishment costs much more if given out earlier in the game, I think the current punishment is appropriate. The excess of this punishment in earlier stages of the game is compensated by the punishment being too little in the latter stages, and on an average I think it is appropriate.

Let’s continue to keep football simple and not clutter it with Duckworth-Lewis kind of rules. And congrats to Suarez for taking the most logical decision at the moment. It is indeed as great a “sacrifice” as Ballack’s tactical yellow card against Korea in the 2002 semis.

And I feel sad for Asamoah Gyan. But then again, with Ghana being in the knockout stages solely on the merit of two Gyan penalties, it is only appropriate that they are going out nowon the demerit of Gyan’s missed penalty.

What Should Mexico Do?

If Mexico and Uruguay draw their last league game, then both of them go through to the second round irrespective of what France and South Africa do. However, on account of a better goal difference, Uruguay will qualify as group winners and face the second-placed team from Group B, while Mexico will qualify second and meet the Group B winners, likely to be Argentina.

Uruguay’s option is clear. Play for a draw. If Mexico go for a win, Uruguay should just sit back and try hit back on the counterattack (and in terms of players and style, they are very well equipped for that). Simple case of getting men behind the ball and putting gaaji.

Mexico’s strategy is not so straightforward. The “greedy” thing to do would be to play for a draw, in which case they will most likely end up facing Argentina in the second round (if you remember, Mexico went out last World Cup by losing to Argentina at the same stage). On the other hand, if Mexico beat Uruguay, they will top Group A and meet a potentially inferior team (Korea or Greece) in the second round.

As mentioned earlier, if Mexico go for a win, Uruguay will simply defend and play a counterattacking game which they are good at, so I don’t know if they are going to go for it.

Thinking about it, it comes down to Mexico’s payoff function. I’m sure their payoff is an increasing function of how far they progress in the tournament. However, we should be able to identify one particular “jump” in payoff – some kind of a discontinuity, where the payoff increases considerably for one additional round of progress in the tournament.

If this “jump” is for the second round, Mexico can afford to put Ranatunga Principle, get a peaceful draw against Uruguay and claim their “jump reward”.

If the “jump” is for the quarter-finals, however, then Mexico will want to take the risk at this round in order to get themselves easier opponents in the round of 16 (I’m assuming here that Mexico consider Korea or Greece as much easier opponents than Argentina).

If the “jump” occurs further down in the tournament, I think there is way too much randomness about their potential quarterfinal opponents (especially given the fuzzy results in Groups C and D) and opponents as hard as  (or harder than ) Argentina cannot be ruled out. Nevertheless, they would rather face one tough opponent than two (and I’m assuming here that no team is significantly “harder” for Mexico than Argentina) and so they should put fight to avoid Argentina and thus go for a win.

Considering that they have reached the Round of 16 with reasonable regularity in the last few World Cups, I presume that their “payoff jump” will occur later in the tournament. And based on the above reasoning that means they should go for a win against Uruguay, so that they can try avoid Argentina.

And it is on this thin thread that the French are hanging their hopes (though first they need to thulp South Africa, no easy task).

Relationship Stimulus

This post doesn’t necessarily restrict its scope to romantic relationships, though I will probably use an example like that in order to illustrate the concept. The concept that I’m going to talk about any kind of bilateral relationship, be it romantic or non-romantic, or between any two people or between man and beast or between two nations.

Let us suppose Alice’s liking for Bob is a continuous variable between 0 and 1. However, Alice never directly states to Bob how much she likes him. Instead, Bob will have to infer this based on Alice’s actions. Based on a current state of the relationship (also defined as a continuous variable between 0 and 1) and on Alice’s latest action, Bob infers how much Alice likes him. There are a variety of reasons why Bob might want to use this information, but let us not go into that now. I’m sure you can come up with quite a few yourself.

Now, my hypothesis is that the relationship state (which takes into account all past information regarding Alice’s and Bob’s actions towards each other) can be modelled as an exponentially-smoothed variable of the time series of Alice’s historical liking for Bob. To restate in English, consider the last few occasions when Alice and Bob have interacted, and consider the data of how much Alice actually liked Bob during each of these rounds. What I say is that the “current level” that I defined in the earlier paragraph can be estimated using this data on how much Alice liked Bob in the last few interactions. By exponentially smoothed, I mean that the last interaction has greater weight than the one prior to that which has more weight than the interaction three steps back, and so on.

So essentially Alice’s liking for Bob cannot be determined by her latest action alone. You use the latest action in conjunction with her last few actions in order to determine how much she likes Bob. If you think of inter-personal romantic relationships, I suppose you can appreciate this better.

Now that you’ve taken a moment to think about how my above hypotheses work in the context of human romantic relationships, and having convinced yourself that this is the right model, we can move on. To simplify all that I’ve said so far, the same action by Alice towards Bob can indicate several different things about how much she now likes him. For example, Alice putting her arm around Bob’s waist when they hardly knew each other meant a completely different thing from her putting her arm around his waist now that they have been married for six months. I suppose you get the drift.

So what I’m trying to imply here is that if you are going through a rough patch, you will need to try harder and send stronger signals. When the last few interactions haven’t gone well, the “state function of the relationship” (defined a few paragraphs above) will be at a generally low level, and the other party will have a tendency to under-guess your liking for them based on your greatest actions. What might normally be seen as a statement of immense love might be seen as an apology of an apology when things aren’t so good.

It is just like an economy in depression. If the government sits back claiming business-as-usual it is likely that the economy might just get worse. What the economy needs in terms of depression is a strong Keynesian stimulus. It is similar with bilateral relationships. When the value function is low, and the relationship is effectively going through a depression, you need to give it a strong stimulus. When Alice and Bob’s state function is low, Alice will have to do something really really extraordinary to Bob in order to send out a message that she really likes him.

And just one round of Keynesian stimulus is unlikely to save the economy. There is a danger that given the low state function, the economy might fall back into depression. Similarly when you are trying to get a relationship out of a “depressed” state, you will need to do something awesome in the next few rounds of interaction in order to make an impact. If you, like Little Bo Peep, decide that “leave ’em alone, they will come home”, you are in danger of becoming like Japan in the 90s when absolute stagnation happened.

Booze and volatility

Another of those things I’ve been intending to write for a really long time. Occasionally when I’m not feeling too good mentally, people ask me to go have a drink telling me that everything will be alright. However, given my limited experience in this I’m not too confident it will work. In fact, the only one time I tried drowning my sorrows in alcohol (this was over four years ago) I ended up feeling significantly worse, worse enough to have not tried it since.

The thing with booze is that it increases the volatility of your state of mind. This means that it will flatten out the curve according to which your mental state moves. So after you’ve had a drink or few, you are unlikely to remain in the same state that you were in that you started off at. You end up feeling either significantly better or significantly worse – and the chances of both these go up tremendously when you drink.

I know I have been so far acting based on one data point that went adversely, but I don’t know what causes the selection bias in people who have been through both sides significantly! Of feeling much worse and feeling much better after having some drinks. Why is it that even though all of them would’ve been through significantly worse after drinking at some point of time or the other, they tend to forget about it and only think of the times when they’ve felt better?

Is it that whether you feel good or not is some kind of a binary payoff depending upon the level of the state of mind (basically state of mind < cutoff => “bad”; state of mind >= cutoff implies “good”)? If this is true, then whenever you are “out of the money” (feeling bad), you dont’ really care if you go even more out of the money – your overall feeling doesn’t change by much. And so you don’t really mind the cases when the alcohol starts making you feel significantly worse. But then the barrier is ahead of you so by increasing volatility, you are giving yourself a better chance of surmounting the barrier so drinking makes sense! But then under this condition it doesn’t make sense to drink at all when you’re already feeling good!

Are there any other reasons you can think of for this selection bias? Why do people give more benefits to positive movement in state of mind as a function of drinking than to negative movement in state of mind? Or is it that volatility is a non-intuitive concept and “there’s a better chance you’ll feel better if you drink” is a simple way of communicating it? And let me know your experience about drink making you feel worse..

Wasting Youth

Nowadays everyone seems to be preparing for JEE. It is almost as if it is a logical progression to join some JEE coaching factory once you are done with 10th standard. Yeah, the numbers were quite large in my time (~10 yrs back) itself. But they are humongous now, and it is not funny.

Yeah, awareness about IIT and people feeling good about themselves and wanting to go study at India’s best undergraduate institutions is great. It is brilliant. Fantastic. What is not so great, brilliant and fantastic is that tens of thousands of youth are wasting two years of their prime youth trying to mug for an entrance exam in which they stand little chance of doing well.

I just hope I’m not sounding condescending here, but it intrigues me that so many people who have very little chances of making it through the JEE slog so much for it. I think it is due toe the unhealthy equilibrium that has been reached with respect to the exam, which makes everyone waste so much time. Let me explain.

So over the years the JEE has got the reputation of being a “tough” exam. And over the years, maybe due to the way papers are structured or the way factories train people, people have figured out that hard work and extra hours of preparation helps. I could get into studsandfighters mode here but in line with my promise let me try and explain without invoking the framework. And you need to remember that the JEE uses “relative grading” – how well you have done is dependent on how badly others have done.

So if everyone has put in that much extra hard work, you are likely to lose out by not putting in that extra work. And so you increase your effort. And so does everyone else. Yeah this is a single iteration game but still looking at the competition and peer pressure eveyone is forced to raise their effort. Everyone is forced to, to quote the Director of my JEE factory, “work up to human limit”.

Yeah, a few hundred people every year manage to “crack” the system and get through without putting in that much effort. But then their numbers are small compared to the number of people who get admitted, so people who get through based on sheer hard work do tend to get noticed more, and spur other aspirants to work even harder. And so forth.

Yes, there is a problem with a system. Something is not right when a large proportion of youth in the country is wasting away two years of prime youth in preparing for some entrance exam. It is easy to see the fundamental problem – shortage of “really good quality” engineering colleges (I argue that this mad fight for IIT seats shows the gap between IITs and the next level of engineering colleges – at least in terms of public perception). But considering that as given I wonder what we could change. I wonder what we could do in order to save our youth.

As an aside, one thing I’ve noticed about several JEE aspirants is that they don’t give up. I don’t know if this is necessarily a good thing – to carry on with the mad fight even if you know that your chances of making it are remote. Yeah I’m sure there is peer pressure and status issues with respect to giving up. But then I suppose I would have a lot more respect for someone who would give up and enjoy life rather than continue the mad fight knowing fully well that his chances are remote.

Looking back, I do regret wasting those two years in mad JEE mugging. Ok I must admit I did have my share of fun back then but still looking back I would have definitely preferred to have not worked so hard back then. And of course I count myself lucky that I got through the JEE and my hard work in those two years wasn’t in vain.

Orange Juice and Petrol

So I was reading this article by Ajay Shah about administered pricing for petroleum. He does an excellent (though it gets a bit technical in terms of statistics) analysis about what could go wrong if the government were to free pricing of petroleum products. He mostly argues in favour of deregulation, and that is a view that I completely endorse.

One of the big fears about deregulation that he mentions is the fear that volatility in retail prices of petroleum products might increase, and he argues that this is a good thing and is much better than the government artificially hiding the prices and subjecting the junata to major price shocks once in a while. While I agree with him on this, I don’t think prices will change frequently in the first place.

While I was reading this article, I started thinking about the neighbourhood Sri Ganesh Fruit Juice (yeah there are a dozen of those in every neighbourhood in Bangalore) center. About how the guy keeps the price of orange juice constant throughout the year, despite the price and availability of oranges themselves fluctuating wildly across seasons. Yeah he might do minor adjustments such as changing the proportion of water but he can’t do too much of it since he needs to maintain quality.

The basic funda here is that customers want certainty. Every time they go to the shop for their fix of orange juice, they want certainty in the prices. Even if you are on an average cheaper, you will lose customers if your price is more volatile than your competitor’s. Of course there are occasions when you can’t help it and are forced to change your price – and on these occasions your competitors are also likely to do the same. But as far as possible, you try your best to decouple the price of orange juice from the price of orange which is pretty volatile.

Now I don’t know if the volatility in crude oil prices is more than the volatility in orange prices (it’s likely to be) but considering that oil companies are supposed to be more sophisticated than your neighbourhood juice shop guy, I would expect similar behaviour from them – of keeping retail prices of petroleum products as stable as they can. Of course they are likely to follow long-term trends but they are surely not going to pass on the short-time noise in prices to the customers.

So this fear of increase in volatility of retail prices is unfounded, assuming of course that the oil marketing companies are good businesspeople!

Horoscopes and Caesarean Births

The fundamental question is about what needs to be considered as a zero point in a person’s life – conception or delivery. I don’t want to start a debate on abortion here, but just wonder what Indian astrology considers to be the zero point of a person’s life. The answer to this question can determine the effectiveness of Indian astrology, even assuming that it is ok that it hasn’t been recalibrated for a few millenia.

Now my argument here is about the numerous instances in Indian mythology where the child’s future is written down by an astrologer even when it is in its mother’s womb. If an astrologer can tell a child’s future when it is in its mother’s womb, isn’t it an indicator that it is the position of stars at conception that matters more than the position of stars at the time of delivery?

The thing is that no one really knows when a child was conceived. Hence, the time of the child’s delivery is usually used as some sort of a proxy to determine when it was conceived. So basically astrology in its current form has a formula to calculate time of conception based on time of delivery, and so effectively what we have as astrology now is a product of two vectors – one that transposes time of birth to time of conception, and another that translates time of conception to position of stars at conception which then gives rise to the horoscope.

I suppose you can understand that there is obviously one source of error in this – regarding the determination of time of conception at the time of birth – basically no two kids born at the same moment would have been conceived at the same moment, right? So this introduces a fundamental error into Indian astrology.

And as if it were not enough, technology has (as usual) stepped in to hinder religion. The concept of Caesarean section has ended up playing complete havoc with the time-tested formulae of determining time of conception based on time of birth. The concept of Caesarean section has ensured that children need not remain in their mothers’ wombs for a “fixed quota” of time, and there is a very good chance they get released early.

So my argument is that Indian astrology as it stands now is inappropriate for people who were born through Caesarean section, since the error in determination of time of conception is extremely high. Also considering how discontinous things are – there are cases where a half an hour’s change in birth time can completely change a person’s horoscope – the impact of this error is too large to be ignored.

The most common use of astrology in recent times is that horoscope-match is considered by some as a necessary condition for matchmaking. Thinking about it, it is bad (and inaccurate) enough if one of the two parties has been born by Caesarean section. I wonder if it has any impact at all if both parties have been born by Caesarean section!

PS: Back when I was in the arranged scissors market, and my mother was around, this is the argument she would give to people who would demand to see my horoscope in the course of matchmaking. That it didn’t make sense given I was born through Caesarean section.

Moron Astrology

So this morning I was discussing my yesterday’s post on astrology and vector length with good friend and esteemed colleague Baada. Some interesting fundaes came out of it. Since Baada has given up blogging (and he’s newly married now so can’t expect him to blog) I’m presenting the stuff here.

So basically we believe that astrology started off as some kind of multinomial regression. Some of ancestors observed some people, and tried to predict their behaviour based on the position of their stars at the time of their birth. Maybe it started off as some arbit project. Maybe if blogs existed then, we could say that it started off as a funda session leading up to a blog post.

So a bunch of people a few millenia ago started off on this random project to predict behaviour based on position of stars at the time of people’s birth. They used a set of their friends as the calibration data, and used them to fix the parameters. Then they found a bunch of acquaintances who then became the test data. I’m sure that these guys managed to predict behaviour pretty well based on the stars – else the concept wouldn’t have caught on.

Actually it could have gone two ways – either it fit an extraordinary proportion of people in which case it would be successful; or it didn’t fit a large enough proportion of people in which case it would have died out. Our hunch is that there must have been several models of astrology, and that natural selection and success rates picked out one as the winner – none of the other models would have survived since they failed to predict as well on the initial data set.

So Indian astrology as we know it started off as a multinomial regression model and was the winner in a tournament of several such models, and has continued to flourish to this day. Some problem we find with the concept:

  • correlation-causation: what the initial multinomial regression found is that certain patterns in the position of stars at the time of one’s birth is heavily correlated with one’s behaviour. The mistake that the modelers and their patrons made was the common one of associating correlation with causation. They assumed that the position of stars at one’s birth CAUSED one’s behaviour. They probably didn’t do much of a rigorous analysis to test this out
  • re-calibration: another problem with the model is that it hasn’t been continuously recalibrated. We continue to use the same parameters as we did several millenia ago. Despite copious quantities of new data points being available, no one has bothered to re-calibrate the model. Times have changed and people have changed but the model hasn’t kept up with either. Now, I think the original information of the model has been lost so no one can recalibrate even if he/she chooses to

Coming back to my earlier post, one can also say that Western astrology is weaker than Indian astrology since the former uses a one-factor regression as against the multinomial regression used by the latter; hence the former is much weaker at predicting.