Just Plot It

One of my favourite work stories is from this job I did a long time ago. The task given to me was demand forecasting, and the variable I needed to forecast was so “micro” (this intersection that intersection the other) that forecasting was an absolute nightmare.

A side effect of this has been that I find it impossible to believe that it’s possible to forecast anything at all. Several (reasonably successful) forecasting assignments later, I still dread it when the client tells me that the project in question involves forecasting.

Another side effect is that the utter failure of standard textbook methods in that monster forecasting exercise all those years ago means that I find it impossible to believe that textbook methods work with “real life data”. Textbooks and college assignments are filled with problems that when “twisted” in a particular way easily unravel, like a well-tied tie knot. Industry data and problems are never as clean, and elegance doesn’t always work.

Anyway, coming back to the problem at hand, I had struggled for several months with this monster forecasting problem. Most of this time, I had been using one programming language that everyone else in the company used. The code was simultaneously being applied to lots of different sub-problems, so through the months of struggle I had never bothered to really “look at” the data.

I must have told this story before, when I spoke about why “data scientists” should learn MS Excel. For what I did next was to load the data onto a spreadsheet and start looking at it. And “looking at it” involved graphing it. And the solution, or the lack of it, lay right before my eyes. The data was so damn random that it was a wonder that anything had been forecast at all.

It was also a wonder that the people who had built the larger model (into which my forecasting piece was to plug in) had assumed that this data would be forecast-able at all (I mentioned this to the people who had built the model, and we’ll leave that story for another occasion).

In any case, looking at the data, by putting it in a visualisation, completely changed my perspective on how the problem needed to be tackled. And this has been a learning I haven’t let go of since – the first thing I do when presented with data is to graph it out, and visually inspect it. Any statistics (and any forecasting for sure) comes after that.

Yet, I find that a lot of people simply fail to appreciate the benefits of graphing. That it is not intuitive to do with most programming languages doesn’t help. Incredibly, even Python, a favoured tool of a lot of “data scientists”, doesn’t make graphing easy. Last year when I was forced to use it, I found that it was virtually impossible to create a PDF with lots of graphs – something that I do as a matter of routine when working on R (I subsequently figured out a (rather inelegant) hack the next time I was forced to use Python).

Maybe when you work on data that doesn’t have meaningful variables – such as images, for example – graphing doesn’t help (since a variable on its own has little information). But when the data remotely has some meaning – sales or production or clicks or words, graphing can be of immense help, and can give you massive insight on how to develop your model!

So go ahead, and plot it. And I won’t mind if you fail to thank me later!

Elegant and practical solutions

There are two ways in which you can tie a shoelace – one is the “ordinary method”, where you explicitly make the loops around both ends of the lace before tying together to form a bow. The other is the “elegant method” where you only make one loop explicitly, but tie with such great skill that the bow automatically gets formed.

I have never learnt to tie my shoelaces in the latter manner – I suspect my father didn’t know it either, because of which it wasn’t passed on to me. Metaphorically, however, I like to implement such solutions in other aspects.

Having been educated in mathematics, I’m a sucker for “elegant solutions”. I look down upon brute force solutions, which is why I might sometimes spend half an hour writing a script to accomplish a repetitive task that might have otherwise taken 15 minutes. Over the long run, I believe, this elegance will pay off, in terms of scaling easier.

And I suspect I’m not alone in this love for elegance. If the world were only about efficiency, brute force would prevail. That we appreciate things like poetry and music and art and what not means that there is some preference for elegance. And that extends to business solutions as well.

While going for elegance is a useful heuristic, sometimes it can lead to missing the woods for the trees (or missing the random forests for the decision trees if you may will). For there are situations that simply don’t, or won’t, scale, and where elegance will send you on a wild goose chase while a little fighter work will get the job done.

I got reminded of this sometime last week when my wife asked me for some Excel help in some work she was doing. Now, there was a recent article in WSJ which claimed that the “first rule of Microsoft Excel is that you shouldn’t let people know you’re good at it”. However, having taught a university course on spreadsheet modelling, there is no place to hide for me, and people keep coming to me for Excel help (though it helps I don’t work in an office).

So the problem wasn’t a simple one, and I dug around for about half an hour without a solution in sight. And then my wife happened to casually mention that this was a one-time thing. That she had to solve this problem once but didn’t expect to come across it again, so “a little manual work” won’t hurt.

And the problem was solved in two minutes – a minor variation of the requirement was only one formula away (did you know that the latest versions of Excel for Windows offer a “count distinct” function in pivot tables?). Five minutes of fighter work by the wife after that completely solved the problem.

Most data scientists (now that I’m not one!) ┬átypically work in production environments, where the result of their analysis is expressed in code that is run on a repeated basis. This means that data scientists are typically tuned to finding elegant solutions since any manual intervention means that the code is not production-able and scalable.

This can mean finding complicated workarounds in order to “pull the bow of the shoelaces” in order to avoid that little bit of manual effort at the end, so that the whole thing can be automated. And these habits can extend to the occasional work that is not needed to be repeatable and scalable.

And so you have teams spending an inordinate amount of time finding elegant solutions for problems for which easy but non-scalable “solutions exist”.

Elegance is a hard quality to shake off, even when it only hinders you.

I’ll close with a fairytale – a deer looks at its reflection and admires its beautiful anchors and admonishes its own ugly legs. Lion arrives, the ugly legs help the deer run fast, but the beautiful antlers get stuck in a low tree, and the lion catches up.

 

Triangle marketing

This blog post is based more on how I have bought rather than how I have sold. The basic concept is that when you hear about a product or service from two or more independent sources, you are more likely to buy it.

The threshold varies by the kind of product you are looking at. When it is a low touch item like a book, two independent recommendations are enough. When it involves higher cost and has higher impact, like a phone, it might be five recommendations. For something life changing like a keto diet, it might be ten (I must mention I tried keto for half a day and gave up, not least because I figured I don’t really need it – I’m barely 3-4 kg overweight).

The important point to note is that the recommendations need to come from independent sources – if two people who you didn’t expect to have a similar taste in books were to recommend the same book, the second of these recommendations is likely to create an “aha moment” (ok I’m getting into consultant-speak now), and that is likely to drive a purchase (or at least trying a Kindle sample).

In some ways, exposure to the same product through independent sources is likely to create a feeling of a self-fulfilling prophecy. “Alice is also using this. Bob is also using this” will soon go into “everybody seems to be using it. I should also use it”.

So what does this mean to you if you are a seller? Basically you need to hit your target audience through various channels. I had mentioned in my post earlier this week about how branding creates a “position of strength“, and how direct sales is normally hard because it is done through a position of weakness.

The idea is that before you hit your audience with a direct sale, you need to “warm them up” with your brand, and you need to do this through various channels. Your brand needs to impact on your audience through multiple independent channels, so that it has become a self-fulfilling prophecy before you approach to make the sale.

What these precise channels are depends on your business and the product that you’re trying to sell, but the important thing is that they are independent. So for example, putting advertisements in various places won’t help since the target will treat all of them as coming from the same source.

Finally, where is the “triangle” in this marketing? It is in the idea that you complete the branding and sales by means of “triangulation”. You send out vectors in seemingly random directions trying to build your brand, and they will get reflected till a time when they intersect, or “triangulate”. Ok I know my maths here is messy ant not up to my usual standard, but I guess you know what I’m getting at!

 

The Derick Parry management paradigm

Before you ask, Derick Parry was a West Indian cricketer. He finished his international playing career before I was born, partly because he bowled spin at a time when the West Indies usually played four fearsome fast bowlers, and partly because he went on rebel tours to South Africa.

That, however, doesn’t mean that I never watched him play – there was a “masters” series sometime in the mid 1990s when he played as part of the ‘West Indies masters” team. I don’t even remember who they were playing, or where (such series aren’t archived well, so I can’t find the score card either).

All I remember is that Parry was batting along with Larry Gomes, and the West Indies Masters were chasing a modest target. Parry is relevant to our discussion because of the commentator’s (don’t remember who – it was an Indian guy) repeated descriptions of how he should play.

“Parry should not bother about runs”, the commentator kept saying. “He should simply use his long reach and smother the spin and hold one end up. It is Gomes who should do the scoring”. And incredibly, that’s how West Indies Masters got to the target.

So the Derick Parry management paradigm consists of eschewing all the “interesting” or “good” or “impactful” work (“scoring”, basically. no pun intended), and simply being focussed on holding one end up, or providing support. It wasn’t that Parry couldn’t score – he had at Test batting average of 22, but on that day the commentator wanted him to simply hold one end up and let the more accomplished batsman do the scoring.

I’ve seen this happen at various levels, but this usually happens at the intra-company level. There will be one team which will explicitly not work on the more interesting part of the problem, and instead simply “provide support” to another team that works on this stuff. In a lot of cases it is not that the “supporting team” doesn’t have the ability or skills to execute the task end-to-end. It just so happens that they are a part of the organisation which is “not supposed to do the scoring”. Most often, this kind of a relationship is seen in companies with offshore units – the offshore unit sticks to providing support to the onshore unit, which does the “scoring”.

In some cases, the Derick Parry school goes to inter-company deals as well, and in such cases it is usually done so as to win the business. Basically if you are trying to win an outsourcing contract, you don’t want to be seen doing something that the client considers to be “core business”. And so even if you’re fully capable of doing that, you suppress that part of your offering and only provide support. The plan in some cases is to do a Mustafa’s camel, but in most cases that doesn’t succeed.

I’m not offering any comment on whether the Derick Parry strategy of management is good or not. All I’m doing here is to attach this oft-used strategy to a name, one that is mostly forgotten.

Working women, maternity and all that

As I write this, my wife is at work. Though her official gainful fulltime employment starts only a few months later (her employers have deferred her joining date thanks to the baby), she is continuing with her work as Marriage Broker Auntie (which she is now pivoting into something like a “Love Training School“).

In fact, our daughter was barely a week old when my wife decided to get back to business, in her quest to get more people “settled down” and “find partners” (she even brokered a deal from her hospital bed as they tried to induce labour in her). And so I’ve been able to observe, at reasonably close quarters, what it’s like to work while having a tiny baby.

Some times, you think it just doesn’t matter. That she works mainly from home means that she’s always with the baby. There are always sufficiently long periods of time when the baby sleeps when she can do her emails and writing. While sleep is definitely disturbed (by at least two hour-long feeding sessions each night), that she doesn’t engage in other strenuous work means she can handle the work stress.

But then there are the minor irritants. Meetings are a no-no, for example, since she can’t go out, and it doesn’t always make sense to call business acquaintances home. She’s been trying to substitute it with Skype/Facetime calls, but the challenge has been in terms of timing.

Given that some of the people she works with are fairly busy, she needs to pre-schedule calls, and with the baby’s feeding and sleeping schedule being rather uncertain, this is not an easy task. And then there is the problem of having someone take care of the baby during the call, which means the call has to take place at a time when I’m at home.

And so she is on a Skype call now. As she went in for the call, she asked me to handle the baby until it was done, promising that it would be a short call. As it usually happens in such situations, Abheri decided to start crying some two minutes after Priyanka went in for the call.

I tried all my usual tricks. I lay her down on my chest, a technique that usually comforts her in no time, but to no avail (I’ve read about the merits of skin-to-skin contact with the baby but given up on it after she decided to eat my chest hair). I then tried this face-down neck-hold (that I’ve nicknamed “choke slam”), which again usually works in calming her. Again no luck.

Then I smelt shit and thought she was crying because she needed a change of diapers. That didn’t help either. Rocking and singing and swaying and talking – all usually have an immediate effect but none whatsoever today. It was obvious that Abheri was hungry.

So I had to call emergency. Thankfully Priyanka’s Skype call is voice only (or maybe she switched, since she typically prefers video), so she managed to take a little break from the call to take Abheri from my hands. She (Abheri) immediately calmed down – food wasn’t far away.

Priyanka is still on her call, cradling Abheri with one hand against her breast, as Abheri feeds. And Priyanka continues to work.

Major level up in respect for her to see her work this way.

And major envy as well – that she can hold the baby and simultaneously work – nearly four weeks in and I’ve still not mastered the art of holding the baby with one hand, so I can’t work while carrying her!

PS: As for the new law that increases maternity leave, I’m sceptical, since I believe that full-time employment is something that will soon be history. More importantly, the law raises the cost of hiring women, so I’m not sure it will have its intended consequences. Read Priyanka’s excellent analysis here.

Batch Parity

From several sources I’ve heard of this bizarre concept called “batch parity”, where you assume that everyone who joined a particular school or company in a particular year is identical. This leads to people passing up on opportunities because they are not given such parity.

I’ve been hearing of this from way too many sources nowadays so I wrote off a rant, on LinkedIn. Here is an extract:

So while it might be tough for people to stomach, this whole concept of batch parity is, to put it simply, nonsense. At its heart is the assumption that the world is linear, and that after a certain arbitrarily chosen point in time, people ought to all run at the same place.

 

Back to bachelorhood

Starting tonight I’ll be a bachelor once again. For the next nineteen months or so. No it’s not that I’m returning my post graduate diploma and hence getting this downgrade (it’s been a while since I cracked a bad joke here so I’m entitled). It’s that the wife is going away. To get herself an MBA (yes I know that after this she will be better qualified than me since she’ll be getting a proper MBA while i have a post graduate diploma only. Maybe I can retire soon? ).

She’ll be going off to Barcelona tonight. The original plan had me moving there too. But then classic old NED happened and I ended up not looking for a job or assignment there and since it’s not an inexpensive place to stay I’m staying back. Plan to visit her every once in a while. And even though tickets to Europe are prohibitively expensive I now have a place to go to in case I need a break.

But for that I need to first get myself a visa. I guess one of my chief tasks in the next few days will be to get this bit of business done. But then I have my own business.

Regulars on this blog might be aware that I haven’t had formal employment for close to three years now. I freelance as a quAnt consultant – helping companies figure out how to make use of the volumes of data they collect in improving their business decision making. It’s been doing quite okay but my plan is to use the next few months when I don’t have any domestic commitments to see if I can take it to the next level.

It might also be pertinent to mention here that the first bit if business I got for this particular venture was through this blog – the last time I put out a post like this one a long time reader who was looking for quant assistance left a comment here and that led to a rather fruitful assignment. Perhaps mentioning this here might result in a repeat?

Now that I’m blogging more than I used to in the recent past I’ll also be using these pages to keep you updated on the long distanceness. I’ve also noticed that since I last put the update on leaving twitter and Facebook that there’s some more activity here. Keep that flowing and I hope for some good conversations on the comment pages here.