Profit and politics

Earlier today I came across this article about data scientists on LinkedIn that I agreed with so much that I started wondering if it was simply a case of confirmation bias.

A few sentences (possibly taken out of context) from there that I agree with:

  • Many large companies have fallen into the trap that you need a PhD to do data science, you don’t.
  • There are some smart people who know a lot about a very narrow field, but data science is a very broad discipline. When these PhD’s are put in charge, they quickly find they are out of their league.
  • Often companies put a strong technical person in charge when they really need a strong business person in charge.
  •  I always found the academic world more political than the corporate world and when your drive is profits and customer satisfaction, that academic mindset is more of a liability than an asset.

Back to the topic, which is the last of these sentences. This is something I’ve intended to write for 5-6 years now, since the time I started off as an independent management consultant.

During the early days I took on assignments from both for-profit and not-for-profit organisations, and soon it was very clear that I enjoyed working with for-profit organisations a lot more. It wasn’t about money – I was fairly careful in my negotiations to never underprice myself. It was more to do with processes, and interactions.

The thing in for-profit companies is that objectives are clear. While not everyone in the company has an incentive to increase the bottom-line, it is not hard to understand what they want based on what they do.

For example, in most cases a sales manager optimises for maximum sales. Financial controllers want to keep a check on costs. And so on. So as part of a consulting assignment, it’s rather easy to know who wants what, and how you should pitch your solution to different people in order to get buy-in.

With a not-for-profit it’s not that clear. While each person may have their own metrics and objectives, because the company is not for profit, these objectives and metrics need not be everything they’re optimising for.

Moreover, in the not for profit world, the lack of money or profit as an objective means you cannot differentiate yourself with efficiency or quantity. Take the example of an organisation which, for whatever reason, gets to advice a ministry on a particular subject, and does so without a fee or only for a nominal fee.

How can a competitor who possibly has a better solution to the same problem “displace” the original organisation? In the business world, this can be done by showing superior metrics and efficiency and offering to do the job at a lower cost and stuff like that. In the not-for-profit setup, you can’t differentiate on things like cost or efficiency, so the only thing you can do is to somehow provide your services in parallel and hope that the client gets it.

And then there is access. If you’re a not-for-profit consultant who has a juicy project, it is in your interest to become a gatekeeper and prevent other potential consultants from getting the same kind of access you have – for you never know if someone else who might get access through you might end up elbowing you out.

Freelancing and transaction costs

In the six years of running my own consulting business, I’d forgotten about an essential part that you need to endure as part of a job – piecemeal work. It is fairly often when you’re working for someone else that you get work that is so tiny or insignificant that you can hardly take ownership of it. The best strategy for dealing with it is to quietly get it over with and hope you won’t get such stuff again.

However, sometimes you can get caught in a rut of continuously getting this kind of work, and start wondering what you actually signed up for. And this is one thing I hadn’t expected to encounter when I got back to full time working earlier this year.

Thinking about why I never had to encounter such stuff during my consulting life, I realised there’s a fairly simple explanation – transaction costs.

Being a consultant is high transaction cost business. Every time you need to take on a new piece of work, you need to go through the charade of negotiating specifics with the client, pricing and drawing up a contract. All put together, the effort is not insignificant.

Moreover, in the line of work that I used to do, there was this massive overhead cost of understanding, cleaning and getting comfortable with the client’s data  – the effort involved in that meant that after a particular point in time I stopped taking work that wasn’t chunky enough. For a while I started refusing such work, but then got smarter and started pricing myself out of such work (though some clients were generous enough to meet that price to get their little tasks done – effectively I’d passed on the transaction costs to them).

The downside of this, of course, was that there was a fair amount of money I could have made taking up small works which I didn’t since the transaction cost was too high – this can be thought of as potential lost revenues. The upside was that whatever work I did was of high quality and (hopefully) made a big impact on the client’s business.

In the nature of the firm, Ronald Coase wrote that the purpose of the corporation was that transaction cost of dealing with co-workers can be eliminated. But then, I realise that sometimes this transaction cost can also be a good thing!

Oh, and obligatory plug here – my book Between the buyer and the seller deals with transaction costs, among other things. It’s available for sale (both in print and digital) on Amazon.

 

Letters to my wife

As I turned Thirty Three yesterday, my wife dug up some letters (emails to be precise) I’d written to her over the years and compiled them for me, urging me to create at “Project Thirty Four” (on the lines of my Project Thirty). What is pleasantly surprising is that I’ve actually managed to make a life plan for myself, and execute it (surprising considering I don’t consider myself to be too good a planner in general).

In February 2011, after having returned from a rather strenuous work trip to New York, this is what I had to say (emphasis added later, typos as in original):

For me steady state is when I’ll be doing lots of part-time jobs, consulting gigs, where I’m mostly owrking from home, getting out only to meet people, getting to meet a lot of people (somethign taht doesn’t happen in this job), having fun in the evenings and all that

I wrote this six months before I exited my last job, and it is interesting that it almost perfectly reflects my life nowadays (except for the “have fun in the evenings” bit, but that can be put down to being long distance).

I’ve just started a part time job. I have a couple of consulting gigs going. I write for a newspaper (and get paid for it). I mostly work from home. I’ve had one “general catch up” a day on an average (this data is from this Quantified Life sheet my wife set up for me).

A week later I had already started planning what I wanted to do next. Some excerpts from a letter I wrote in March 2011:

Ok so I plan to start a business. I don’t know when I’ll start, but I’m targeting sometime mid 2012.

I want to offer data consultancy services.

Basically companies will have shitloads of data that they can’t make sense of. They need someone who is well-versed in working with and looking at data, who can help them make sense of all that they’ve got. And I’m going to be that person.

Too many people think of data analysis as a science and just through at data all the analytical and statistical weapons that they’ve got. I believe that is the wrong approach and leads to spurious results that can be harmful for the client’s business.

However, I think it is an art. Making sense of data is like taming a pet dog. There is a way you communicate with it. There is a way you make it do tricks (give you the required information). And one needs to proceed slowly and cautiously in order to get the desired results.

I think of myself as a “semi-quant”. While I am well-versed in all the quantitative techniques in data analysis and financial modeling, I’m also deeply aware that using quantitative tools indiscriminately can lead to mismanagement of risks, which can be harmful to the client. I believe in limited and “sustainable” use of quantitative tools, so that it can lead without misleading.

 

My past experience with working with data is that data analysis can be disruptive. I don’t promise results that will be of particular liking for the client – but I promise that what I diagnose is good for the client’s business. When you dig through mountains of data, you are bound to get some bitter pills. I expect my clients to handle the bad news professionally and not shoot the messenger.

I don’t promise to find a “signal” in every data set that I’m given. There are chances that what I’m working with is pure noise, and in case I find that, I’ll make efforts to prove that to the client (I think that is also valuable information).

And these paragraphs, written a full year before I started out doing what I’m doing now, pretty much encapsulate what I’m doing now. Very little has changed over nearly five years! I feel rather proud of myself!

And a thousand thanks to my wife for picking out these emails I had sent her and showing me that I can work to a plan.

Now on to making Project Thirty Four, which I hope to publish by the end of today, and hope to execute by the end of next year.

Back to bachelorhood

Starting tonight I’ll be a bachelor once again. For the next nineteen months or so. No it’s not that I’m returning my post graduate diploma and hence getting this downgrade (it’s been a while since I cracked a bad joke here so I’m entitled). It’s that the wife is going away. To get herself an MBA (yes I know that after this she will be better qualified than me since she’ll be getting a proper MBA while i have a post graduate diploma only. Maybe I can retire soon? ).

She’ll be going off to Barcelona tonight. The original plan had me moving there too. But then classic old NED happened and I ended up not looking for a job or assignment there and since it’s not an inexpensive place to stay I’m staying back. Plan to visit her every once in a while. And even though tickets to Europe are prohibitively expensive I now have a place to go to in case I need a break.

But for that I need to first get myself a visa. I guess one of my chief tasks in the next few days will be to get this bit of business done. But then I have my own business.

Regulars on this blog might be aware that I haven’t had formal employment for close to three years now. I freelance as a quAnt consultant – helping companies figure out how to make use of the volumes of data they collect in improving their business decision making. It’s been doing quite okay but my plan is to use the next few months when I don’t have any domestic commitments to see if I can take it to the next level.

It might also be pertinent to mention here that the first bit if business I got for this particular venture was through this blog – the last time I put out a post like this one a long time reader who was looking for quant assistance left a comment here and that led to a rather fruitful assignment. Perhaps mentioning this here might result in a repeat?

Now that I’m blogging more than I used to in the recent past I’ll also be using these pages to keep you updated on the long distanceness. I’ve also noticed that since I last put the update on leaving twitter and Facebook that there’s some more activity here. Keep that flowing and I hope for some good conversations on the comment pages here.

Countercyclical business

I realize being a freelance management consultant is countercyclical business. For two years in succession, I’ve had a light March – both years I’ve ended up finishing projects in Jan/Feb. With March being the end of the Indian financial year, most companies are loathe to commit additional spending in March, and it is a bad time to start new projects!

This is counter-cyclical because most other businesses end up having a bumper March, since they have end-of-year targets, and with a short sales cycle, they push their salespersons hard to achieve this target in March!

Coase

In the wake of the passing of Ronald Coase, two incidents, both professional. The first was with an established company to whom I suggested a partnership – they are in a space where I don’t have much skill, but have access to companies who I would love to sell to, and they don’t have my skill and our skills are complementary. So I reached out to them (through common contacts) suggesting that we could work together. They came back to me saying they would love to work with me, but would want me to join them as an employee.

The second was an incoming lead. This was a rather small company doing something similar to what I’m doing but with bigger ideas. They want me to join this “innovation hub” they are trying to create. This is a loose federation they are creating including professionals from various fields. Nobody is obliged to work full time for the hub, but this gives people an opportunity to get together and work together on projects where their respective expertise can combine well.

As the more perceptive of you who would have read every Coase obituary in the last two weeks would have figured out, the piece of work that Coase is most well known for is about the theory of the firm. The question is rather simple – why should you and I get together and form a firm if we have to work together, if we can remain independent and just come together for projects. The answer lies in transaction costs.

The advantage of coming together as a firm is that you negotiate only once. Let us suppose you are a graphic designer and I’m a data scientist. If we decide to work together on a visualization project, how do we decide how much you get and how much I get? We will need to negotiate. Let’s say we negotiate and agree on a price. And complete a project and split the spoils. What would happen the next time we were to bid for a project? We will need to negotiate again on how we will share the spoils.

If on the other hand we were to form a partnership firm, then for every project that we do, our respective share is fixed! Thus we don’t have to negotiate before every single projects. Thus, firms exist so that you don’t have to repeatedly negotiate.

However, there is a downside to this. What if I form a firm with a graphic designer, and then we see a significant opportunity in projects that involve a lot of analysis but little visualization? In that case, I have no use of my partner, and would loathe to pay him his share of the profits. Or consider if I were to somehow become much better at my job, while my partner stagnates. There is little I can do, for we’ve been locked in into the financial arrangement.

These are only some of the complications that arise when you need to decide whether you want to become a firm. I just thought it is pertinent that I’m having some of these dilemmas (employee versus consultant versus partner versus member of federation) in the few weeks after Coase’s passing.

Fighterization of food

One of the topics that I’d introduced on my blog not so long ago was “fighterization“. The funda was basically about how professions that are inherently stud are “fighterzied” so that a larger number of people can participate in it, and a larger number of people can be served. In the original post, I had written about how strategy consulting has completely changed based on fighterization.

After that, I pointed out about how processes are set – my hypothesis being that the “process” is something that some stud would have followed, and which some people liked because of which it became a process. And more recently, I wrote about the fighterization of Carnatic music, which is an exception to the general rule. Classical music has not been fighterized so as to enable more people to participate, or to serve a larger market. It has naturally evolved this way.

And even more recently, I had talked about how “stud instructions” (which are looser, and more ‘principles based’) are inherently different from “fighter instructions” (which are basically a set of rules). Ravi, in a comment on Mohit‘s google reader shared items, said it’s like rule-based versus principles-based regulation.

Today I was reading this Vir Sanghvi piece on Lucknowi cuisine, which among other things talks about the fact that it is pulao that is made in Lucknow, and now biryani; and about the general declining standards at the Taj Lucknow. However, the part that caught my eye, which has resulted in this post with an ultra-long introduction was this statement:

The secret of good Lucknowi cooking, he said, is not the recipe. It is the hand. A chef has to know when to add what and depending on the water, the quality of the meat etc, it’s never exactly the same process. A great chef will have the confidence to improvise and to extract the maximum flavour from the ingredients.

This basically states that high-end cooking is basically a stud process. That the top chefs are studs, and can adapt their cooking and methods and styles to the ingredients and the atmosphere in order to churn out the best possible product.You might notice that most good cooks are this way. There is some bit of randomness or flexibility in the process that allows them to give out a superior product. And a possible reason why they may not be willing to give out their recipes even if they are not worried about their copyright is that the process of cooking is a stud process, and is hence not easily explained.

Publishing recipes is the attempt at fighterization of cooking. Each step is laid down in stone. Each ingredient needs to be exactly measured (apart from salt which is usually “to taste”). Each part of the process needs to be followed properly in the correct order. And if you do everything perfectly,  you will get the perfect standardized product.

Confession time. I’ve been in Gurgaon for 8 months and have yet to go to Old Delhi to eat (maybe I should make amends this saturday. if you want to join me, or in fact lead me, leave a comment). The only choley-bhature that I’ve had has been at Haldiram’s. And however well they attempt to make it, all they can churn out is the standardized “perfect” product. The “magic” that is supposed to be there in the food of Old Delhi is nowhere to be seen.

Taking an example close to home, my mother’s cooking can be broadly classified into two. One is the stuff that she has learnt from watching her mother and sisters cook. And she is great at making all of these – Bisibelebhath and masala dosa being her trademark dishes (most guests usually ask her to make one of these whenever we invite them home for a meal). She has learnt to make these things by watching. By trying and erring. And putting her personal touch to it. And she makes them really well.

On the other hand, there are these things that she makes by looking at recipes published in Women’s Era. Usually she messes them up. When she doesn’t, it’s standardized fare. She has learnt to cook them by a fighter process. Though I must mention that the closer the “special dish” is to traditional Kannadiga cooking (which she specializes in), the better it turns out.

Another example close to home. My own cooking. Certain things I’ve learnt to make by watching my mother cook. Certain other things I’ve learnt from this cookbook that my parents wrote for me before I went to England four years ago. And the quality of the stuff that I make, the taste in either case, etc. is markedly different.

So much about food. Coming to work, my day job involves fighterization too. Stock trading is supposed to be a stud process. And by trying to implement algorithmic trading, my company is trying to fighterize it. The company is not willing to take any half-measures in fighterization, so it is recruiting the ultimate fighter of ’em all – the computer – and teaching it to trade.

Preliminary reading on studs and fighters theory:

Studs and Fighters

Extending the studs and fighters theory