The Office!

For the first time in nearly ten years, I went to an office where I’m employed to work. I’m not going to start going regularly, yet. This was a one off since I had to meet some people who were visiting. On the evidence of today, though, I think i once again sort of enjoy going to an office, and might actually look forward to when I start going regularly again.

Metro

I had initially thought I’d drive to the office, but white topping work on CMH Road means I didn’t fancy driving. Also, the office being literally a stone’s throw away from the Indiranagar Metro Station meant that taking the Metro was an easy enough decision.

The walk to South End Metro station was uneventful, though I must mention that the footpath close to the metro station works after a very long time! However, they’ve changed the gate that’s kept open to enter the station which means that the escalator wasn’t available.

The first order of business upon entering the station was to show my palm to one reader which took my temperature and let me go past. As someone had instructed me on twitter, I put my phone, wallet and watch in my bag as I got it scanned.

Despite not having taken the metro for at least 11 months, the balance on my card remained, and as I swiped it while entering, I heard announcements of a train to Peenya about to enter the station. I bounded up the stairs, only to see that the train was a little distance away.

In 2019, when I had just moved back to Bangalore from London, I had declared that the air conditioning in the Bangalore Metro is the best ever in the city. Unfortunately post-covid protocols mean that the train is kept at a much warmer temperature than usual. So on the way to the office, I kept sweating like a pig.

The train wasn’t too crowded, though. On the green line (till Majestic), everyone was comfortably seated  (despite every alternate seat having been blocked off). I panicked once, though, when a guy seated two seats away from me sneezed. I felt less worried when I saw he was wearing a mask.

The purple line from Majestic was another story. It felt somewhat silly that every alternate seat remained blcoked off when plenty of people were crowding around standing. I must mention, though, that the crowd was nothing like what it normally is. In any case, most of the train emptied out at Vidhana Soudha, and it was a peaceful ride from there on.

40 minute from door to door. Once office starts regularly, I plan to take the metro every day.

The Office

While the office was thinly populated, it felt good being back there. I was meeting several of my colleagues for the first time ever, and it was good to see them in person. We sat together for lunch (ordered from Thai House), and spoke about random things while eating. There was an office boy who, from time to time, ensured that my water glass and bottle were always filled up.

In the evening, one colleague and I went for coffee to the darshini next door. That the coffee was provided in paper cups meant we could safely socially distance from the little crowd at that restaurant. The coffee at this place is actually good – which again bodes well for my office.

And then some usual office-y things happened. I was in a meeting room doing a call with my team when someone else knocked asking if he could use the room. I got into a constant cycle of “watering and dewatering”, something I always do when I’m in an office. The combination of the thin attendance and the office boy, though, meant that there was no need to crowd around the water cooler.

I guess this is what 2020 has done to us. Normally, going to office to work should be the “most normal and boring thing ever”. However, 2020 means that it is now an event worth blogging about. Then again, I don’t need much persuasion to write about anything, do I?

Proper Job

For the first time in over nine years, I’m taking up one of these.

If someone, sometime, were to do a compendium of stories of people whose careers changed because of covid-19, then I might feature in it. To be very honest, my present career change had been in the works for a while now. However, a bunch of things that covid-19 forced upon me this year made it that much easier to take the plunge.

As the more perceptive of you might have observed by now, I quit full time employment to embark on a “portfolio life” in late 2011. Apart from getting control over my own time, this change allowed me to do a lot of interesting things apart from my “core work”, which I took on such that most of the work I did was things I was good at or interested in.

So over the last nine years, apart from doing a lot of very interesting consulting work around data and analytics and AI and ML and “data science” and all that, I did a lot of interesting stuff otherwise as well. I wrote a book. I wrote a column for Mint. I taught at IIMB. I did public policy work for Takshashila.

I met lots of people and had loads of interesting discussions. There were times, yes, when I went into every meeting or catchup with a “sales mindset”, trying to sell something to someone. Thankfully these times were infrequent, and short. At all other times, I enjoyed all these random catchups, without any expectation  that anything come out of it.

My network expanded like crazy during these years. For the first time in my life, I came to be known for something apart from entrance exams. I spent time living in other places. I “followed my wife” when she first went to Barcelona, and then to London. It was all smooth.

In any case, you might be wondering how the pandemic resulted in my transition to employment being easier. The main way in which it has eased this transition is by ruining my carefully constructed lifestyle of the last nine years.

I’ve loved going around and meeting people. On an average, I would meet two to three people a week, for things completely unrelated to work. That has come down to nearly zero in the last nine months.

I had grown used to having massive control of my time and schedule. The prolonged school shutdown has completely sent it for a toss, with shared childcare responsibilities. “If I don’t have control over my time any ways, I might as well take up a job”, went one line of my reasoning.

I sometimes think I have a fear of open offices (I’ve felt this even during my consulting times when some clients have asked me to do “face time” in their offices). I hate having other people looking at my screen when I’m working. Maybe it has to do with some bad bosses / colleagues I’ve had over the years. The pandemic means I start working from the comfort of my home. And by the time I go to an office I will have hopefully settled down in this job.

And speaking of offices, the pandemic has normalised remote or hybrid working to an extent that I applied to jobs without having the constraint that they necessarily need to have an office in Central Bangalore. The company I’m joining – I’m not sure I would have thought of them in a “normal job search”. As it happens, while they’re not primarily based here, they do have a small office not far from Central Bangalore, and I’ll be going there once it reopens.

Then, thanks to the pandemic, I have successfully concluded my jobhunt without stepping out of home. All interviews, with a big range of companies, happened through video conferencing. In terms of my personal experience, Zoom >> Teams >> Meet.

But yeah, the biggest impact of the pandemic has  been on my lifestyle. So many things that I craved, and took as given, have been taken away from my life, that changing lifestyle seems to have become far easier than I had imagined. It’s like the tube strike model. I got shaken out of my earlier local optimum, and that has enabled me to convince myself that this new lifestyle will work.

In any case, I hope this works out. Just before joining, I feel positive, and excited in a good way.

Oh, and I guess I need to add here, and maybe at the beginning of every subsequent post.

All opinions expressed here on this blog are mine, and only mine. They don’t reflect the thoughts or opinions or positions of any organisation(s) that I might be associated with. Also, none of what I write on this blog is to be taken as investment advice. 

 

Record of my publicly available work

A few people who I’ve spoken to as part of my job hunt have asked to see some “detailed descriptions” of work that I’ve done. The other day, I put together an email with some of these descriptions. I thought it might make sense to “document” it in one place (and for me, the “obvious one place” is this blog). So here it is. As you might notice, this takes the form of an email.


I’m putting together links to some of the publicly available work that i’ve done.
1. Cricket
I have a model to evaluate and “tell the story of a cricket match”. This works for all limited overs games, and is based on a dynamic programming algorithm similar to the WASP. The basic idea is to estimate the odds of each team winning at the end of each ball, and then chart that out to come up with a “match story”.
And through some simple rules-based intelligence, the key periods in the game are marked out.
The model can also be used to evaluate the contributions of individual batsmen and bowlers towards their teams’ cause, and when aggregated across games and seasons, can be used to evaluate players’ overall contributions.
Here is a video where I explain the model and how to interpret it:
The algorithm runs live during a game. You can evaluate the latest T20 game here:
Here is a more interactive version , including a larger selection of matches going back in time.
Related to this is a cricket analytics newsletter I actively wrote during the World Cup last year. Most Indians might find this post from the newsletter interesting:
2. Covid-19
At the beginning of the pandemic (when we had just gone under a national lockdown), I had built a few agent based models to evaluate the risk associated with different kinds of commercial activities. They are described here.
Every morning, a script that I have written parses the day’s data from covid19india.org and puts out some graphs to my twitter account  This is a daily fully automated feature.
Here is another agent based model that I had built to model the impact of social distancing on covid-19.
tweetstorm based on Bayes Theorem that I wrote during the pandemic went viral enough that I got invited to a prime time news show (I didn’t go).
3. Visualisations
I used to collect bad visualisations.
I also briefly wrote a newsletter analysing “good and bad visualisations”.
4. I have an “app” to predict which single malts you might like based on your existing likes. This blogpost explains the process behind (a predecessor of ) this model.
5. I had some fun with machine learning, using different techniques to see how they perform in terms of predicting different kinds of simple patterns.
6. I used to write a newsletter on “the art of data science”.
In addition to this, you can find my articles for Mint here. Also, this page on my website  as links to some anonymised case studies.

I guess that’s a lot? In any case, now I’m wondering if I did the right thing by choosing “skthewimp” as my Github username.

Core quants and desk quants on main street

The more perceptive of you might have realised that I’m in the job market.

Over the last one month, my search has mostly be “breadth first” (lots of exploratory conversations with lots of companies), and I’m only now starting to “go deep” into some of them. As part of this process, I need to send out a pitch to a company I’ve been in conversation with regarding what I can do for them.

So I’ve been thinking of how to craft my mandate while keeping in mind that they have an existing data science team. And while I was thinking about this problem, I realised that I can model it like how investment banks (at least one that I worked for) do – in terms of “core quants” and “desk quants”.

I have written about this on my blog before – most “data scientists” in industry are equivalent to what investment banks call “core quants”. They are usually highly technically accomplished people; in many cases they are people who were on an academic path that they left to turn to industry. They do very well in “researchy” environments.

They’re great at running long-gestation-period assignments, working on well defined technical problems and expressing their ideas in code. In general, though (I know I’m massively generalising), they are not particularly close to the business and struggle to deal with the ambiguities that business throws at them from time to time.

What I had mentioned in my earlier post is that “main street” (the American word for “general industry”) lacks “desk quants”. In investment banks, desk quants are attached to trading desks and work significantly closer to the business. They may work less on firmwide or long term strategic projects, but their strength is in blending the models and the markets, and building and making simple tweaks to models so that they remain relevant to the business.

And this is the sort of role in which I’m planning to pitch myself – to all potential employers. That while I’m rather comfortable technically, and all sorts of different modelling techniques, I’m not “deep into tech” and like to work close to the markets. I realise that this analogy will be lost on most people, so I need to figure out a better way of marketing myself. Any ideas will be appreciated.

Over the last month or so I’ve been fairly liberal and using my network to get introductions and references. The one thing I’ve struggled with there is how they describe me as. Most people end up describing me as a “data scientist”, and I’m not sure that’s an accurate description of what I do. Then again, it’s my responsibility to help them figure out how best to describe me. And that’s another thing I’m struggling in. “Desk quant” doesn’t translate well.

Coming back to life

On Sunday, I met a friend for coffee. In normal times that would be nothing extraordinary. What made this extraordinary was that this was the first time since the lockdown started that I was actually meeting a non-family member casually, for a long in-person conversation.

I’m so tired of the three pairs of shorts and five T-shirts that I’ve been wearing every day since the lockdown started that I actually decided to dress up that day. And bothered to take a photo at a signal on the way to meeting him.

We met at a coffee shop in Koramangala, from where we took away coffees and walked around the area for nearly an hour, talking. No handshakes. No other touches. Masks on for most of the time. And outdoors (I’m glad I live in Bangalore whose weather allows you to be outdoors most of the year). Only issue was that wearing a mask and walking and talking for an hour can tire you out a bit.

The next bit of resurrection happened yesterday when I had an in-person business meeting for the first time in three months. Parking the car near these people’s office was easier than usual (less business activity I guess?), though later I found that my windshield was full of bird shit (I had parked under a tree).

For the first time ever while going into this office, I got accosted by a security guard at the entrance, asking where I was headed, taking my temperature and offering me hand sanitiser. Being a first time, I was paranoid enough to use the umbrella I was carrying to operate the lift buttons, and my mask was always on.

There were no handshakes. The room was a bit stuffy and I wasn’t sure if they were using the AC, so I asked for the windows to be opened (later they turned on the AC saying it’s standard practice there nowadays). Again, no handshakes or anything. We kept our masks on for a long time. They offered water in a bottle which I didn’t touch for a long time.

Until one of them suggested we could order in dosas from a rather famous restaurant close to their office (and one that I absolutely love). The dosas presently arrived, and then all masks were off. For the next half hour as the dosas went down it was like we were back in “normal times” again, eating together and talking loudly without masks. I must say I missed it.

I took the stairs down to avoid touching the lift. Walked back to the car (and birdshit-laden windshield) and quickly used hand sanitiser. I hadn’t carried my laptop or notebook for the meeting, and I quickly made notes using the voice notes app of my phone.

Yes, in normal times, a lot of this might appear mundane. But given that we’re now sort of “coming back to life” after a long and brutal lockdown, a lot of this deserves documentation.

Oh, and I’m super happy to meet people now. Given a choice, I prefer outdoors. Write in if you want to meet me.

Meetings from home

For the last eight years, I’ve worked from home with occasional travel to clients’ offices. How occasional this travel has been has mostly depended on how far away the client is, and how insistent they are on seeing my face. Nevertheless, I’ve always made it a point to visit them for any important meetings, and do them in person.

Now, with the Covid-19 crisis, this hybrid model has broken down. Like most other people in the world, I work entirely from home nowadays, even for important meetings.

At the face of this, this seems like a good thing – for example, nowadays, however important a meeting is, the transaction cost is low. An hour long meeting means spending an hour for it (the time taken for prep is separate and hasn’t changed), and there’s no elaborate song-and-dance about it with travel and dressing up and all that.

While this seems far more efficient use of my time, I’m not sure I’m so happy about it. Essentially, I miss the sense of occasion. Now, an important meeting feels no different from an internal meeting with partners, or some trivial update.

Travel to and from an important meeting was a good time to mentally prepare for it, and then take stock of how it was gone. Now, until ten minutes before a meeting, I’m living my life as usual, and the natural boundaries that used to help me prep are also gone.

The other problem with remotely being there in large but important meetings is that it’s really easy to switch off. If you’re not the one who is doing a majority of the talking (or even the listening), it becomes incredibly hard to focus, and incredibly easy to get distracted elsewhere in the computer (it helps if your camera is switched off).

In a “real” physical meeting, however, large the gathering is, it is naturally easy for you to focus (and naturally more difficult to be distracted), and also easier to get involved in the meeting. An online meeting sometimes feels a bit too much like a group discussion, and without visual cues involved, it becomes really hard to butt in and make a point.

So once we are allowed to travel, and to meet, I’m pretty certain that I’ll start travelling a bit for work again. I’ll start with meetings in Bangalore (inter-city travel is likely to be painful for a very long time).

It might involve transaction cost, but a lot of the transaction cost gets recovered in terms of collateral benefits.

Ganesha Workflow

I have a problem with productivity. It’s because I follow what I call the “Ganesha Workflow”.

Basically there are times when I “get into flow”, and at those times I ideally want to just keep going, working ad infinitum, until I get really tired and lose focus. The problem, however, is that it is not so easy to “get into flow”. And this makes it really hard for me to plan life and schedule my day.

So where does Ganesha come into this? I realise that my workflow is similar to the story of how Ganesha wrote the Mahabharata.

As the story goes, Vyasa was looking for a scribe to write down the Mahabharata, which he knew was going to be a super-long epic. And he came across Ganesha, who agreed to write it all down under one condition – that if Vyasa ever stopped dictating, Ganesha would put his pen down and the rest of the epic would remain unwritten.

So Ganesha Workflow is basically the workflow where as long as you are going, you go strong, but the moment you have an interruption, it is really hard to pick up again. Putting it another way, when you are in Ganesha Workflow, context switches are really expensive.

This means the standard corporate process of drawing up a calendar and earmarking times of day for certain tasks doesn’t really work. One workaround I have made to accommodate my Ganesha Workflow is that I have “meeting days” – days that are filled with meetings and when I don’t do any other work. On other days I actively avoid meetings so that my workflow is not disturbed.

While this works a fair bit, I’m still not satisfied with how well I’m able to organise my work life. For one, having a small child means that the earlier process of hitting “Ganesha mode” at home doesn’t work any more – it’s impossible to prevent context switches on the child’s account. The other thing is that there is a lot more to coordinate with the wife in terms of daily household activities, which means things on the calendar every day. And those will provide an interruption whether I like it or not.

I’m wondering what else I can do to accommodate my “Ganesha working style” into “normal work and family life”. If you have any suggestions, please let me know!

Context switches and mental energy

Back in college, whenever I felt that my life needed to be “resurrected”, I used to start by cleaning up my room. Nowadays, like most other things in the world, this has moved to the virtual world as well. Since I can rely on the wife (:P) to keep my room “Pinky clean” all the time, resurrection of life nowadays begins with going off social media.

My latest resurrection started on Monday afternoon, when I logged off twitter and facebook and linkedin from all devices, and deleted the instagram app off my phone. My mind continues to wander, but one policy decision I’ve made is to both consume and contribute content only in the medium or long form.

Regular readers of this blog might notice that there’s consequently been a massive uptick of activity here – not spitting out little thoughts from time to time on twitter means that I consolidate them into more meaningful chunks and putting them here. What is interesting is that consumption of larger chunks of thought has also resulted in greater mindspace.

It’s simple – when you consume content in small chunks – tweets or instagram photos, for example, you need to switch contexts very often. One thought begins and ends with one tweet, and the next tweet is something completely different, necessitating a complete mental context switch. And, in hindsight, I think that is “expensive”.

While the constant stream of diverse thoughts is especially stimulating (and that is useful for someone like me who’s been diagnosed with ADHD), it comes with a huge mental cost of context switch. And that means less energy to do other things. It’s that simple, and I can’t believe I hadn’t thought of it so long!

I still continue to have my distractions (my ADHD mind won’t allow me to live without some). But they all happen to be longish content. There are a few blog posts (written by others) open in my browser window. My RSS feed reader is open on my browser for the first time since possibly my last twitter break. When in need of distraction, I read chunks of one of the articles that’s open (I read one article fully until I’ve finished it before moving on to the next). And then go back to my work.

While this provides me the necessary distraction, it also provides the distraction in one big chunk which doesn’t take away as much mental energy as reading twitter for the same amount of time would.

I’m thinking (though it may not be easy to implement) that once I finish this social media break, I’ll install apps on the iPad rather than having them on my phone or computer. Let’s see.

Housewife Careers

This is something I’ve been wanting to write about for a very long time, but have kept putting it off. The ultimate trigger for writing this is this article about women with children in Amazon asking for backup child care at work. Since this hits rather close home, this is a good enough trigger to write.

Quoting the article:

“Everyone wants to act really tough and pretend they don’t have human needs,” says Kristi Coulter, who worked in various roles at Amazon for almost 12 years and observed that many senior executives had stay-at-home wives.

(emphasis mine)

While this might be true of Amazon (though not necessarily for other large tech companies), it is true for other careers as well. The nature of the job means that it is impossible to function if you even have partial child-care responsibilities. And that implies that the only way you can do this job is if you have a spouse whose full time job is bringing up the kids.

Without loss of generality (considering that in most cases it’s the women who give up their careers for child-rearing), we can call these jobs “housewife jobs”.

Housewife jobs are jobs where you can do a good job if an only if you have a spouse who spends all her time taking care of the kids. 

The main feature (I would say it is a bug, but whatever) of such a job is usually long work hours that require you to “overlap both ways” – both leave home early in the morning and return late every night, implying that even if you have to drop your kid to day care, it is your spouse who has to do so. And as I’ve found from personal experience, it is simply not possible to work profitably when you have both child-dropping and child-picking-up duties on a single day (unless you have zero commute, like I’ve had for the last eight months).

Housewife jobs also involve lots of travel. Whether it is overnight or not doesn’t matter, since you are likely to be away early mornings and late evenings at least, and this means (once again) that the spouse has to pick up the slack.

Housewife jobs also involve a lot of pressure, which means that even when you are done with work and want to relax with the kids, you are unable to take your mind off work. So it turns out to be rather unprofitable time with the kids – so you might as well spend that working. Which again means the spouse picks up the slack.

Sometimes a job may not be inherently stressful or require long hours, but might be housewife because the company is led by a bunch of people with housewives (the article linked above claims this about Amazon). What this means is that when there is a sufficient number of (mostly) men in senior management who have housewives taking care of kids, their way of working percolates through the culture of the organisation.

These organisations are more likely to demand “facetime” (not the Apple variety). They are more likely to value input more than output (thus privileging fighter work?). And soon people without housewives get crowded out of such organisations, making it even more housewife organisations.

Finally, you may argue that I’ve used UK-style nurseries as the dominant child care mechanism in my post (these usually run 8-6), and that it might be possible to hedge the situations completely with 24/7 nannies or Singapore-style “helpers”. Now, even with full time child care, there are some emergencies that occur from time to time which require the presence of at least one parent. And it can’t be the same parent providing that presence all the time. So if one of the parents is in a “housewife job”, things don’t really work out.

I guess it is not hard to work out a list of jobs or sectors which are inherently “housewife”. Look at where people quit once they have kids. Look at where people quit once they get married. Look at jobs that are staffed by rolling legions of fresh graduates (if you don’t have a kid, you don’t need a housewife).

The scary realisation I’m coming to is that most jobs are housewife jobs, and it is really not easy being a DI(>=1)K household.

Just Plot It

One of my favourite work stories is from this job I did a long time ago. The task given to me was demand forecasting, and the variable I needed to forecast was so “micro” (this intersection that intersection the other) that forecasting was an absolute nightmare.

A side effect of this has been that I find it impossible to believe that it’s possible to forecast anything at all. Several (reasonably successful) forecasting assignments later, I still dread it when the client tells me that the project in question involves forecasting.

Another side effect is that the utter failure of standard textbook methods in that monster forecasting exercise all those years ago means that I find it impossible to believe that textbook methods work with “real life data”. Textbooks and college assignments are filled with problems that when “twisted” in a particular way easily unravel, like a well-tied tie knot. Industry data and problems are never as clean, and elegance doesn’t always work.

Anyway, coming back to the problem at hand, I had struggled for several months with this monster forecasting problem. Most of this time, I had been using one programming language that everyone else in the company used. The code was simultaneously being applied to lots of different sub-problems, so through the months of struggle I had never bothered to really “look at” the data.

I must have told this story before, when I spoke about why “data scientists” should learn MS Excel. For what I did next was to load the data onto a spreadsheet and start looking at it. And “looking at it” involved graphing it. And the solution, or the lack of it, lay right before my eyes. The data was so damn random that it was a wonder that anything had been forecast at all.

It was also a wonder that the people who had built the larger model (into which my forecasting piece was to plug in) had assumed that this data would be forecast-able at all (I mentioned this to the people who had built the model, and we’ll leave that story for another occasion).

In any case, looking at the data, by putting it in a visualisation, completely changed my perspective on how the problem needed to be tackled. And this has been a learning I haven’t let go of since – the first thing I do when presented with data is to graph it out, and visually inspect it. Any statistics (and any forecasting for sure) comes after that.

Yet, I find that a lot of people simply fail to appreciate the benefits of graphing. That it is not intuitive to do with most programming languages doesn’t help. Incredibly, even Python, a favoured tool of a lot of “data scientists”, doesn’t make graphing easy. Last year when I was forced to use it, I found that it was virtually impossible to create a PDF with lots of graphs – something that I do as a matter of routine when working on R (I subsequently figured out a (rather inelegant) hack the next time I was forced to use Python).

Maybe when you work on data that doesn’t have meaningful variables – such as images, for example – graphing doesn’t help (since a variable on its own has little information). But when the data remotely has some meaning – sales or production or clicks or words, graphing can be of immense help, and can give you massive insight on how to develop your model!

So go ahead, and plot it. And I won’t mind if you fail to thank me later!