Studs and Fighters and Attack and Defence

The general impression in sport is that attack is “stud” and defence is “Fighter“. This is mainly because defence (in any game, pretty much) is primarily about not making errors, and being disciplined. Flamboyance can pay off in attack, when you only need to strike occasionally, but not in defence, where the real payoff comes from being consistent and excellent.

However, attack need not always be stud, and defence need not always be fighter. This is especially true in team sports such as football, where there can be a fair degree of organisation and coaching to get players to coordinate.

This piece in The Athletic (paywalled) gives an interesting instance of how attacking can be fighter, and how modern football is all about fighter attacking. It takes the instance of this weekend’s game between Tottenham Hotspur and Liverpool F.C., which the latter won.

Jack Pitt-Brooke, the author, talks about how Liverpool is fighter in attack because the players are well-drilled in attacking, and practice combination play, or what are known in football as “automisations”.

But in modern football, the opposite is true. The best football, the type played by Pep Guardiola’s Manchester City or Jurgen Klopp’s Liverpool, is the most rigorously planned, drilled and co-ordinated. Those two managers have spent years teaching their players the complex attacking patterns and synchronised movements that allow them to cut through every team in the country. That is why they can never be frustrated by opponents who just sit in and defend, why they are racking up points totals beyond the reach of anyone else.

Jose Mourinho, on the other hand, might be fighter in the way he sets up his defence, but not so when it comes to attacking. He steadfastly refuses to have his teams train attacking automisations. While defences are extremely well drilled, and know exactly how to coordinate, attackers are left to their own forces and creativity. What Mourinho does is to identify a handful of attackers (usually the centre forward and the guy just behind him) who are given “free roles” and are expected to use their own creativity in leading their team’s attacks.

As Pitt-Brooke went on to write in his article,

That, more than anything else, explains the difference between Klopp and Mourinho. Klopp wants to plan his way out of the randomness of football. Mourinho is more willing to accept it as a fact and work around it. So while the modern manager — Klopp, Guardiola, Antonio Conte — coaches players in ‘automisations’, pre-planned moves and patterns, Mourinho does not.

Jurgen Klopp the fighter, and Jose Mourinho the stud. That actually makes sense when you think of how their teams attack. It may not be intuitive, but upon some thought it makes sense.

Yes, attack is also being fighterised in modern sport.

“Principal Component Analysis” for shoes

OK, this is not a technical post. This is more in the realm of “life hacks“. It has everything to do with an observation I made a couple of months back, and how that has helped significantly combat decision fatigue.

I currently own eight pairs of shoes, which is perhaps a lifetime high. And lifetime high means that I was spending a lot of time each time I went out on which shoe to wear.

I have two pairs of open shoes, which I can’t wear for long periods of time, but are convenient in terms of time spent in wearing and taking off. I have two pairs of “semi-formal” ankle-high shoes – one an old pair that refuse to wear out, and another a rather light new one with sneaker bottoms. There are two pairs of “formal shoes”, one black and one brown. And then there are two sneakers – one pair of running shoes and one more general-purpose “fancy” one (this last one looks great with jeans, but atrocious with chinos, which I wear a lot of).

The running shoes have resided in my gym bag for the last nine months, and I use them exclusively indoors in the gym. So they’re “sorted”.

The problem I was facing was that among my seven other pairs of shoes I would frequently get confused on which one to wear. I would have to evaluate the fit with the occasion, how much I would have to stand (I need really soft-bottom shoes if I’ve to stand for a significant period of time), what trousers I was wearing and all such. It became nerve-wracking. Also, our shoe box, which was initially designed for two people and now serves three, placed its own constraints.

So as I somehow cut through the decision fatigue and managed to wear some shoes while stepping out of home, I noticed that a large proportion of the time (maybe 90%) I was wearing only three pairs of shoes. The other shoes were/are still good and I wouldn’t want to give them away, but I found that three shoes would serve the purpose on most occasions.

This is like in principal component analysis, where a small number of “components” (linear combination of variables) predict most of the variance in all the variables put together. In some analysis, you simply use these components rather than all the variables – that rather simplifies the analysis and makes it more tractable.

Since three pairs of shoes would serve me on 90% of the occasions, I decided it was time to take drastic action. I ordered a set of shoe bags from Amazon, and packed up four pairs of shoes and put them in my wardrobe inside. If I really need one of those four, it means I can put the effort at that point in time to go get that from inside. If not, it is rather easy to decide among the three outside on which one to wear (they’re rather dissimilar from each other).

I no longer face much of a decision when I’m stepping out on what shoes to wear. The shoe box has also become comfortable (thankfully the wife and daughter haven’t encroached on my space there even though I use far less space than before). Maybe sometime if I get really bored of these shoes outside, I might swap some of them with the shoes inside. But shoe life is much more peaceful now.

However, I remain crazy in some ways. I still continue to shop for shoes despite owning a lifetime high number of pairs of them. That stems from the belief that it’s best to shop for something when you don’t really need it. I’ll elaborate more on that another day.

Meanwhile I’m planning to extend this “PCA” method for other objects in the house. I’m thinking I’ll start with the daughter’s toys.

Wish me luck.

Distribution of political values

Through Baal on Twitter I found this “Political Compass” survey. I took it, and it said this is my “political compass”.

Now, I’m not happy with the result. I mean, I’m okay with the average value where the red dot has been put for me, and I think that represents my political leanings rather well. However, what I’m unhappy about is that my political views have been all reduced to one single average point.

I’m pretty sure that based on all the answers I gave in the survey, my political leaning across both the two directions follows a distribution, and the red dot here is only the average (mean, I guess, but could also be median) value of that distribution.

However, there are many ways in which people can have a political view that lands right on my dot – some people might have a consistent but mild political view in favour of or against a particular position. Others might have pretty extreme views – for example, some of my answers might lead you to believe that I’m an extreme right winger, and others might make me look like a Marxist (I believe I have a pretty high variance on both axes around my average value).

So what I would have liked instead from the political compass was a sort of heat map, or at least two marginal distributions, showing how I’m distributed along the two axes, rather than all my views being reduced to one average value.

A version of this is the main argument of this book I read recently called “The End Of Average“. That when we design for “the average man” or “the average customer”, and do so across several dimensions,  we end up designing for nobody, since nobody is average when looked at on many dimensions.

External membership of unions

The ostensible reason for the violent crackdown on protesting students at Alighar Muslim University and Jamia Millia Islamia last month was the involvement of “outsiders” in these protests. In both cases, campus authorities claimed that student protestors had been joined by “outsiders” who had gone violent, which forced them to call in the cops.

And then the cops did what cops do – making the protest more violent and increasing the damage all round, both physically and otherwise.

I’m reminded of this case from a few years ago from some automobile company – possibly Maruti. The company had refused to recognise an employee’s union at a new plant they were starting, because of an argument on the membership of non-employees in the unions.

The unions’ argument in that case was that external (non-employee) membership was necessary to provide the organisational and union skills to the union. If I remember correctly, they wanted one third of the union to consist of members who were not employees of the firm. The firm contended that they wouldn’t want to negotiate with outsiders, and so they wouldn’t recognise the union with external members.

I don’t remember how that story played out but this issue of external membership of unions, whether student or employees, is pertinent.

At the fundamental level, unions need to exist because of the balance of power – the dominance in favour of an institution over an individual employee or student is too great to always produce rational outcomes in the short term (in the long term it evens out, but you know what Keynes is supposed to have said). The formation of unions corrects this imbalance since the collection of employees or students can have significant bargaining power vis-a-vis the institution, and negotiations can result in more rational decisions in the short term.

The problem I have is with external membership of unions. The problem there is that external members (who usually provide leadership and “organisation” to the unions) lack skin in the game, and the union’s incentives need not always be aligned with the incentives of the employees or students.

Consider, for example, the protests in the universities last month which became violent. The incentive of the protests would have been to peacefully protest (to register their dissatisfaction with a recent law), and then get back to their business of being students. The students themselves have no incentive to be violent and damage stuff in their own institution, since that will negatively impact their own futures and studies at the institution.

External members of the unions don’t share this incentive – their incentive is in making the union activities (the protest in this case) more impactful. And if the protest creates damage, that can make it more impactful. The external members don’t particularly care about damage to the institution (physical and otherwise), as long as the union’s show of strength is successful.

It is similar in organisations. It is in the interest of both the employees and the management that the company does well, since that means a larger pie that can be split among them. The reason employees organise themselves, and sometimes go on limited strikes, is to ensure that they get what they think is a fair share of the pie.

The problem, of course, is that negotiations aren’t that simple, and they frequently break down. The question is about what to do when that inevitably happens. Each employee has his own threshold in terms of how long to strike, and at what point it makes sense to back down and accept the deal on the table.

In an employee-only union, the average view of the employees (effectively) guides when the strike gets called off and the negotiations end. External members of the union lack skin in the game, and they have a really long threshold on when to back down from the strike. And this makes strikes longer than employees want them to be, which can make the strikes counterproductive for the employees.

One infamous example is of the textile mills in Mumbai in the late 70s, and early 80s. There was massive union action there in those times, with strikes going on for months together. Ultimately the mills packed up and relocated to Gujarat and other places. The employees were the ultimate losers there, either losing their jobs or having to move to another city. If the employees themselves had controlled the union it is likely that they might have come to a settlement sooner or later, and managed to keep their jobs.

In the automobile case I mentioned earlier, if I remember correctly, the union demanded that up to 33% of the membership of the union be comprised of outsiders – a demand the company flatly refused to entertain. Now think about it – if external members control a third of the union, all it takes is one fourth of the employees, acting in concert with the union, for something to happen. And there is a real agency problem there!

Two steps back, one step forward

In his excellent piece on Everton’s failed recruitment strategy (paywalled), Oliver Kay of the Athletic makes an interesting point – that players seldom do well when they move from a bigger club to a smaller club.

During his time in charge at Arsenal, George Graham used to say that the key to building a team was to buy players who were on the way up — or, alternatively, players who were desperate to prove a point — but to avoid those who might see your club as a soft landing, a comfort zone. “Never buy a player who’s taking a step down to join you,” Graham said. “He will act as if he’s doing you a favour.”

This, I guess, is not unique to football alone – it applies to other jobs as well. When someone joins a company that they think they are “too cool for”, they  look at it as a step down, and occasionally behave as if they’re doing the new employer a favour.

One corollary is that working for “the best” can be a sort of lock in for an employee, since wherever he will move from there will be a sort of step down in some way or the other, and that will mean compromises on the part of all parties involved.

Thinking about footballers who have moved from big clubs and still not done badly, I notice one sort of pattern that I call “two steps back and one step forward”. Evidently, I’m basing this analysis on a small number of data points, which might be biased, but let me play management guru and go ahead with my theory.

Basically, if you want to take a “step down” from the best, one way of doing well in the longer term is to take “two steps down” and then later take a step up. The advantage with this approach is that when you take two steps down, you get to operate in an environment far easier than the one you left, and even if you act entitled and take time to adjust you will be able to prove yourself and make an impact in due course.

And at that point in time, when you’ve started making an impact, you are “on the way up”, and can then step up to a club at the next level where you can make an impact.

Players that come to mind that have taken this approach include Jonny Evans, who moved from Ferguson-era Manchester United to West Brom, and then when West Brom got relegated, moved “up” to Leicester. And he’s doing a pretty good job there.

And then there is Xherdan Shaqiri. He made his name as a player at Bayern Munich, and then moved to Inter where he struggled. And then he made what seemed like a shocking move for the time – to Stoke City (of the “cold Thursday night at Stoke” fame) in the Premier League. Finally, last year, after Stoke got relegated from the Premier League, he “stepped up” to Liverpool, where, injuries aside, he’s been doing rather well.

The risk with this two steps down approach, of course, is that sometimes it can fail to come off, and if you don’t make an impact soon enough, you start getting seen as a “two steps down guy”, and even “one step down” can seem well beyond you.

Statistical analysis revisited – machine learning edition

Over ten years ago, I wrote this blog post that I had termed as a “lazy post” – it was an email that I’d written to a mailing list, which I’d then copied onto the blog. It was triggered by someone on the group making an off-hand comment of “doing regression analysis”, and I had set off on a rant about why the misuse of statistics was a massive problem.

Ten years on, I find the post to be quite relevant, except that instead of “statistics”, you just need to say “machine learning” or “data science”. So this is a truly lazy post, where I piggyback on my old post, to talk about the problems with indiscriminate use of data and models.

I had written:

there is this popular view that if there is data, then one ought to do statistical analysis, and draw conclusions from that, and make decisions based on these conclusions. unfortunately, in a large number of cases, the analysis ends up being done by someone who is not very proficient with statistics and who is basically applying formulae rather than using a concept. as long as you are using statistics as concepts, and not as formulae, I think you are fine. but you get into the “ok i see a time series here. let me put regression. never mind the significance levels or stationarity or any other such blah blah but i’ll take decisions based on my regression” then you are likely to get into trouble.

The modern version of this is – everybody wants to do “big data” and “data science”. So if there is some data out there, people will want to draw insights from it. And since it is easy to apply machine learning models (thanks to open source toolkits such as the scikit-learn package in Python), people who don’t understand the models indiscriminately apply it on the data that they have got. So you have people who don’t really understand data or machine learning working with those, and creating models that are dangerous.

As long as people have idea of the models they are using, and the assumptions behind them, and the quality of data that goes into the models, we are fine. However, we are increasingly seeing cases of people using improper or biased data and applying models they don’t understand on top of them, that will have impact that affect the wider world.

So the problem is not with “artificial intelligence” or “machine learning” or “big data” or “data science” or “statistics”. It is with the people who use them incorrectly.

 

Big Data and Fast Frugal Trees

In his excellent podcast episode with EconTalk’s Russ Roberts, psychologist Gerd Gigerenzer introduces the concept of “fast and frugal trees“. When someone needs to make decisions quickly, Gigerenzer says, they don’t take into account a large number of factors, but instead rely on a small set of thumb rules.

The podcast itself is based on Gigerenzer’s 2009 book Gut Feelings. Based on how awesome the podcast was, I read the book, but found that it didn’t offer too much more than what the podcast itself had to offer.

Coming back to fast and frugal trees..

In recent times, ever since “big data” became a “thing” in the early 2010s, it is popular for companies to tout the complexity of their decision algorithms, and machine learning systems. An easy way for companies to display this complexity is to talk about the number of variables they take into account while making a decision.

For example, you can have “fin-tech” lenders who claim to use “thousands of data points” on their prospective customers’ histories to determine whether to give out a loan. A similar number of data points is used to evaluate resumes and determine if a candidate should be called for an interview.

With cheap data storage and compute power, it has become rather fashionable to “use all the data available” and build complex machine learning models (which aren’t that complex to build) for decisions that were earlier made by humans. The problem with this is that this can sometimes result in over-fitting (system learning something that it shouldn’t be learning) which can lead to disastrous predictive power.

In his podcast, Gigerenzer talks about fast and frugal trees, and says that humans in general don’t use too many data points to make their decisions. Instead, for each decision, they build a quick “fast and frugal tree” and make their decision based on their gut feelings about a small number of data points. What data points to use is determined primarily based on their experience (not cow-like experience), and can vary by person and situation.

The advantage of fast and frugal trees is that the model is simple, and so has little scope for overfitting. Moreover, as the name describes, the decision process is rather “fast”, and you don’t have to collect all possible data points before you make a decision. The problem with productionising the fast and frugal tree, however, is that each user’s decision making process is different, and about how we can learn that decision making process to make the most optimal decisions at a personalised level.

How you can learn someone’s decision-making process (when you’ve assumed it’s a fast and frugal tree) is not trivial, but if you can figure it out, then you can build significantly superior recommender systems.

If you’re Netflix, for example, you might figure that someone makes their movie choices based only on age of movie and its IMDB score. So their screen is customised to show just these two parameters. Someone else might be making their decisions based on who the lead actors are, and they need to be shown that information along with the recommendations.

Another book I read recently was Todd Rose’s The End of Average. The book makes the powerful point that nobody really is average, especially when you’re looking a large number of dimensions, so designing for average means you’re designing for nobody.

I imagine that is one reason why a lot of recommender systems (Netflix or Amazon or Tinder) fail is that they model for the average, building one massive machine learning system, rather than learning each person’s fast and frugal tree.

The latter isn’t easy, but if it can be done, it can result in a significantly superior user experience!

What Makes The Athletic Great

In recent times I’ve bought subscriptions to two online media outlets – The Ken and The Athletic. I’d subscribed to the Ken a year ago, and was happy enough with the hit rate of their pieces (I’d find one in two pieces insightful) that I extended my subscription for three years earlier this year.

And since I did that extension, the product has been disappointing. They lost half their team to The Morning Context, a breakaway (and similar) outlet. They decided to expand in South East Asia, and since I have little interest in articles about that reason (at least not enough to pay for the writing), that automatically means less content that interest me. In some senses their quality is slipping. All this together means that I find less than one in five articles in The Ken compelling, and with the frequency of their publication (one article every weekday) I’m pretty disappointed.

Maybe it has to do with Marie Kondo’s popularity, or interest in behavioural economics research about the paradox of choice, but organisations are starting to make minimalism and limitations in inventory a virtue. The Ken started with the aim of “exactly one long form article every day”.

Having less choice, and being minimalistic, is good when this limited choice fits the appetite of the customer. However, if the choice isn’t particularly relevant, then minimalism becomes a bug rather than a feature – the customer doesn’t find what she is looking for and goes on to another outlet.

In that sense, I quite like the model of The Athletic, which I bought a year-long subscription to a year back. The Athletic’s model is just the opposite – massively high volumes with a highly curated personal feed. And maybe they’ve got their curation right, in terms of getting customers to click on the right kind of tags at the time of sign up, but so far I’ve found at least two useful articles on their site every single day since I turned up. And that’s insane value for money!

And that is despite me being interested in exactly one out of the nine sports that The Athletic covers (it’s mostly US-centric, and I don’t follow American sport at all. However I guess I’ll find it useful when I have to follow any controversy in American sport). And I’m interested in a subset of that – I follow one league (English Premier League) and games played by a handful of clubs in that league.

If I compare The Athletic to Netflix (both subscription-driven media outlets with large volumes of content), where the former scores is in its discoverability.

Maybe sport is easier compared to movies/tv shows in order to understand someone’s interests. Maybe it is that The Athletic, right up front, asked me to identify which sports, leagues, authors and teams I’m interested in (Netflix never made an attempt to do that). Maybe it is that The Athletic, with loads of fresh content every single day, is able to serve my preferences far easier than Netflix.

In any case, reading the Athletic makes me think that if I were to run a media outlet some day, I would want to follow that kind of a model – produce lots of content, so that lots of people will be interested in buying subscriptions, and then hope to use superior algorithms to make sure that people can see what they want and not have to cut through too much noise in order to do so!

The Business Standard is innumerate

I guess there is not that much information in the headline here – claiming that a bunch of journalists and editors are innumerate is like saying that the sky is blue. You would be hard-pressed to find journalists and editors who can actually parse numbers, though I must mention that I’ve been lucky enough to work with a few editors who actually understand arithmetic!

So what happened today? Basically in today’s front page, BS journalists (one Vinay Umarji in particular) and editors have displayed an utter lack of understanding on how relative grading and percentiles work. The context is CAT results, which came out yesterday.

(I’ve put a scan since the online version is behind a paywall).

There is information in saying that “number of candidates scoring 100 percentile is lowest in six years”, and the information I take out of that is that the number of test takers this year is the lowest in six years.

And for four of those six years, the numbers were inflated, since double the number of people who were supposed to get 100 percentile actually got 100 percentile. Since CAT percentiles are given to two decimal places, you get 100 percentile if you are in the top 0.005% of all candidates who took the exam. Or – if your “percentile” is higher than 99.995, it gets rounded up to 100.

For three years in the middle, the CAT administrators (usually they’re Quantitative Methods professors at IIMs), for whatever reason, rounded up everyone who got a percentile higher than 99.990 to 100. I’d written about that in my article for Mint three years back.

Coming back, CAT is an exam that follows relative grading. All that someone  has got “100 percentile” means is that they are within the top 0.005% of all candidates who wrote the exam. So if more candidates write the exam, more people will get “100 percentile”. In my time, for example (CAT 2003-4) some 1.3 lakh people had written the exam, so 7 of us got “100 percentile”. Nowadays the number of test takers has gone up, so more people get that score.

And then I found the rest of the article funny in a way as well, trying to do some sort of sociological analysis of the backgrounds of the people who had scored highly in the exam.

PS: The graph doesn’t give out much information (and I don’t know why the 2019 data point is missing there), but I guess it’s been put in there to make the journalists and editors seem more numerate than they are.

 

Experience and Cows

A lot of people make a big deal about experience. If some people (and some companies) are to be believed, the number of years in a job should be the only criterion of what someone needs to be paid and whether they deserve to be promoted.

However, not all experience is created equal. Experience matters when you are learning on the job, and where you learn the patterns that are inherent in your job, and you can over time replace your “slow thinking” about the job with more “fast thinking”.

If you continue to do the same thing in the same way throughout the years of experience, not bothering to figure out why things are done certain ways, and how things can be done better, the experience isn’t of that much use.

I leave it to former Tottenham Hotspur manager Mauricio Pochettino to explain this concept with a beautiful and profound analogy (there’s a video in this link which I’m somehow unable to embed here).

It is like a cow that, every day in 10 years, sees the train cross in front at the same time.

If you ask the cow, ‘what time is the train going to come’, it is not going to know the right answer.

In football, it is the same. Experience, yes, but hunger, motivation, circumstance, everything is so important.

It is unfortunate that the journalist who covered this story for Sky Sports thought this analogy was bizarre. Maybe he has been doing his job reporting on press conferences in the same way a cow sees a train passing by at a particular time every day?