Ratings revisited

Sometimes I get a bit narcissistic, and check how my book is doing. I log on to the seller portal to see how many copies have been sold. I go to the Amazon page and see what are the other books that people who have bought my book are buying (on the US store it’s Ray Dalio’s Principles, as of now. On the UK and India stores, Sidin’s Bombay Fever is the beer to my book’s diapers).

And then I check if there are new reviews of my book. When friends write them, they notify me, so it’s easy to track. What I discover when I visit my Amazon page are the reviews written by people I don’t know. And so far, most of them have been good.

So today was one of those narcissistic days, and I was initially a bit disappointed to see a new four-star review. I started wondering what this person found wrong with my book. And then I read through the review and found it to be wholly positive.

A quick conversation with the wife followed, and she pointed out that this reviewer perhaps reserves five stars for the exceptional. And then my mind went back to this topic that I’d blogged about way back in 2015 – about rating systems.

The “4.8” score that Amazon gives as an average of all the ratings on my book so far is a rather crude measure – since one reviewer’s 4* rating might differ significantly from another reviewer’s.

For example, my “default rating” for a book might be 5/5, with 4/5 reserved for books I don’t like and 3/5 for atrocious books. On the other hand, you might use the “full scale” and use 3/5 as your average rating, giving 4 for books you really like and very rarely giving a 5.

By simply taking an arithmetic average of ratings, it is possible to overstate the quality of a product that has for whatever reason been rated mostly by people with high default ratings (such a correlation is plausible). Similarly a low average rating for a product might mask the fact that it was rated by people who inherently give low ratings.

As I argue in the penultimate chapter of my book (or maybe the chapter before that – it’s been a while since I finished it), one way that platforms foster transactions is by increasing information flow between the buyer and the seller (this is one thing I’ve gotten good at – plugging my book’s name in random sentences), and one way to do this is by sharing reviews and ratings.

From this perspective, for a platform’s judgment on a product or seller (usually it’s the seller, but for products such as AirBnb, information about buyers also matters) to be credible, it is important that they be aggregated in the right manner.

One way to do this is to use some kind of a Z-score (relative to other ratings that the rater has given) and then come up with a normalised rating. But then this needs to be readjusted for the quality of the other items that this rater has rated. So you can think of some kind of a Singular Value Decomposition you can perform on ratings to find out the “true value” of a product (ok this is an achievement – using a linear algebra reference given how badly I suck in the topic).

I mean – it need not be THAT complicated, but the basic point is that it is important that platforms aggregate ratings in the right manner in order to convey accurate information about counterparties.

What sets Uber apart from other marketplaces

While at the gym this evening I was thinking of marketplaces.  To give some context, the reason I went there was that there were too many thoughts running around my head, so I needed to focus on something mindless or something that required so much concentration that I could only hold one other thought in my mind at that point in time. In fact, when you go “under the bar” (do a back squat),  even that one thought will vanish – you need all your physical and mental energy to complete the squat.

Anyway so I was thinking of marketplaces, and marvelling at the kind of impact companies like Uber and Ola have had. They have been an absolute gamechanger in their business in that it has completely changed the way that people and cabs get matched to each other. This was a matching that had been extremely inefficient in the past, but with these apps, they have become better by an order of magnitude. And it is this order of magnitude that sets apart Uber/Ola from other marketplace businesses.

And as I was moving between weights, I had another thought – the trick with Uber/Ola as a marketplace is that it is near impossible to do “side deals”. The ultimate nightmare for a platform/marketplace provider is to let the two sides “discover each other” and conduct further deals “offline”. This can be the bane of services such as dating services, where once you go on your first date (as recommended by OkCupid or Tinder), the dating service need not know anything about your relationship after that! They’ve “lost” you. In fact, talking to someone from the industry recently, I learnt that they do dating rather than marriage since in the former there is the hope of “repeat happy customers”.

It is similar with a service such as Airbnb, where once you’ve located a B&B you like, you can cut airbnb out of the deal from the next time on. Of course availability and stuff matter, but given how much in advance you book, a quick call to check availability is a small cost vis-a-vis the benefit of cutting out the middle man.

The beauty of Uber/Ola, however, is that it is impossible to do deals offline. Yes, after a ride, the driver and the passenger have each other’s numbers. But the next time the passenger wants a ride, the probability that the same driver is in the vicinity and free to give a ride is infinitesimal. So the passenger has to go to the app again. Moreover, it is the app that takes care of the pricing (using GPS, etc.) – something that is impossible to estimate if you try to cut out the app.

So when people say that they are building the “Uber for <some other service>”, in most cases it is not exactly the case – not all marketplace transactions are like Uber. For to be like Uber, you need to be an instant matching mechanism that changes the way the industry fundamentally operates; and you need a mechanism that keeps deals “online” by force.

Chew on it!