## A one in billion trillion event

It seems like capital markets quants have given up on the lognormal model for good, for nobody described Facebook’s stock price drop last Thursday as a “one in a billion trillion event”. For that is the approximate probability of it happening, if we were to assume a lognormal model of the market.

Without loss of generality, we will use 90 days trailing data to calculate the mean and volatility of stock returns. As of last Thursday (the day of the fall), the daily mean returns for FB was 0.204%, or an annualised return of 51.5% (as you can see, very impressive!). The daily volatility in the stock (using a 90-day lookback period again) was 1.98%, or an annualised volatility of 31.4% . While it is a tad on the higher side, it is okay considering the annual return of 51.5%.

Now, traditional quantitative finance models have all used a lognormal distribution to represent stock prices, which implies that the distribution of stock price returns is normal. Under such an assumption, the likelihood of a 18.9% drop in the value of Facebook (which is what we saw on Thursday) is very small indeed.

In fact, to be precise, when the stock is returning 0.204% per day with a vol of 1.98% per day, the an 18.9% drop is a 9.7 sigma event. In other words, if the distribution of returns were to be normal, Thursday’s drop is 9 sigmas away from normal. Remember that most quality control systems (admittedly in industrial settings, where faults are indeed governed by a nearly normal distribution) are set for a six sigma limit.

Another way to look at Thursday’s 9.7 sigma event is that again under the normal distribution, the likelihood of seeing this kind of a fall in a day is \$math ~10^{-21}\$. Or one in a billion trillion. In terms of the number of trading days required for such a fall to arrive at random, it is of the order of a billion billion years, which is an order of magnitude higher than the age of the universe!

In fact, when the 1987 stock market crash (black monday) happened, this was the defence the quants gave for losing their banks’ money – that it was an incredibly improbable event. Now, my reading of the papers nowadays is sketchy, and I mostly consume news via twitter, but I haven’t heard a single such defence from quants who lost money in the Facebook crash. In fact, I haven’t come across too many stories of people who lost money in the crash.

Maybe it’s the power of diversification, and maybe indexing, because of which Facebook is now only a small portion of people’s portfolios. A 20% drop in a stock that is even 10% of your portfolio erodes your wealth by 2%, which is tolerable. What possibly caused traders to jump out of windows on Black Monday was that it was a secular drop in the US market then.

Or maybe it’s that the lessons learnt from Black Monday have been internalised, and included in models 30 years hence (remember that concepts such as volatility smiles and skews, and stochastic volatility, were introduced in the wake of the 1987 crash).

That a 20% drop in one of the five biggest stocks in the United States didn’t make for “human stories” or stories about “one in a billion billion event” is itself a story! Or maybe my reading of the papers is heavily biased!

PostScript

Even after the spectacular drop, the Facebook stock at the time of this update is trading at 168.25, a level last seen exactly 3 months ago – on 26th April, following the last quarter results of Facebook. That barely 3 months’ worth of earnings have been wiped out by such a massive crash suggests that the only people to have lost from the crash are traders who wrote out of the money puts.

## Algorithmic curation

When I got my first smartphone (a Samsung Galaxy Note 2) in 2013, one of the first apps I installed on it was Flipboard. I’d seen the app while checking out some phones at either the Apple or Samsung retail outlets close to my home, and it seemed like a rather interesting idea.

For a long time, Flipboard was my go-to app to check the day’s news, as it conveniently categorised news into “tech”, “business” and “sport” and learnt about my preferences and fed me stuff I wanted. And then after some update, it suddenly stopped working – somehow it started serving too much stuff I didn’t want to read about, and when I tuned (by “following” and “unfollowing” topics) my feed, it progressively got worse.

I stopped using it some 2 years back, but out of curiosity started using it again recently. While it did throw up some nice articles, there is too much unwanted stuff in the app. More precisely, there’s a lot of “clickbaity” stuff (“10 things about Narendra Modi you would never want to know” and the like) in my feed, meaning I have to wade through a lot of such articles to find the occasional good ones.

(Aside: I dedicate about half a chapter to this phenomenon in my book. The technical term is “congestion”. I talk about it in the context of markets in relationships and real estate)

Flipboard is not the only one. I use this app called Pocket to bookmark long articles and read later. A couple of years back, Pocket started giving “recommendations” based on what I’d read and liked. Initially it was good, and mostly curated from what my “friends” on Pocket recommended. Now, increasingly I’m getting clickbaity stuff again.

I stopped using Facebook a long time before they recently redesigned their newsfeed (to give more weight to friends’ stuff than third party news), but I suspect that one of the reasons they made the change was the same – the feed was getting overwhelmed with clickbaity stuff, which people liked but didn’t really read.

Basically, there seems to be a widespread problem in a lot of automatically curated news feeds. To put it another way, the clickbaity websites seem to have done too well in terms of gaming whatever algorithms the likes of Facebook, Flipboard and Pocket use to build their automated recommendations.

And more worryingly, with all these curators starting to do badly around the same time (ok this is my empirical observation. Given few data points I might be wrong), it suggests that all automated curation algorithms use a very similar algorithm! And that can’t be a good thing.

## Commenting on social media

While I’m more off than on in terms of my consumption of social media nowadays, I find myself commenting less and less nowadays.

I’ve stopped commenting on blogs because I primarily consume them using an RSS reader (Feedly) on my iPad, and need to click through and use my iPad keyboard to leave comments, a hard exercise. And comments on this blog make me believe that it’s okay to not comment on blogs any more.

On Facebook, I leave the odd comment but find that most comments add zero value. “Oh, looking so nice” and “nice couple” and things like that which might flatter some people, but which make absolutely no sense once you start seeing through the flattery.

So the problem on Facebook is “congestion”, where a large number of non-value-adding comments may crowd out the odd comment that actually adds value, so you as a value-adding-commentor decide to not comment at all.

The problem on LinkedIn is that people use it mostly as a medium to show off (that might be true of all social media, but LinkedIn is even more so), and when you leave a comment there, you’re likely to attract a large number of show-offers who you are least interested in talking to. Again, there’s the Facebook problem here in terms of congestion. There is also the problem that if you leave a comment on LinkedIn, people might think you’re showing off.

Twitter, in that sense, is good in that you can comment and selectively engage with people who reply to your comment (on Facebook, when all replies are in one place, such selective engagement is hard, and you can offend people by ignoring them). You can occasionally attract trolls, but with a judicious combination of ignoring, muting and blocking, those can be handled.

However, in my effort to avoid outrage (I like to consume news but don’t care about random people’s comments on it), I’ve significantly pruned my following list. Very few “friends”. A few “twitter celebrities”. Topic-specific studs. The problem there is that you can leave comments, but when you see that nobody is replying to them, you lose interest!

So it’s Jai all over the place.

One of the reasons that sparked my departure from social media platforms such as Facebook and Twitter two weeks back was an argument with my wife where she claimed that Twitter had made me too negative, and highly prone to trolling (even in “real life”). Accepting a challenge from her, I offered to go through my tweets over the last few months, and identify those that were negative. I also offered to perform a similar exercise with my blog.

I started off with the intention to go through tweets in the last one year and delete anything that was negative or “troll-y”. I allocated myself an hour to accomplish this, along with a similar exercise for my blog.

I must have spent fifty minutes going through my twitter feed, and didn’t manage to go back more than two months. I was surprised by my own sheer volume of tweeting. What was more surprising was the amazing lack of insight in most of those tweets – there were horrible PJs that I’d cracked just because I could, there were random replies to other people which didn’t add any kind of value, there was outrage about the lack of outrage and some plain banal life stuff (apart from some downright trolly stuff which I deleted).

It made for extremely painful reading, and I could hardly recognise myself from my own tweets. Apart from some personal markers, I would find it hard to recognise most of these tweets as my own if they were to be presented to me a few months later. It was a clear indication that it was time to exit twitter (though since I have a rather kickass username there I’m not deleting my account).

The ten minutes I spent that day going through this blog, however, was a sheer delight. I did end up deleting a couple of outragey posts (both of which were essentially collections of tweets which I’d collated for posterity), but most of my posts were mostly sheer delight! There was some kind of insight in each of my posts, and I’d lie if I were to say that I’m not proud of what I’ve written.

It’s not that I’ve not written shit on this blog (or its predecessor), having written posts as late as 2008 which I’m definitely not proud of. What I’ve noticed, however, is that I’ve evolved over time, and my writing style has been refined, and I think I continue to add significant value to my readers.

Twitter’s constant engagement feature, however, meant that it was hard to evolve there and hard to escape from the cycle of banal and negative tweets. My tweets from this February are unlikely to be qualitatively very different from those 5 years back, and that’s not a positive thing to say.

The thing with Twitter is that its short format encourages a “shoot first ask questions later” kind of thinking. You end up posting shit without thinking through it, and without having to construct a reasonable argument. This encourages outrage, and posting banal stuff. Spending one minute typing out a banal tweet is far lower cost than spending 20 minutes typing out a banal blog post – the latter is unlikely to be written unless there’s some kind of insight in it.

Outrage is one thing, but what’s really got to me with respect to twitter is its sheer ordinariness, and temporality (most tweets lose value a short period of time after they’re posted). It’s insane that it’s taken me so long (and three longish sabbaticals from twitter) to find out!

Starting from the mid-2000s, the dominant method to consume content was to follow individual blogs through RSS Feed readers such as Bloglines or Google Reader. You followed specific blogs, most of which (unlike this one) had content on specific topics.

So when I wanted to learn up on economics, I started following Marginal Revolution and Econlog. When I wanted to follow the global financial crisis, I added Felix Salmon and a couple of other blogs (which I don’t remember now). All I needed to do to read on specific topics was to follow specific people.

And then Google Reader Shared Items happened. Now, you didn’t really need to follow specific blogs, for there was a social network where people would share interesting stuff that they read. Now you could outsource following blogs to friends who became curators. So there was this one friend who would share pretty much every interesting post on Mashable. Another shared every interesting post from this blog called The Frontal Cortex. I didn’t need to follow these blogs. My “curator friends” shared the best pieces with me (and I know people relied on me for Econlog etc.).

Then around the turn of the decade, Twitter replaced Google Reader Shared Items as the primary content discovery platform. A couple of years later, Google would decommission Reader. The thing with Twitter was that the movement from following specific ideas and sites to following “curators” was complete.

While twitter also functions as a “normal” social network, a major function is the sharing of ideas, and so everyone on twitter is essentially a curator, sharing with her followers what she wants them to read. There is also scope for adding comments here, and adding one’s opinion to the content. This adds a sort of richness to the content, and people can filter stuff accordingly, without consuming everything one’s friend has shared.

The downside, however, is that you are forced to consume the opinions and links shared by everyone you follow. There might be someone who I might be following for his curation of technology links, but it might happen that he might also tweet heavily on politics, which I’m hardly interested in. There is an option to turn off retweets (which I’ve used liberally) but even so, there is a lot of “unwanted content” you have to consume from people. And since it is “opinion first” (and link later), you are forced to consume people’s opinion even if you’re just browsing their timeline.

What we need in Twitter is a way to curate people’s opinions on topics. For example, I might be interested in Person A’s opinion on politics but not anything else. Person B might offer good opinions on economics but might be lousy on other things. Person C might be good for technology and sports. And so forth.

Of course, you can’t charge people with classifying their own tweets – that will add too much friction to the process. What you need is an intelligent process or app that can help classify people’s tweets and show you only what you want to know.

I can think of a couple of designs for the app – one could be where you could tell it not to show any more tweets from someone on a particular topic (or block a topic itself). Another is for you to upvote and downvote tweets, so that the app learns your preferences and shows you what you want.

Yet, I’m not confident that such an app will be built. The problem is that twitter has been notorious in terms of cutting off access to its API to apps built on it, or cutting permissions of what apps can see (Facebook is as guilty here). So it’s a massive challenge to get people to actually invest in building twitter apps.

Twitter as it exists currently doesn’t work for me, though. I repeatedly find the problem that there is way too much outrage on my timeline, and despite mercilessly cutting the number of people I follow, I find that it’s a slippery slope and otherwise interesting people continue to tweet about stuff that I don’t want to read about. And so my engagement is dipping.

I don’t need twitter itself to do anything about it. All they should do is to send out credible signals that they’ll not pull the rug under the feet of developers, so that APIs can be developed, which can make the platform a much more pleasant experience for users.

## On apps tracking you and turning you into “lab rats”

Tech2, a division of FirstPost, reports that “Facebook could be tracking all rainbow profile pictures“. In what I think is a nonsensical first paragraph, the report says:

Facebook’s News Feed experiment received a huge blow from its social media networkers. With the new rainbow coloured profile picture that celebrates equality of marriage turned us into ‘lab rats’ again? Facebook is probably tracking all those who are using its new tool to change the profile picture, believes The Atlantic.

I’m surprised things like this still makes news. It is a feature (not a bug) of any good organisation that it learns from its user interactions and user behaviour, and hence tracking how users respond to certain kinds of news or updates is a fundamental part of how Facebook should behave.

And Facebook is a company that constantly improves and updates the algorithm it uses in order to decide what updates to show whom. And to do that, it needs to maintain data on who liked what, commented on what, and turned off what kind of updates. Collecting and maintaining and analysing such data is a fundamental, and critical, part of Facebook’s operations, and expecting them not to do so is downright silly (and it would be a downright silly act on part of the management if they stop experimenting or collecting data).

Whenever you sign on to an app or a service, you need to take it as a given that the app is collecting data and information from you. And that if you are not comfortable with this kind of data capture, you are better off not using the app. Of course, network effects mean that it is not that easy to live like you did in “the world until yesterday”.

This seems like yet another case of Radically Networked Outrage by outragers not having enough things to outrage about.

My friend Joy Bhattacharjya is a star on Facebook. He has a large number of friends (I haven’t bothered to see how many), most of whom seem to have him on their “good friends” list thanks to which they get each and every one of his updates (I had recently cribbed about Facebook’s algorithm, but when your friends love you, it doesn’t matter). And most of his updates are extremely insightful, some of them funny. If you are his friend, it is not hard to guess why his updates are so popular.

There is only one problem – it is impossible to comment on them. I mean, the comments section is always open, but the problem is that by the time you see an update, so many people would have commented on them that adding one more comment there doesn’t add any value. Writing something there, it seems, is not worth the time, for you assume that given the sea of comments the author won’t have time to read and appreciate your wisecrack. And so you move on.

The last time my birthday was listed on Facebook (2010, if I’m not wrong), I got so many scraps on my wall that I had no time to read them, let alone respond to them. I promptly delisted my birthday from Facebook, with the result that nowadays hardly anyone wishes me on my birthday. Not on Facebook, at least, and I’m happy about not having to respond to a mechanical action!

On a similar note, one thing I get very pissed off (on Facebook) is “thread hijacking”. You get a nice discussion going in the comments thread on some post, and then someone else comes in (usually an aunty) and says something so banal that you don’t want to be seen on that thread any more, and the discussion goes for a toss. Oh, and such thread hijacking is more prevalent on Facebook’s other product Whatsapp (:P ), especially on groups where lack of threaded conversation means deep discussions are highly prone to being disrupted by long forwards someone sends!

I sometimes wonder why I’m on Facebook at all. I used to use it at one point in time to look at people’s photos, and what they were up to. But now i find that it’s impossible to subscribe to a person’s photos without subscribing to her political views also, which are generally downright uninformed and sometimes extreme. And thanks to blogger-style comments, you cannot keep uninformed people out of your discussion on Facebook, unlike Twitter – they just keep popping up.

And there is no way for me to explicitly tell Facebook I want to see more or less of someone’s feed (like I could with Pandora, back when I used it). I have to rely on the algorithm.

All in all, Facebook seems like a dumb social network. To use a concept I’d mentioned here a few months back, it’s an “events and people” social network, with Twitter being more conducive to ideas. I sometimes end up asking myself why I’m on Facebook at all. And then I realise that there is no other way for me to access Joy’s updates!

## Pricing likes and the facebook algorithm

There is a good friend of mine who is a compulsive “LinkedIn liker”. Anything anyone in his network writes (either a LinkedIn blog or a status update or a job announcement), he is extremely likely to “like” them. While that helps the authors of such updates in getting their messages across to this guy’s networks also, the thing is that such likes add little value. If an update has come on my timeline because this guy has liked it, I’ll take it with salt since I know that this guy’s likes are “cheap”.

I don’t want to single out this guy, but there are several others on my Facebook friend list who are also compulsive likers. They like just about everything that they see, but the Facebook algorithm (by which not all of your updates are shared with all of your friends) means that their incidence is less than that of the LinkedIn liker. Then I have this one follower on Twitter who unfailingly likes each tweet of mine with a link. He engages in conversation very very sporadically, but like he does all the time!

So this got me thinking on the value of people’s likes, and what would happen if likes were to be rationed. I know it’s going to be hard to implement, but if you wee told that you had a quota of 10 likes that you could dole out in a day, how would you then ration your likes? Would such a cap make likes more valuable?

The reason this matters is that the number of likes has now become a metric that social media marketers track, and if some people’s likes are less valuable than others’, it is essentially a useless metric (and I know the problem is with the metric, not with likes). Even otherwise, from an information perspective, knowing the value of each person’s likes is useful for you in making up your mind on something!

So if say facebook decides that you get 10 free likes a day and have to pay for any more, how does that change your liking behaviour? For your 11th like, will you pay or go unlike something you’ve already liked? As a thought experiment, it is fascinating!

And while we are discussing Facebook, I must mention that I absolutely loathe its algorithm. I don’t know how it works, but it seems to me that the better updates that I put there just never get carried to my network, but some random updates that I sometimes put get propagated like crazy. I’ve been trying to reduce the number of updates there so that each update has a greater probability of getting propagated, but it just doesn’t seem to help!

And I was thinking about Facebook’s algorithm, and Twitter’s non-algorithm where every tweet you put gets carried to all your followers. Since Twitter doesn’t filter, all your followers have an opportunity to see all that you say. But the problem there is that since your followers see tweets of everyone on their timelines, your tweet is likely to get lost in the competition for attention.

So basically Twitter is like a free market where you have everyone’s tweets that get shown and compete for a follower’s attention. Facebook is like a more regulated market where there is no clutter, so every update gets undivided attention, but there is a Big Brother which decides who should see what!

I wonder if Facebook has considered making its algorithm public, and if it does, if it will have any impact on how people share. The value it will have for me is that at least I will know whether an update will get carried or not, and time and space my updates properly. But considering that one of Facebook’s revenue sources is to be paid by users to propagate their updates further, revelation of the algorithm will result in lower revenues for Facebook, so they’ll never do that.

I might just get all disgusted with the algorithm and quit Facebook some day.

## Guarantees in meetings

There are some events/meetings which involve strong network effects. People want to attend such events if and only if a certain number of other people are going to attend it. But then they don’t know before hand as to who else is coming, and hence are not sure whether to accept the invitation. These are events such as school reunions, for example, where if only a few people come, there isn’t much value. And it’s hard to coordinate.

In such events it’s always useful to provide a guarantee. For example, a friend from (B) school was in town last week and expressed an interest in meeting other batchmates in Bangalore. A mail thread was promptly started but until the morning of the event, people remained mostly noncommittal. Not many of us knew this guy particularly well, though he is generally well-liked. So none of us really wanted to land up and be among only one or two people along with this guy.

And then there was a guarantee. One other guy sent a mail saying he’d booked a table at a bar, and this sent a strong signal that this guy was going to be there too. Then there were a couple of other very positive replies and the guarantee having been set, some seven or eight people turned up and the meeting can be called a “success”.

Sometimes when you’re trying to organise an event, it makes sense to get unconditional attendance guarantees from a couple of people before you send out the invite to the wider world. So you tell people that “X and Y” (the early guarantors) are definitely coming, and that will pull in more people, and that can be the trigger in making the event a success! In certain circles, X and Y need to be celebrities. In smaller circles, they can be common men (or women), but people whose guarantees of attendance are generally trusted (i.e. people who don’t have a history of standing up people)!

Another small reunion of my B-school batch happened last month and in the run-up to that I realised another thing about RSVPs – yeses should be public and noes private. One guy took initiative and mailed a bunch of us proposing we meet. I hit reply all on purpose to say that it was a great idea and confirm my attendance. Soon there was another public reply confirming attendance and this snowballed to give us a successful event. There were a few invitees we didn’t hear from, who didn’t attend, and I assume they had replied privately to the invite in the negative.

The problem with events on Facebook is that your RSVP is public irrespective of your reply – so even if you say no, everyone knows you’ve said “no”. And so you think it’s rude to say “no”, and say “yes” just out of politeness, even though you have no intentions of attending.

I’ve attended a few events where the hosts estimated attendance based on a Facebook invite and grossly overestimated attendance – too many people had hit “yes” out of sheer politeness.

So the ideal protocol should be “public yes, private no”. Facebook should consider giving this as an option to event creators so that people reveal their true preferences in the RSVP rather than saying “yes” out of sheer politeness.

In that sense it’s like a Vickery auction whose basic design principle is that people reveal their true willingness to pay and not underbid to avoid the winner’s curse!

## A month of detox

I cheated a little bit this morning. Since it’s been a month now since I got off Twitter and Facebook, I logged in to both for about a minute each, to check if I have any messages. The ones on Facebook weren’t of much use – just some general messages. There was one DM on twitter which had value, and I sent the guy an email explaining I don’t use twitter any more. I presently logged out.

The one month off Twitter and Facebook has so far gone off fantastically. For starters it’s given me plenty of time to read, meet people, talk to people and other useful stuff. And apart from some interesting links that people post on Twitter, I haven’t really missed either of them.

There have been times when there have been thoughts that would have earlier led to a tweet. However, given that the option exists no more, I end up doing one of two things – if there is substance to the tweet and I can elaborate on it, then I do so and it results in a blog post (you must have noticed that the frequency of blogging has gone up significantly in the last one month).

If it’s not really blog worthy but just something that I want to share with someone, I think of whose attention I would have liked to have caught by putting that tweet. In most cases I have found that there is a small set of people whose attention I would have liked to catch with a tweet – every time I tweeted, I would think of how a particular set of people would respond. So what I do when I have something to say and a particular set of people to say it to is to just message it to them.

While this gives a much better chance of them responding to the message than if they just saw it on their timeline (or missed seeing it), it also has the added benefit of starting conversations. Which is not a bad thing at all. In the last one month I’ve seen that my usage of WhatsApp and Google Talk has gone up significantly.

The only thing I miss about twitter is the interesting links that people post. I’ve tried a few things to remedy that. Firstly I tried to see if I could write a script that crawls my timeline, gets popular links (based on a set of defined metrics), and then bookmarks the top five each day. I went some way with the code (pasted below the fold here) but couldn’t figure how to post the linked articles to Pocket (my article bookmarker of choice). So I ended up tweeting those chosen links (!!) with a #looksinteresting hashtag, so that ifttt does the job of adding to Pocket.

It went for a bit till multiple people told me the tweets were spammy. And then I realized I needed to tweak the algorithm, and it needed significant improvement. And then I realized the solution was at hand – Flipboard.

If you have an android phone or an iPad and not used FlipBoard you’re really missing something. it’s a great app that curates articles based on your indicated areas of interest and history, and one of the sources it can get links from is your own Twitter and Facebook accounts. It is generally good in terms of its algo and good links usually bubble up there.

When I went off Twitter and Facebook on the 6th of August (in a fit of rage, outraged by all the outrage and negativity on the two media) I wanted complete isolation. And thus I deleted Twitter and Facebook from my FlipBoard also. Now I realized that adding back twitter on FlipBoard will allow me to access the nice links shared there without really getting addicted back to twitter, or partaking all the outrage.

For the last two weeks it’s worked like a charm. That twitter is present only on FlipBoard, which I use not more than twice a day (once in the morning, once at night), means that I’ve had the best of both worlds. And not being on twitter has meant that i’ve been able to get a fair bit of work done, finished three books (my first attempt at reading fiction in ten years fizzled out midway, though – Joseph Conrad’s Heart of Darkness failed to sustain my interest beyond about 40% (I have it on kindle) ), written dozens of blog posts across the three blogs and had more meaningful conversations with people.

I hereby extend my sabbatical from Twitter and Facebook for another month.

Below the fold is the code I wrote. It’s in R. I hope you can make some sense of it.