A one in billion trillion event

It seems like capital markets quants have given up on the lognormal model for good, for nobody described Facebook’s stock price drop last Thursday as a “one in a billion trillion event”. For that is the approximate probability of it happening, if we were to assume a lognormal model of the market.

Created using Quantmod package. Data from Yahoo.

Without loss of generality, we will use 90 days trailing data to calculate the mean and volatility of stock returns. As of last Thursday (the day of the fall), the daily mean returns for FB was 0.204%, or an annualised return of 51.5% (as you can see, very impressive!). The daily volatility in the stock (using a 90-day lookback period again) was 1.98%, or an annualised volatility of 31.4% . While it is a tad on the higher side, it is okay considering the annual return of 51.5%.

Now, traditional quantitative finance models have all used a lognormal distribution to represent stock prices, which implies that the distribution of stock price returns is normal. Under such an assumption, the likelihood of a 18.9% drop in the value of Facebook (which is what we saw on Thursday) is very small indeed.

In fact, to be precise, when the stock is returning 0.204% per day with a vol of 1.98% per day, the an 18.9% drop is a 9.7 sigma event. In other words, if the distribution of returns were to be normal, Thursday’s drop is 9 sigmas away from normal. Remember that most quality control systems (admittedly in industrial settings, where faults are indeed governed by a nearly normal distribution) are set for a six sigma limit.

Another way to look at Thursday’s 9.7 sigma event is that again under the normal distribution, the likelihood of seeing this kind of a fall in a day is $math ~10^{-21}$. Or one in a billion trillion. In terms of the number of trading days required for such a fall to arrive at random, it is of the order of a billion billion years, which is an order of magnitude higher than the age of the universe!

In fact, when the 1987 stock market crash (black monday) happened, this was the defence the quants gave for losing their banks’ money – that it was an incredibly improbable event. Now, my reading of the papers nowadays is sketchy, and I mostly consume news via twitter, but I haven’t heard a single such defence from quants who lost money in the Facebook crash. In fact, I haven’t come across too many stories of people who lost money in the crash.

Maybe it’s the power of diversification, and maybe indexing, because of which Facebook is now only a small portion of people’s portfolios. A 20% drop in a stock that is even 10% of your portfolio erodes your wealth by 2%, which is tolerable. What possibly caused traders to jump out of windows on Black Monday was that it was a secular drop in the US market then.

Or maybe it’s that the lessons learnt from Black Monday have been internalised, and included in models 30 years hence (remember that concepts such as volatility smiles and skews, and stochastic volatility, were introduced in the wake of the 1987 crash).

That a 20% drop in one of the five biggest stocks in the United States didn’t make for “human stories” or stories about “one in a billion billion event” is itself a story! Or maybe my reading of the papers is heavily biased!

PostScript

Even after the spectacular drop, the Facebook stock at the time of this update is trading at 168.25, a level last seen exactly 3 months ago – on 26th April, following the last quarter results of Facebook. That barely 3 months’ worth of earnings have been wiped out by such a massive crash suggests that the only people to have lost from the crash are traders who wrote out of the money puts.

Algorithmic curation

When I got my first smartphone (a Samsung Galaxy Note 2) in 2013, one of the first apps I installed on it was Flipboard. I’d seen the app while checking out some phones at either the Apple or Samsung retail outlets close to my home, and it seemed like a rather interesting idea.

For a long time, Flipboard was my go-to app to check the day’s news, as it conveniently categorised news into “tech”, “business” and “sport” and learnt about my preferences and fed me stuff I wanted. And then after some update, it suddenly stopped working – somehow it started serving too much stuff I didn’t want to read about, and when I tuned (by “following” and “unfollowing” topics) my feed, it progressively got worse.

I stopped using it some 2 years back, but out of curiosity started using it again recently. While it did throw up some nice articles, there is too much unwanted stuff in the app. More precisely, there’s a lot of “clickbaity” stuff (“10 things about Narendra Modi you would never want to know” and the like) in my feed, meaning I have to wade through a lot of such articles to find the occasional good ones.

(Aside: I dedicate about half a chapter to this phenomenon in my book. The technical term is “congestion”. I talk about it in the context of markets in relationships and real estate)

Flipboard is not the only one. I use this app called Pocket to bookmark long articles and read later. A couple of years back, Pocket started giving “recommendations” based on what I’d read and liked. Initially it was good, and mostly curated from what my “friends” on Pocket recommended. Now, increasingly I’m getting clickbaity stuff again.

I stopped using Facebook a long time before they recently redesigned their newsfeed (to give more weight to friends’ stuff than third party news), but I suspect that one of the reasons they made the change was the same – the feed was getting overwhelmed with clickbaity stuff, which people liked but didn’t really read.

Basically, there seems to be a widespread problem in a lot of automatically curated news feeds. To put it another way, the clickbaity websites seem to have done too well in terms of gaming whatever algorithms the likes of Facebook, Flipboard and Pocket use to build their automated recommendations.

And more worryingly, with all these curators starting to do badly around the same time (ok this is my empirical observation. Given few data points I might be wrong), it suggests that all automated curation algorithms use a very similar algorithm! And that can’t be a good thing.

Commenting on social media

While I’m more off than on in terms of my consumption of social media nowadays, I find myself commenting less and less nowadays.

I’ve stopped commenting on blogs because I primarily consume them using an RSS reader (Feedly) on my iPad, and need to click through and use my iPad keyboard to leave comments, a hard exercise. And comments on this blog make me believe that it’s okay to not comment on blogs any more.

On Facebook, I leave the odd comment but find that most comments add zero value. “Oh, looking so nice” and “nice couple” and things like that which might flatter some people, but which make absolutely no sense once you start seeing through the flattery.

So the problem on Facebook is “congestion”, where a large number of non-value-adding comments may crowd out the odd comment that actually adds value, so you as a value-adding-commentor decide to not comment at all.

The problem on LinkedIn is that people use it mostly as a medium to show off (that might be true of all social media, but LinkedIn is even more so), and when you leave a comment there, you’re likely to attract a large number of show-offers who you are least interested in talking to. Again, there’s the Facebook problem here in terms of congestion. There is also the problem that if you leave a comment on LinkedIn, people might think you’re showing off.

Twitter, in that sense, is good in that you can comment and selectively engage with people who reply to your comment (on Facebook, when all replies are in one place, such selective engagement is hard, and you can offend people by ignoring them). You can occasionally attract trolls, but with a judicious combination of ignoring, muting and blocking, those can be handled.

However, in my effort to avoid outrage (I like to consume news but don’t care about random people’s comments on it), I’ve significantly pruned my following list. Very few “friends”. A few “twitter celebrities”. Topic-specific studs. The problem there is that you can leave comments, but when you see that nobody is replying to them, you lose interest!

So it’s Jai all over the place.

No comments.

Twitter and negativity

One of the reasons that sparked my departure from social media platforms such as Facebook and Twitter two weeks back was an argument with my wife where she claimed that Twitter had made me too negative, and highly prone to trolling (even in “real life”). Accepting a challenge from her, I offered to go through my tweets over the last few months, and identify those that were negative. I also offered to perform a similar exercise with my blog.

I started off with the intention to go through tweets in the last one year and delete anything that was negative or “troll-y”. I allocated myself an hour to accomplish this, along with a similar exercise for my blog.

I must have spent fifty minutes going through my twitter feed, and didn’t manage to go back more than two months. I was surprised by my own sheer volume of tweeting. What was more surprising was the amazing lack of insight in most of those tweets – there were horrible PJs that I’d cracked just because I could, there were random replies to other people which didn’t add any kind of value, there was outrage about the lack of outrage and some plain banal life stuff (apart from some downright trolly stuff which I deleted).

It made for extremely painful reading, and I could hardly recognise myself from my own tweets. Apart from some personal markers, I would find it hard to recognise most of these tweets as my own if they were to be presented to me a few months later. It was a clear indication that it was time to exit twitter (though since I have a rather kickass username there I’m not deleting my account).

The ten minutes I spent that day going through this blog, however, was a sheer delight. I did end up deleting a couple of outragey posts (both of which were essentially collections of tweets which I’d collated for posterity), but most of my posts were mostly sheer delight! There was some kind of insight in each of my posts, and I’d lie if I were to say that I’m not proud of what I’ve written.

It’s not that I’ve not written shit on this blog (or its predecessor), having written posts as late as 2008 which I’m definitely not proud of. What I’ve noticed, however, is that I’ve evolved over time, and my writing style has been refined, and I think I continue to add significant value to my readers.

Twitter’s constant engagement feature, however, meant that it was hard to evolve there and hard to escape from the cycle of banal and negative tweets. My tweets from this February are unlikely to be qualitatively very different from those 5 years back, and that’s not a positive thing to say.

The thing with Twitter is that its short format encourages a “shoot first ask questions later” kind of thinking. You end up posting shit without thinking through it, and without having to construct a reasonable argument. This encourages outrage, and posting banal stuff. Spending one minute typing out a banal tweet is far lower cost than spending 20 minutes typing out a banal blog post – the latter is unlikely to be written unless there’s some kind of insight in it.

Outrage is one thing, but what’s really got to me with respect to twitter is its sheer ordinariness, and temporality (most tweets lose value a short period of time after they’re posted). It’s insane that it’s taken me so long (and three longish sabbaticals from twitter) to find out!

The problem with Twitter

Starting from the mid-2000s, the dominant method to consume content was to follow individual blogs through RSS Feed readers such as Bloglines or Google Reader. You followed specific blogs, most of which (unlike this one) had content on specific topics.

So when I wanted to learn up on economics, I started following Marginal Revolution and Econlog. When I wanted to follow the global financial crisis, I added Felix Salmon and a couple of other blogs (which I don’t remember now). All I needed to do to read on specific topics was to follow specific people.

And then Google Reader Shared Items happened. Now, you didn’t really need to follow specific blogs, for there was a social network where people would share interesting stuff that they read. Now you could outsource following blogs to friends who became curators. So there was this one friend who would share pretty much every interesting post on Mashable. Another shared every interesting post from this blog called The Frontal Cortex. I didn’t need to follow these blogs. My “curator friends” shared the best pieces with me (and I know people relied on me for Econlog etc.).

Then around the turn of the decade, Twitter replaced Google Reader Shared Items as the primary content discovery platform. A couple of years later, Google would decommission Reader. The thing with Twitter was that the movement from following specific ideas and sites to following “curators” was complete.

While twitter also functions as a “normal” social network, a major function is the sharing of ideas, and so everyone on twitter is essentially a curator, sharing with her followers what she wants them to read. There is also scope for adding comments here, and adding one’s opinion to the content. This adds a sort of richness to the content, and people can filter stuff accordingly, without consuming everything one’s friend has shared.

The downside, however, is that you are forced to consume the opinions and links shared by everyone you follow. There might be someone who I might be following for his curation of technology links, but it might happen that he might also tweet heavily on politics, which I’m hardly interested in. There is an option to turn off retweets (which I’ve used liberally) but even so, there is a lot of “unwanted content” you have to consume from people. And since it is “opinion first” (and link later), you are forced to consume people’s opinion even if you’re just browsing their timeline.

What we need in Twitter is a way to curate people’s opinions on topics. For example, I might be interested in Person A’s opinion on politics but not anything else. Person B might offer good opinions on economics but might be lousy on other things. Person C might be good for technology and sports. And so forth.

Of course, you can’t charge people with classifying their own tweets – that will add too much friction to the process. What you need is an intelligent process or app that can help classify people’s tweets and show you only what you want to know.

I can think of a couple of designs for the app – one could be where you could tell it not to show any more tweets from someone on a particular topic (or block a topic itself). Another is for you to upvote and downvote tweets, so that the app learns your preferences and shows you what you want.

Yet, I’m not confident that such an app will be built. The problem is that twitter has been notorious in terms of cutting off access to its API to apps built on it, or cutting permissions of what apps can see (Facebook is as guilty here). So it’s a massive challenge to get people to actually invest in building twitter apps.

Twitter as it exists currently doesn’t work for me, though. I repeatedly find the problem that there is way too much outrage on my timeline, and despite mercilessly cutting the number of people I follow, I find that it’s a slippery slope and otherwise interesting people continue to tweet about stuff that I don’t want to read about. And so my engagement is dipping.

I don’t need twitter itself to do anything about it. All they should do is to send out credible signals that they’ll not pull the rug under the feet of developers, so that APIs can be developed, which can make the platform a much more pleasant experience for users.

On apps tracking you and turning you into “lab rats”

Tech2, a division of FirstPost, reports that “Facebook could be tracking all rainbow profile pictures“. In what I think is a nonsensical first paragraph, the report says:

Facebook’s News Feed experiment received a huge blow from its social media networkers. With the new rainbow coloured profile picture that celebrates equality of marriage turned us into ‘lab rats’ again? Facebook is probably tracking all those who are using its new tool to change the profile picture, believes The Atlantic.

I’m surprised things like this still makes news. It is a feature (not a bug) of any good organisation that it learns from its user interactions and user behaviour, and hence tracking how users respond to certain kinds of news or updates is a fundamental part of how Facebook should behave.

And Facebook is a company that constantly improves and updates the algorithm it uses in order to decide what updates to show whom. And to do that, it needs to maintain data on who liked what, commented on what, and turned off what kind of updates. Collecting and maintaining and analysing such data is a fundamental, and critical, part of Facebook’s operations, and expecting them not to do so is downright silly (and it would be a downright silly act on part of the management if they stop experimenting or collecting data).

Whenever you sign on to an app or a service, you need to take it as a given that the app is collecting data and information from you. And that if you are not comfortable with this kind of data capture, you are better off not using the app. Of course, network effects mean that it is not that easy to live like you did in “the world until yesterday”.

This seems like yet another case of Radically Networked Outrage by outragers not having enough things to outrage about.

Value addition through comments

My friend Joy Bhattacharjya is a star on Facebook. He has a large number of friends (I haven’t bothered to see how many), most of whom seem to have him on their “good friends” list thanks to which they get each and every one of his updates (I had recently cribbed about Facebook’s algorithm, but when your friends love you, it doesn’t matter). And most of his updates are extremely insightful, some of them funny. If you are his friend, it is not hard to guess why his updates are so popular.

There is only one problem – it is impossible to comment on them. I mean, the comments section is always open, but the problem is that by the time you see an update, so many people would have commented on them that adding one more comment there doesn’t add any value. Writing something there, it seems, is not worth the time, for you assume that given the sea of comments the author won’t have time to read and appreciate your wisecrack. And so you move on.

Recently one friend announced his engagement. Another announced the birth of her child. It was again impossible to add value via comments to either – there had already been so many comments that adding one more wouldn’t add any value! I doubt if these “announcers” even bothered to read through all the comments people had posted. A compression algorithm might have done the trick for them, for most of them were extremely banal and non-value-adding “congrats” posts!

The last time my birthday was listed on Facebook (2010, if I’m not wrong), I got so many scraps on my wall that I had no time to read them, let alone respond to them. I promptly delisted my birthday from Facebook, with the result that nowadays hardly anyone wishes me on my birthday. Not on Facebook, at least, and I’m happy about not having to respond to a mechanical action!

On a similar note, one thing I get very pissed off (on Facebook) is “thread hijacking”. You get a nice discussion going in the comments thread on some post, and then someone else comes in (usually an aunty) and says something so banal that you don’t want to be seen on that thread any more, and the discussion goes for a toss. Oh, and such thread hijacking is more prevalent on Facebook’s other product Whatsapp (:P ), especially on groups where lack of threaded conversation means deep discussions are highly prone to being disrupted by long forwards someone sends!

Recently, Facebook introduced the threaded comments feature, one that I loved so much that I resisted a move away from Livejournal for ages just for that one feature, and when I moved to this blog, one of the first plugins I installed was one that allowed for threaded comments. Facebook has done badly, though. I use it primarily through the iPad app, and the threaded comments suck big time, requiring way too many clicks to navigate. If done so badly, I’d prefer blogspot-type dumb linear comment scheme only!

I sometimes wonder why I’m on Facebook at all. I used to use it at one point in time to look at people’s photos, and what they were up to. But now i find that it’s impossible to subscribe to a person’s photos without subscribing to her political views also, which are generally downright uninformed and sometimes extreme. And thanks to blogger-style comments, you cannot keep uninformed people out of your discussion on Facebook, unlike Twitter – they just keep popping up.

And there is no way for me to explicitly tell Facebook I want to see more or less of someone’s feed (like I could with Pandora, back when I used it). I have to rely on the algorithm.

All in all, Facebook seems like a dumb social network. To use a concept I’d mentioned here a few months back, it’s an “events and people” social network, with Twitter being more conducive to ideas. I sometimes end up asking myself why I’m on Facebook at all. And then I realise that there is no other way for me to access Joy’s updates!