Conductors and CAPM

For a long time I used to wonder why orchestras have conductors. I possibly first noticed the presence of the conductor sometime in the 1990s when Zubin Mehta was in the news. And then I always wondered why this person, who didn’t play anything but stood there waving a stick, needed to exist. Couldn’t the orchestra coordinate itself like rockstars or practitioners of Indian music forms do?

And then i came across this video a year or two back.

And then the computer science training I’d gone through two decades back kicked in – the job of an orchestra conductor is to reduce an O(n^2) problem to an O(n) problem.

For a  group of musicians to make music, they need to coordinate with each other. Yes, they have the staff notation and all that, but still they need to know when to speed up or slow down, when to make what transitions, etc. They may have practiced together but the professional performance needs to be flawless. And so they need to constantly take cues from each other.

When you have n musicians who need to coordinate, you have \frac{n.(n-1)}{2} pairs of people who need to coordinate. When n is small, this is trivial, and so you see that small ensembles or rock bands can easily coordinate. However, as n gets large, n^2 grows well-at-a-faster-rate. And that is a problem, and a risk.

Enter the conductor. Rather than taking cues from one another, the musicians now simply need to take cues from this one person. And so there are now only n pairs that need to coordinate – each musician in the band with the conductor. Or an O(n^2) problem has become an O(n) problem!

For whatever reason, while I was thinking about this yesterday, I got reminded of legendary finance professor R Vaidya‘s class on capital asset pricing model (CAPM), or as he put it “Sharpe single index model” (surprisingly all the links I find for this are from Indian test prep sites, so not linking).

We had just learnt portfolio theory, and how using the expected returns, variances and correlations between a set of securities we could construct an “efficient frontier” of securities that could give us the best risk-adjusted return. Seemed very mathematically elegant, except that in case you needed to construct a portfolio of n stocks, you needed n^2 correlations. In other word, an O(n^2) problem.

And then Vaidya introduced CAPM, which magically reduced the problem to an O(n) problem. By suddenly introducing the concept of an index, all that mattered for each stock now was its beta – the coefficient of its returns proportional to the index returns. You didn’t need to care about how stocks reacted with each other any more – all you needed was the relationship with the index.

In a sense, if you think about it, the index in CAPM is like the conductor of an orchestra. If only all O(n^2) problems could be reduced to O(n) problems this elegantly!

Conductors and CAPM

Recently I watched this video that YouTube recommended to me about why orchestras have conductors.

The basic idea is that an orchestra  needs a whole lot of coordination, in terms of when to begin and end, when to slow down or speed up, when to move to the next line and so on. And in case there is no conductor, the members of the orchestra need to coordinate among themselves.

This is easy enough when there is a small number of members, so you don’t see bands (for example) needing conductors. However, notice that if the orchestra has to coordinate among themselves, coordination is an O(n^2) problem. By appointing an external conductor whose only job is to conduct and not play, this O(n^2) problem is reduced to an O(n) problem.

When I saw this, this took me back to my Investments course in IIMB, where the professor one day introduced what he called the “Sharpe single index model“, which is sort of similar to the CAPM.

Just before learning the Sharpe Single Index Model, we had been learning about Markowitz’s portfolio theory. And then, as he introduced the Sharpe Single Index Model, Vaidya said something to the effect that “instead of knowing so many correlation terms” (which is an O(n^2) problem), “we only need to know the correlation of each stock to the market index” (makes it an O(n) problem).

As someone who has studied computer science formally, converting O(n^2) problems to O(n) problems is a massive fascination. It is interesting how I connected two such reductions from completely different fields.

In other words, conductors are the “market of the orchestra”.

Diamonds and Rust

So this post is going to piss off the wife on at least two counts. Firstly, she thinks I’m “spending too much time on the computer” nowadays, and not enough with her. Secondly, this post refers to an old crush who my wife thinks I had “blogged too much about” (the implication is that I don’t blog enough about my wife).

Then again, I think I’ve been taking myself too seriously on this blog of late, and so need something to break out of this rut, and this post is something I’ve been intending to write for a long time. So I’m taking a chance here.

The song in question is Diamonds and Rust, originally performed by Joan Baez, and then covered by Judas Priest in their album Sin After Sin.

I was first introduced to this song by the Judas Priest version. It was that time back in college where I had computer, and access to a LAN full of pirated music, and was sampling all the bands that I thought might be cool (it’s another matter that I ended up liking a lot of these “cool” bands, including Judas Priest).

As was my wont then whenever I “discovered” some artist, I would listen to all their works in order, album by album. I do this nowadays as well, when I “rediscover” artists. And so I got introduced to Diamonds and Rust. I remember the song immediately making an impression on me, but not too much (the other song that that made an immediate impact was called “between the hammer and the anvil“, and I’d wondered if it was about the mechanics of the inner ear).

Anyway, in the middle of discovering Judas Priest for the first time, I met this girl. I mean I’d known her for a really long time but this was the first time we were “having a conversation”. We had met at this tiny cafe full of college kids (we were also college kids then) where she had made a big fuss about being a “low calorie person”. Music was playing. Soon a vaguely familiar sounding song played, in a voice that wasn’t familiar at all. Between bits of the conversation, all I caught from the song was that it was “_____ and _____ “. Surprisingly for me, I didn’t try to immediately figure out which song it was upon returning to my room that night.

The years went by. I probably ended up blogging this girl a bit too much for my own good later on. The person who is now my wife read some of those posts and thought she had found a guy who would write loads about her as well. I started off brightly, but in the long term I don’t think I’ve lived up to the expectation.

I don’t recall the circumstances in which I rediscovered Diamonds And Rust. It happened in London, either towards the end of 2017 or the beginning of 2018. I think the rediscovery again happened through Judas Priest – I was working through their albums one by one after a 12 year gap, and chanced upon Diamonds And Rust again. Some chord (not literally) hit. I went down a rabbit hole.

I realised this was possibly the song that had initially registered all those years ago, and that I had heard in the cafe. Googling revealed it was a cover, and the original did sound very familiar (I think this is the story. I’ve sat on this post for so long now I’ve really forgotten). I was convinced. The Joan Baez version did seem very familiar. It all started coming back to me. The next couple of days I was careful around the wife so she wouldn’t realise that I had gotten excited about something vaguely related to an old crush.

In any case, I liked the cover so much that soon I started creating a playlist of “metal covers of non-metal songs”.

I called it “Rust Covers Diamonds” (get the clever pun?). I’m listening to that playlist right now as I write this. It’s a public playlist, so feel free to listen to it. You’ll love a lot of the songs in it! Especially the first “title track”.

Update

There is one thing I don’t like about Diamonds and Rust, and I blame Joan Baez for it (Judas Priest simply copied it without checking it seems). The song is not dimensionally consistent. Check the lyrics:

And here I sit, hand on the telephone
Hearing the voice I’d known
A couple of light years ago
Heading straight for a fall

Light year is a unit of distance, not time. So “a couple of light years ago” makes absolutely no sense. I really don’t know how the editors let that pass. Then again, you don’t expect most editors to know physics!

The difficulty of song translation

One of my wife’s favourite nursery rhymes is this song that is sung to the tune of “for he’s a jolly good fellow”, and about a bear going up a mountain.

For a long time I only knew of the Kannada version of this song (which is what the wife used to sing), but a year or two back, I found the “original” English version as well.

And that was a revelation, for the lyrics in the English version make a lot more sense. They go:

The bear went over the mountain;
The bear went over the mountain
The bear went over the mountain, to see what he could see.
And all that he could see, and all that he could see
Was the other side of the mountain,
The other side of the mountain
The other side of the mountain, was all that he could see.

Now, the Kannada version, sung to the same tune, obviously goes “???? ??????? ??????” (karaDi beTTakke hoithu). That part has been well translated. However, the entire stanza hasn’t been translated properly, because of which the song goes a bit meaningless.

The lyrics, when compared to the original English version, are rather tame. Since a large part of my readership don’t understand Kannada, here is my translation of the lyrics (btw, the lyrics used in these YouTube versions are different from the lyrics that my wife sings, but both are similar):

The bear went to the mountain.
The bear went to the mountain.
The bear went to the mountain.
To see the scenery

And what did it see?
What did it see?
The other side of the mountain.
The other side of the mountain.
It saw the scenery of the other side of the mountain.

Now, notice the important difference in the two versions, which massively changes the nature of the song. The Kannada version simply skips the “all that he could see” part, which I think is critical to the story.

The English version, in a way, makes fun of the bear, talking about how it went over the mountain thinking it’s a massive task, but “all that he could see” from there was merely the other side of the mountain. This particular element is missing in Kannada – there is nothing in the lyrics that suggests that the bear’s effort to climb the mountain was a bit of a damp squib.

And that,  I think, is due to the difficulty of translating songs. When you translate a song, you need to get the same letter and spirit of the lyrics, while making sure they can follow the already-set music as well (and even get the rhyming right). And unless highly skilled bilingual poets are involved, this kind of a translation is really difficult.

So you get half-baked translations, like the bear story, which possibly captures the content of the story but completely ignores its spirit.

After I had listened to the original English version, I’ve stopped listening to the Kannada version of the bear-mountain song. Except when the wife sings it, of course.

 

This year on Spotify

I’m rather disappointed with my end-of-year Spotify report this year. I mean, I know it’s automated analytics, and no human has really verified it, etc.  but there are some basics that the algorithm failed to cover.

The first few slides of my “annual report” told me that my listening changed by seasons. That in January to March, my favourite artists were Black Sabbath and Pink Floyd, and from April to June they were Becky Hill and Meduza. And that from July onwards it was Sigala.

Now, there was a life-changing event that happened in late March which Spotify knows about, but failed to acknowledge in the report – I moved from the UK to India. And in India, Spotify’s inventory is far smaller than it is in the UK. So some of the bands I used to listen to heavily in the UK, like Black Sabbath, went off my playlist in India. My daughter’s lullaby playlist, which is the most consumed music for me, moved from Spotify to Amazon Music (and more recently to Apple Music).

The other thing with my Spotify use-case is that it’s not just me who listens to it. I share the account with my wife and daughter, and while I know that Spotify has an algorithm for filtering out kid stuff, I’m surprised it didn’t figure out that two people are sharing this account (and pitched us a family subscription).

According to the report, these are the most listened to genres in 2019:

Now there are two clear classes of genres here. I’m surprised that Spotify failed to pick it out. Moreover, the devices associated with my account that play Rock or Power Metal are disjoint from the devices that play Pop, EDM or House. It’s almost like Spotify didn’t want to admit that people share accounts.

Then some three slides on my podcast listening for the year, when I’ve overall listened to five hours of podcasts using Spotify. If I, a human, were building this report, I would have dropped this section citing insufficient data, rather than wasting three slides with analytics that simply don’t make sense.

I see the importance of this segment in Spotify’s report, since they want to focus more on podcasts (being an “audio company” rather than a “music company”), but maybe something in the report to encourage me to use Spotify for more podcasts (maybe recommending Spotify’s exclusive podcasts that I might like, be it based on limited data?) might have helped.

Finally, take a look at my our most played songs in 2019.

It looks like my daughter’s sleeping playlist threaded with my wife’s favourite songs (after a point the latter dominate). “My songs” are nowhere to be found – I have to go all the way down to number 23 to find Judas Priest’s cover of Diamonds and Rust. I mean I know I’ve been diversifying the kind of music that I listen to, while my wife listens to pretty much the same stuff over and over again!

In any case, automated analytics is all fine, but there are some not-so-edge cases where the reports that it generates is obviously bad. Hopefully the people at Spotify will figure this out and use more intelligence in producing next year’s report!

Lullabies and walled gardens

There’s still a bit of walled gardens going on in the device and voice control space. About two years ago, in London, we acquired an Amazon Echo, and found that Alexa voice assistant could be used to play songs through either Spotify or Amazon Music, but not through Apple Music, which we then used.

And so, we got rid of Apple Music and took a subscription to Spotify. And among the things we would make Alexa do was to play the daughter’s lullabies on Spotify. And that is how, at the age of two, Berry spoke her first complete sentence, “Alexa, use Spotify to play Iron Man by Black Sabbath”.

We don’t have that Echo any more, and as a household are in a complete “apple ecosystem” as far as devices are concerned. Two Macs, two iPhones, an iPad and now a pair of AirPods. However, we had quite got used to Spotify and its playlists and its machine learning, and even though the India catalogue is nowhere as good as the one in the UK, we continued our subscription.

However, bands such as Black Sabbath, Led Zeppelin and Iron Maiden are critical for us, not least because their songs are part of the daughter’s sleeping portfolio. So we need something other than Spotify. And then we discovered that in India, Amazon Prime Music comes bundled with the Amazon Prime membership. And so we created the daughter’s sleeping playlist there, and started using it for bands not available on Spotify.

It was an uncomfortable arrangement, not least because Amazon Music is a terrible software product. Since family subscriptions are still not a thing with Spotify India, during periods of deep work the wife and I would fight over who would get Spotify and who had to make do with Amazon Music.

And then there is voice. Being in a complete Apple EcoSystem now, we found that Siri couldn’t control Spotify or Amazon Music, and for seamless voice experience (especially given I use it in car, using Apple Carplay) we needed Apple Music. And given how painful Amazon Music is to use, I thought spending ?149 a month on Apple Music Family Subscription is worth it, and took the subscription yesterday.

Since then I’ve been happily using it using voice control on all devices. Except until an hour back when I was putting the daughter to sleep. She requested for “baby has he”, which is her way of saying she wants Iron Man by Rockabye Baby (rather than by Black Sabbath). And so I held down the home button of the iPad and barked “play lullaby renditions of Black Sabbath”.

I don’t know what Siri interpreted (this is a standard command I’d been giving it back in the day when I used to exclusively use Apple Music), but rather than playing Lullaby Renditions of Black Sabbath, it played some “holy lullabies”, basically lullaby versions of some Christian songs. I tried changing but the daughter insisted that I let it be.

And so she kept twisting and turning in her bed, not going to sleep. I soon lost patience. Abandoning voice, I opened the iPad and switched from Apple Music to Spotify, where I knew the Rockabye Baby album was open (from last night – we hardly use the iPad otherwise nowadays), and started playing that.

Before Iron Man was halfway through, the daughter was fast asleep.

Wheels of the bus went swimming one day

One story I like to tell is about how Mozart charged so much for setting Twinkle Twinkle to tune (if he did set it to tune, that is) that propagators of nursery rhymes decided to use the same tune for several other popular songs – most prominently for ABCDEFG and with a small variation for Ba Ba Black Sheep. It’s confusing, not just for kids but also for the parents. I’d written here a month or so back about how I would play tunes on the keyboard and Berry would try to guess the song and sing along. As someone who sets quizzes occasionally, the lack of “a unique answer” drives me nuts. And it possibly drives Berry nuts as well, since she changes from twinkle twinkle to ABCD within the course of one stanza. I wonder why this is the case. Using my one data point (Berry) kids can catch on to tunes pretty quickly (she was barely a year old when she started humming the tune of Black Sabbath’s Iron Man. Now she knows the full lyrics). And having unique tunes for songs means that kids are able to make easy associations between music and words – always a desirable thing. And the lack of one-to-one correspondence doesn’t just run one way – sometimes there are multiple ways in which the same song can be sung. For some songs, such as Happy Birthday, this is due to copyright issues. I’m not sure why other songs are sung in different tunes. For example, there are two clearly different ways in which the third line of itsy-bitsy/incy-wincy (depending on which side of the pond you’re on) spider is sung, and it gets especially confusing when I’m playing on the keyboard, since I don’t know which version Berry is expecting ( we invariably sing/play the “other” way). The usage of voice controlled players has made things worse. In fact, the first time I appreciated Siri on my phone was when Berry was just born, and I needed both hands to hold her and put her to sleep, and then someone turn on a lullaby (“Hey Siri, play iron man by rockabye baby”). Now, the problem with voice-controlled playing is that when there are multiple versions of the same song you don’t know which one will get played. An extreme case is like earlier today we asked Alexa to use Amazon Unlimited (we have a 3-month free trial, possibly because of my Prime membership) to play “london bridge”. It belted out some dhinchak EDM song! Within the realms of nursery rhymes itself there are songs that are sung to completely new tunes (like I had never expected that there exists a version of Jack and Jill sung to the tune of Yankee Doodle. It is most annoying). It is extremely disorienting for me – and I guess it is for kids as well, for I’m told they like predictability. I don’t know what can be done to restore the sanity of one-song-one-tune.  Yes, I can record a set of songs in unique and popular tunes, but there is no guarantee that it will take off. And with the increase of voice controlled music playing, there is no guarantee that the “bad tunes” won’t get any air time. The title, for those that didn’t get it, is a portmanteau of two songs that share a name. I must mention I have no intention of popularising these two precise renditions of these songs – they were simply on top of the search engine results.
Mommy duck said quack quack quack, all day long! PS: There are differing versions in lyrics as well. One version says “all day long”; another says “all the way to town”. As Aditya Narayan sang in Rangeela Re 23 years ago, it’s complicated being a kid.

Why Indian Classical Music is Superior to Western Classical Music

I’ve been half-watching this atrocious movie called “Thank You“. Rather, the wife has been watching and I’ve been eavesdropping once in a while. Apart from the odd lame joke, it’s a horrible movie, so I wouldn’t recommend you to watch it.

But there’s one scene in that that illustrates that Indian Classical Music is superior to Western Classical Music. So the plot of the movie is that there are three stupid guys who are trying to find a conman who has been messing with them. Despite mostly obvious clues, they fail to identify him.

Until this day when they are all in his office, and one of them finds some sheet music and starts playing the notes on a conveniently located keyboard. This piece of music is something associated with the conman through the movie, and the three stupid guys immediately figure the identity of the conman.

So what does this have to do with Western Classical music? One of the key differences between Indian and Western Classical music is that in the former the performers mug up the notes of the songs – at least the parts where they don’t have to improvise. Once you know a song, you can dispense with the book. It is almost unknown for professionals to look at notes while performing.

Western Classical, on the other hand, spares performers of using up valuable memory space in their heads from remembering music, and has performers read the music from a sheet as they play it. While this has its advantages – notes are never “forgotten”, and all performers are easily in sync, and valuable memory space in the brain is not wasted – there are disadvantages as well.

Like if you have a signature tune, and if you play it often, you are likely to leave the sheet music of the tune lying around in a convenient location – which can then be found by your pursuers who can then identify you. If Akshay Kumar’s signature tune in the movie was Indian classical, he is unlikely to have had sheet music lying around in his office, and thus not got caught!

Now, if this is the way that stupid guys identify a conman, you can imagine how bad the rest of the movie might be. As if it wasn’t absurd enough, they’ve even tried to shoehorn some senti-max social messaging into the movie, making it utterly bizarre.

And once again I must point out that I didn’t really watch the movie – I just occasionally  eavesdropped as the wife watched it!

Songs for sleeping

As I write this, Berry is fast asleep next to me. It took a long time, and a fair amount of effort, to get her to sleep, as has become the routine everyday. Finally, she fell asleep as Pink Floyd’s Comfortably Numb was playing. This was no coincidence. This is part of a careful sleeping routine I’ve developed over the last month.

It started with a bit of what I can describe as “reinforcement learning”. We were on the way to the airport sometime last month and Berry was getting cranky in the cab, so I started singing to her. On a whim I started singing Pink Floyd songs (maybe because I know the lyrics of a lot of them). She passed out halfway through Wish You Were Here. A couple of hours later on the flight, she felt drowsy during the same song, and then slept when I started singing Comfortably Numb.

So every time I found that she would sleep to a particular song, I started singing that the next time I was putting her to sleep. Obviously it didn’t work like that – her falling asleep was a random event, which I chose to infer was a cause of my singing. And I’m someone who gives lectures on not mistaking correlation for causation.

Singing got tiring, so soon enough I had created a playlist. The playlist to which she invariably falls asleep every day nowadays is called “lullabies“.

Here is what it looks like.

Now, you might just think that it’s a random list of Pink Floyd songs, with one LedZep song thrown in. It’s not. The songs have all been carefully selected.

The first set of songs have been chosen because they are heavy on lyrics, don’t have long instrumentals and are easy to sing along to. These are songs that play when Berry is about to fall asleep, and I sing them while patting her. And invariably she falls asleep during this time.

The next few songs are long soothing songs, that will keep her asleep until she gets into deep sleep. As I write this, Atom Heart Mother is playing.

But getting Berry to sleep is not easy. I don’t start the evening with these lullabies – they come in only when I know that Berry is sufficiently sleepy and will sleep in the next 10-15 minutes (like the closer in Baseball). When she comes into the bedroom, I start with this playlist that I created a couple of months back, and which I had then named as “Berry’s Education“. 

As you can see, Black Sabbath’s Iron Man heads this list. It is Berry’s favourite song. In fact, when she gets on to the bed, she says “has he lost his mind, appa”.

This playlist is not intended for sleeping, and I randomly choose a few songs to play. When Berry gets into the next stage of her slumber, where she is now ready to sleep, but not sleepy enough, she needs some lullabies. And it’s the time for Iron Man again, except this time it’s the version by RockaBye Baby.

This is the song she used to fall asleep to when she was a baby, from the time when she was barely a couple of days old. And from there I let the album play for a while until she is really ready to sleep. Which is when the lullabies playlist takes over.

As you might imagine, having multiple playlists is a pain. I normally use the kinda old iPad4 to play, and changing playlists means entering my passcode, going up one folder and then going into another playlist. You might wonder why I haven’t created one integrated playlist.

The reason is randomness, on two counts. The amount of time Berry takes to pass each stage of sleepiness is variable. So I don’t know how long I will have to play each kind of music. Also, she is moody and the way she reacts to each kind of music is a bit random. So I need to switch back and forth between the kinds of music, and so having multiple playlists is better.

On good days, I will have my phone with me, which makes it easier to switch playlists (one hand operation, touch ID to login etc) – though it’s invariably the iPad that plays the music.

So as you might have figured out, putting babies to sleep is not an easy task, which is why I’m sharing my method with you, in the hope that it might help you. What do you do to make your baby sleep?

 

Dreaming on about machine learning

I don’t know if I’ve written about this before (that might explain how I crossed 2000 blogposts last year – multiple posts about the same thing), but anyway – I’m writing this listening to Aerosmith’s Dream On.

I don’t recall when the first time was that I heard the song, but I somehow decided that it sounded like Led Zeppelin. It was before 2006, so I had no access to services such as Shazam to search effectively. So for a long time I continued to believe it was by Led Zep, and kept going through their archives to locate the song.

And then in 2006, Pandora happened. It became my full time work time listening (bless those offshored offices with fast internet and US proxies). I would seed stations with songs I liked (back then there was no option to directly play songs you liked – you could only seed stations). I discovered plenty of awesome music that way.

And then one day I had put on a Led Zeppelin station and started work. The first song was by Led Zeppelin itself. And then came Dream On. And I figured it was a song by Aerosmith. While I chided myself for not having identified the band correctly, I was happy that I hadn’t been that wrong – given that Pandora uses machine learning on song patterns to identify similar songs, that Dream On had appeared in a LedZep playlist meant that I hadn’t been too far off identifying it with that band.

Ten years on, I’m not sure why I thought Dream On was by Led Zeppelin – I don’t see any similarities any more. But maybe the algorithms know better!