So after much deliberation and procrastination, I’ve finally started a newsletter. I call it “the art of data science” and the title should be self-explanatory. It’s pure unbridled opinion (the kind of which usually goes on this blog), except that I only write about one topic there.
I intend to have three sections and then a “chart of the edition” (note how cleverly I’ve named this section to avoid giving much away on the frequency of the newsletter!). This edition, though, I ended up putting too much harikathe, so I restricted to two sections before the chart.
I intend to talk a bit each edition about some philosophical part of dealing with data (this section got a miss this time), a bit on data analysis methods (I went a bit meta on this this time) and a little bit on programming languages (which I used for bitching a bit).
And that I plan to put a “chart of the edition” means I need to read newspapers a lot more, since you are much more likely to find gems (in either direction) there than elsewhere. For the first edition, I picked off a good graph I’d seen on Twitter, and it’s about Hull City!
Anyway, enough of this meta-harikathe. You can read the first edition of the newsletter here. In case you want to get it in your inbox each week/fortnight/whenever I decide to write it, then subscribe here!
And put feedback (by email, not comments here) on what you think of the newsletter!
A rather uncharitable view of the book advances that legacy publishing houses give out to established (non first-time) authors is to look at it as “convertible debt”, as this piece by Matthew Yglesias points out.
An advance is bundled with a royalty agreement in which a majority of the sales revenue is allocated to someone other than the author of the book. In its role as venture capitalist, the publisher is effectively issuing what’s called convertible debt in corporate finance circles — a risky loan that becomes an ownership stake in the project if it succeeds.
Now, as I consider possibly self-publishing my book, while simultaneously attempting to sell it to established publishing houses, I realise that apart from the convertible debt, publishing a book also involves a massive sale of equity.
I’ve finished the manuscript, and edited it once. It needs further editing, but I’ve put it off so far in the hope that I can sell the book to a mainstream publisher, who will then take care of the publishing. However, given that I might end up self-publishing, and from what I read that publishers don’t do a great job of editing anyway, I might need another pass or two.
And then there are other things to be done before the book comes out in print – a cover needs to be designed, illustrations need to be put in, maybe we should get someone to do an audiobook, and all such. Now, if a mainstream publisher picks up the book, I’d expect them to take care of these. Else I’ll need to spend to get people to do these things for me.
When people first told me that royalties in book publishing are of the order of 7.5% of cover price, it was a little hard to believe. However, looking at the costs involved in the publishing process, it’s not hard to see why publishers take the cut that they take. The problem, though, is that it involves you selling equity in your book.
By going for a mainstream publisher (rather than self-publishing), you are saving yourself the upfront cost of getting your book edited, designed and “typeset”, in exchange for a large portion of the equity of the book.
Looking at it in another way, you are trading in your limited downside (what you spend in designing, printing, etc.) for what might be a massively unlimited upside (in case my book is a runaway success). For the most part, considering that most books don’t do that well, it isn’t a very bad deal. However, considering that downside is limited (in terms of costs) I wonder if it makes sense to trade it in for a large stake of what could be a large upside.
In any case, the main reason I’m still pushing to get mainstream publishers is because the self-publishing market is a “market for lemons“. With barrier to entry not being too high, lots of bad books are self-published, and so anyone who thinks they’ve written a half decent book will try to find a mainstream publisher. And this further diminishes the average quality of self-published books. And further dissuades people like me from self-publishing!
It is now a well-documented fact (that I’m too lazy to google and provide links) that there exists a relationship between mental illness and creative professions such as writing.
Most pieces that talk about this relationship draw the causality in one way – that the mental illness helped the writer (or painter or filmmaker or whoever) focus and channel emotions into the product.
Having taken treatment for depression in the past, and having just finished a manuscript of a book, I might tend to agree that there exists a relationship between creativity and depression. However, I wonder if the causality runs the other way.
I’ve mentioned here a couple of months back that writing a book is hard because you are working months together with little tangible feedback, and there’s a real possibility that it might flop miserably. Soncequently, you put fight to make the product as good as you can.
In the absence of feedback, you are your greatest critic, and you read, and re-read what you’ve written; you edit, and re-edit your passages until you’re convinced that they’re as good as they can be.
You get obsessed with your product. You start thinking that if it’s not perfect it is all doomed. You downplay the (rather large) random component that might affect the success of the product, and instead focus on making it as perfect as you can.
And this obsession can drive you mad. There are days when you sit with your manuscript and feel useless. There are times when you want to chuck months’ effort down the drain. And that depresses you. And affects other parts of your life, mostly negatively!
Again it’s rather early that I’m writing this blog post now – at a time when I’m yet to start marketing my book to publishers. However, it’s important that I document this relationship and causality now – before either spectacular success or massive failure take me over!
While I look for publishers for the manuscript that I’ve just finished (it’s in “alpha testing” now), I think it’s a good time to write about what it was like to write the book. Now, I should ideally be writing this after it has been published and declared a grand success.
But there are two problems with that. Firstly, the book may not be a success of any kind. Secondly, it will be way too long after having finished it to remember what it was like to write it. In fact, a week after the first draft, I’ve almost already forgotten what it was like. So I’m writing this now.
Writing is a full-time job. I got this idea for the book in October 2014 when I was visiting Barcelona for the first time. I wrote the outline in November 2014. Despite several attempts to write, nothing came out of it.
During a break from work in October 2015 I managed to get started, but I’ve re-written all that I wrote then. Part-time effort doesn’t just cut it. It wasn’t until I came to Barcelona in February that I could focus completely on the book and write it.
You need discipline. This probably doesn’t need to be explicitly stated, but writing a book, unlike writing a blog post, is a fighter process, and you need a whole load of discipline and focus. After a week or two of preparing the outline, I prepared fairly strict deadline regarding when I would finish the book. I had to reset the deadline a couple of times, but finally managed it.
There is no feedback. I think I wrote about this a few days back. The big problem with writing a book is that you spend a significant amount of effort before even a small fraction of your customers have seen the product. So you soldier on without any feedback, and it can occasionally be damn frustrating.
You feel useless. Writing a book can introduce tremendous amounts of self-doubt. One day you think you’ve completely cracked it, and your book will change the world. The next day you start wondering if there’s any substance at all to what you’re writing, and there’s any point in going ahead with it. On several occasions, I’ve had thoughts on abandoning it.
Getting away helps. The only reason I didn’t abandon the book when I had my bouts of self-doubt was that I was away in Barcelona with nothing else to do. It wasn’t as if I could ditch the book and find some work to do the next day. Being away meant that the TINA factor pushed me on. There was no alternative but to write the book.
Getting in a draft is important. You are likely to have bad days when you’re writing. On those days you feel like giving up. On putting things off for another day. Reams have been written about great writers stalling their books for several days because they couldn’t find the “right word”. I don’t buy that.
Found that when I’m in a rut, it’s better I simply push through and finish the chapter. Editing it later on is far easier than writing it again from scratch.
There is a limit to how much you can write. When I said it’s a full time job you might think I spent 8 hours a day on the book. I took around 70 days to write it (including a 10-day vacation), and the draft weighs in at 75,000 words (I intend to cut it before publication). So it’s less than 1200 words per day on an average.
That doesn’t sound like a lot, but trust me, writing on a continuous basis is quite hard. A lot of time goes in fact checks and in getting links (I don’t think I still have all the footnotes and endnotes I need for the book). Writing a book is far more complex than writing a blog post.
Writing is tiring. This isn’t something I figured out while writing the 2000 odd posts I’ve put on this blog. When you’re writing a book, and for an audience, you realise that you get tired pretty quickly. I don’t think I was able to work more than four hours a day on any of my “writing days”. And four hour-days would leave me a zombie.
You need a schedule, and a workplace. I did the pseud romantic thing. The entire book was written at this WiFi enabled cafe near my place in Barcelona. Pseud value apart, the point of having the workplace was that it brought a schedule and some discipline to my days. I would go there every morning on writing days (exact time varied), get a coffee and sit down to write. And not rise until I had finished my target for the session.
Two days back I went there to work on something else. I figured I couldn’t – that cafe is now forever tied to my writing the book. The kind of focus required there was of a different kind.
I’ll stop for now. I hope to republish this blog post once the book has hit the stands!
When Dhoni was bantering with the Australian, the other journalists in the hall were laughing. They would, no sports journalist would want to be anything but nice to the formidable Indian captain. That’s why this piece had to be written by someone whose day job is to write on business and economy.
Looking at the reports of the incidents from both Sinha and EspnCricinfo’s standpoints, it is clear to me that Sinha’s view is more logical. That Dhoni’s calling of the journalist to the press conference table and cross-questioning him was unprofessional on the one hand and showed his lack of defences on the other.
Yet, the ending to Sinha’s piece also explains why other sports journalists have taken to lauding Dhoni’s view rather than critisicing him – for them, access to the Indian limited overs captain is important, and they wouldn’t like to damage that by taking an Australian colleague’s side.
The problem with a lot of sports journalism in general, and Indian cricket journalism in particular, is that jingoism and support for one’s team trumps objective reporting and analysis. One example of this was coverage from Indian and Australian newspapers of the Monkeygate scandal in 2007-08 (when Harbhajan Singh called Andrew Symonds a monkey).
More recently, there was the controversy about India losing games because of the tendency of Rohit Sharma (and Indian batsmen in general) to slow down in their 90s. Again, commentary about that took jingoistic tones, with the Indian sports media coming out strongly in favour of Sharma. There were reports defending his “commitment” and “grit” and all such flowery language sports journalists love, and that Glenn Maxwell’s comment was entirely unwarranted. Maxwell even backed down on his comments.
Data, however, showed that Maxwell need not have backed down on his comments. Some analysis based on ball-by-ball data that I published in Mint showed clearly that Indian batsmen do slow down in their 90s, and of all recent players, Sharma was the biggest culprit.
The piece was a hit and was widely shared on social media. What was more interesting, however, was the patterns in which it was shared. For one, the editors at Mint loved it and shared it widely. It was also shared widely by mango people and people with a general interest in cricket.
The class of people which was conspicuous by its absence of commentary on my piece was sports journalists. While it could be reasoned that they didn’t see the piece (appearing as it did in a business publication, though I did send emails to some of them), my reasoning is that this piece didn’t gain much traction among them because it didn’t fit their priors, and didn’t fit the jingoistic narrative they had been building.
It is not necessary, though, that someone only shares pieces that they completely agree with – it is a fairly common practice to share (and abuse) pieces which you vehemently disagree with. The commentary I found about this piece was broadly positive – few people who had shared the piece disagreed with it.
My (untested) hypothesis on this is that this analysis flew in the face of all that mainstream sports journalists had been defending over the previous few days – that Maxwell’s comments were simply not true, or that Sharma was a committed cricketer, and all such hyperbole. With data being harder to refute (only option being to poke holes in the analysis, but this analysis was rather straightforward), they chose to not give it further publicity.
Of course, I might be taking too much credit here, but that doesn’t take away from the fact that there is a problem with sports (and more specifically, cricket) writing. Oh, and as for the ultra-flowery language, I’ll save my comments for another day and another post.
I watched half of Exodus: Gods and Kings last night (I’d DVRd it a few days back seeing it’s by Ridley Scott). The movie started alright, and the story was well told. Of Moses’s fight with Rameses, of Moses being found out, of his exile and struggle and love story and finding god on a mountain. All very nice and well within the realms of good mythology.
And then Moses decides to hear god’s word and goes to Memphis to free his fellow Hebrews. There’s a conspiracy hatched. Sabotage begins. Standard guerrilla stuff that slaves ought to do to revolt against their masters. Up to that point in time I’d classified Exodus as a good movie.
And then things started getting bad. God told Moses that the latter wasn’t “doing enough” and god would do things his way. And so the Nile got polluted. Plants died. Animals died. Insects attacked. Birds attacked (like in that Hitchcock movie). What had been shaping up to be a good slave-revolt story suddenly went awry. The entire movie could be described by this one scene in Indiana Jones and Raiders of the Lost Ark:
When you see the guy twirling the sword, you set yourself up for a good fight. And then Indiana just pulls out a gun and shoots him! As a subplot in that movie, it was rather funny. But if the entire plot of a movie centres around one such incident (god sending the plague to Egypt, in this case), it’s hard to continue watching.
Checking out the movie on IMDB, I realised that it has a pretty low rating and didn’t recover its investment. While this is surprising given the reputation of Scott, and how the first part of the movie is set up and made, looking at the overall plot it isn’t that surprising. The problem with the movie is that it builds on an inherently weak plot, so the failure is not unexpected.
It did not help that I was reading mythology, or a realistic mythological interpretation, earlier in the day – the English translation of SL Bhyrappa’s Parva. In that, Bhyrappa has taken an already complex epic, and added his own degrees of complexity to it by seeking to remove all divinity and humanise the characters. Each major character has a long monologue (I’m about a third into the book), which explores deep philosophical matters such as “what is Dharma”, etc.
While moving directly from humanised philosophical myth to unabashedly religious story might have prevented me from appreciating the latter, it still doesn’t absolve the rather simplistic nature of the latter myth. I admit I’m generalising based on one data point, not having read any Christian myth, but from this one data point, it seems Christian myth seems rather weak compared to Hindu or Greek or Roman myth.
My explanation for this is that unlike other myths, Christian myth didn’t have enough time to evolve before it was written down. While the oral tradition meant that much valuable human memory was wasted in mugging up stories and songs, and that transmission was never exact, it also meant that there was room for the stories to evolve. Having been transmitted through oral tradition for several centuries, Hindu, Greek and Roman stories were able to evolve and become stronger. Ultimately when they got written down, it was in much evolved “best of” form. In fact, some of these myths got written down in multiple forms which allowed them to evolve even after writing came by.
While writing saves human memory space and prevents distortions, it leaves no room for variations or improvisation. Since there is now an “original book”, and such books are determined to be “words of God”, there is no room for improvisation or reinterpretation. So we are left with the same simplistic story that we started of with. I hope this explains why Exodus, despite a stud director, is a weak movie.
There are certain pieces that are a breeze to write – you start writing, the thoughts flow, the words flow, your fingers do the needful and before you know it you’ve written a thousand words. Once you’ve published you’re feeling all good and high and kicked to take on your next task for the day.
But then there’s writing that can drain you. For example I just wrote a piece on my policy blog. It took forty minutes to write. I kept hitting backspace and cancelling out sentences. It was extremely laboured. And having written that I’m feeling all completely drained out. This blog post on the other hand is unlikely to have that effect.
I realise that there are two kinds of pieces that I write – the first are “flow pieces” – where I do the thinking as I write. I start writing only with an initial sketch, or paragraph, or opening line. And then I build the piece linearly thinking as I write along. These pieces are a breeze and a pleasure to write. And it is incredible the number of insights I stumble upon while writing such pieces.
On the other hand you have “planned pieces” – where you know exactly what you want to communicate and how, and you only have to implement it and put it all together while you’re writing. The problem with such writing is that you would have imagined certain sentences at different points in time while thinking of the ideas and now you try and fit all those sentences into a coherent piece. And that leads to a lot of jigsaw-fitting and labouring and backspacing. It’s hell!
For a while I wanted to write Op-Eds, but I’ve now simply given up on those. It requires me to write in an impersonal formal voice which is something I find extremely hard to summon. And such pieces are more likely than not planned pieces, and writing them can be extremely draining. I’d rather write when I want to, building pieces as I please, and publishing them as blog posts! The effort to write Op-Eds is simply not worth it!
Andrew Gelman has a nice piece on why academic writing is bad. Basically two points – writing is hard, and academic pieces are not selected based on their quality of writing. So the quality of writing in such pieces is far inferior to say writing in a newspaper!
As you might have noticed from my blog posts over the years, I like writing long essays. By long, I mean blog post long. Somewhere of the length of 800-1000 words. I can’t write longer than that, because of which my attempts to write a book have come to nought.
Now, thanks to regular blogging for over nine years, I think I’ve become better at writing rather than speaking when I have to explain a complicated concept. Writing allows me to structure my thoughts better, whereas while speaking I sometimes tend to think ahead of what I’m talking, and end up making a mess of it (I had a major stammer when I was in school, by the way).
Given that I like explaining concepts in writing rather than in speech, I write long mails even when it comes to work. Writing long emails is like writing blog posts – you have the time and space to structure your thought well and present it to your readers. This especially helps if the thoughts you are to communicate are complex.
The problem, however, is that most people are not used to reading long emails in a work contexts. People prefer to do meetings instead. Or they just call you up. For whatever reason, the art of long emails has never really taken off in the corporate sphere, Maybe people just want to talk too much.
This, of course, has never deterred me from using my favourite means of communication. It didn’t stop me when I was an employee and the people I wrote to were colleagues. It still doesn’t stop me now, when I’m a consultant, writing to people who are paying me for a piece of work. If they are paying me, I should communicate things to them in a form they are most comfortable with, you might argue. If they are paying me, I should communicate things as well as I can, I argue back, and my best means of communication is writing long emails.
The problem with long emails, however, is that, like long-form articles you send to a Pocket or an Instapaper, you tend to bookmark these long mails for later, intending to read and digest them when you have the time. So, when you send a long email, you are unlikely to get a quick response (note that you can sometimes use it to your advantage). This means that when you write long mails, you might have to follow it up with an SMS or a phone call to the effect of “read and digest and let me know if you have any questions”.
In my last organization, I worked with a number of technical people, some of whom had PhDs. It was interesting to contrast the way they communicated with my long emails. They too would put complex thoughts in writing, except that they would use Latex and make a PDF out of it. It would be littered with equations and greek symbols, in a way that is extremely intuitive for an academic to read.
And here I was, eschewing all that Greek, preferring to write in plain text in the body of emails. No wonder some of my colleagues started terming my emails “blogposts”.
There have been times in the past when I would have read a book and then concluded that “it’s a blog post expanded into a book”. One book that I clearly remember that followed this model was Malcolm Gladwell’s “The Tipping Point”. An idea that can be easily explained in 3000 words instead taking 30000, so that it can then qualify to be a “book”, the economics of whose publications are much different from that of a “long form article”. I remember thinking this even more about this book called “Why Popcorn Costs So Much At The Movies“. It was all about price discrimination, a concept that could have been explained well in a blog length article (500-1000 words). Even a long-form article would have been too much for it.
The topic of this post, however, is not about books that should have been blogposts. It is about bloggers writing books. For dinner on Saturday I met two friends who also happen to be renowned twitter trolls. Somewhere between the soups and the pizzas the conversation moved to books being written by bloggers (and there are many of those). And the three of us came to the unanimous decision that bloggers are lousy at writing books (I haven’t read any of the books they were talking about, but could attest to it since I’ve been trying to write a couple of books for a couple of years now and getting nowhere).
The fundamental point is that the art of holding someone’s attention over 1000 words (the normal length of a blog post) is very different from holding someone’s attention over 50000 words (the length of a typical book). So if you’ve been a blogger for a few years now, through sheer practice you would be great at using 1000 words to put across your ideas. However, when you want to write something longer, you either get discontinuous (with lots of mini-chapters of 1000 words each) or you end up saying the same thing over and over again.
So yes, as you might have figured out from my Project Thirty/Thirty One filings, I’m writing a book. And no, it’s not about Studs and Fighters (thanks to your valuable feedback I’ve given up on that concept). I’ve been trying to write lots of small chapters. Somehow, I’m not able to go beyond 1000 words per chapter (2000 is the intention). There is a bigger problem. I begin to take myself too seriously when I think I’m writing a book. I stop writing in the informal conversational style I normally use on my blog. And it becomes excruciating, both to write and to read (I’ve tried reading some of my own “serious” pieces and given up).
Maybe all this tells me something. That having been writing this blog (and its predecessor on LiveJournal) for 9 years now, and having got many an accolade for it, I should simply stick to writing blog posts. Maybe it’s time to accept that when it comes to writing books mEre sE nahIn hOga
So today I made my way to Gayatri Vihar in the Palace Grounds to visit the Bangalore Book Festival, on its last day. It was interesting, though a bit crowded (what would you expect on the last day of an exhibition? and that too, when it’s a Sunday?). I didn’t buy much (just picked up two books) given the massive unread pile that lies at home. However, there was much scope for pertinent observations. Like I always do when I have a large number of unrelated pertinent observations, I’ll write this in bullet point form.
There were some 200 stalls. Actually, there might have been more. I didn’t keep count, despite the stalls having been numbered. Yeah, you can say that I wasn’t very observant.
All the major bookshops in Bangalore barring the multicity ones had set up shop there. I don’t really know what they were doing there. Or were they just trying to capture the market that only buys in fairs? Or did they set up stall there just to advertise themselves?
It seems like a lot of shops were trying to use the fair to get rid of inventory they wanted to discard. All they had to do was to stack all of this on one table and put a common price tag (say Rs. 50) on every book in that collection, and it was enough to draw insane crowds
One interesting stall at the fair had been set up by pothi.com an online self-publishing company. I’ll probably check them out sometime next year when I might want to publish a blook. Seems like an interesting business model they’ve got. Print on demand!
I also met the flipkart.com guys at the fair. Once again, they were there for advertising themselves. Need to check them out sometime. Given the kind of books I buy, I think online is the best place to get long tail stuff.
There was an incredibly large number of islamic publishing houses at the fair! And have you guys seen the “want qur an? call 98xxxxxxxx for free copy” hoardings all over the city? Wonder why the Bajrang Dal doesn’t target those
There was large vernacular presence at the fair. I remember reading in the papers that there was a quota for Kannada publishers, but there was reasonable presence for other languages also, like Gult, Tam, Mellu, Hindi
A large number of stalls were ideology driven. Publishing houses attached to cults had set up stalls, probably to further the cause of their own cult. So there was an ISKCON stall, a Ramakrishna Mutt stall, a Ramana Maharshi stall, etc.
Attendance at most of these niche stalls was quite thin, as people mostly crowded the stalls being run by bookstores in order to hunt for bargains. Attendance was also mostly thin at publisher-run stalls, making me wonder why most of these people had bothered to come to the fair at all.
I saw one awesomely funny banner at the place. It was by “Dr Partha Bagchi, the world leader in stammering for last 20 years” or some such thing. Was too lazy to pull out my phone and click pic. But it was a masterpiece of a banner
Another interesting ideological publisher there was “Leftword books”. Their two sales reps were in kurtas and carrying jholas (ok I made the latter part up). And they were sellling all sorts of left-wing books. Wonder who funds them! And they were also selling posters of Che for 10 bucks each
I wonder what impact this fair will have on bookstores in Bangalore in the next few days. Or probably it was mostly the non-regular book buyers who did business at the fair and so the regulars will be back at their favourite shops tomorrow.
I bought two books. Vedam Jaishankar’s Casting A Spell: A history of Karnataka cricket (I got it at Rs. 200, as opposed to a list price of Rs 500) and Ravi Vasudevan’s “Making Meaning in Indian Cinema”.