R, Windows, Mac, and Bangalore and Chennai Auto Rickshaws

R on Windows is like a Bangalore auto rickshaw, R on Mac is a Chennai auto rickshaw. Let me explain.

For a long time now I’ve been using R for all my data management and manipulation and analysis and what not. Till two months back I did so on a Windows laptop and a desktop. The laptop had 8 GB RAM and the desktop had 16GB RAM. I would handle large datasets, and sometimes when I would try to do something complicated that required the use of more memory space than the computer had, the process would fail, saying “fail to allocate X GB of memory”. On Windows R would not creep into the hard disk, into virtual memory territory.

In other words it was like a Bangalore auto rickshaw, which plies mostly on meter but refuses to come to areas that are outside the driver’s “zone”. A binary decision. A yes or a no. No concept of price discrimination.

The Mac, which I’ve been using for the last two months, behaves differently. This one has only 8GB of RAM, but I’m able to handle large datasets without ever running out of memory. How is this achieved? By means of using the system’s Virtual Memory. This means the system doesn’t run out of memory, I haven’t received the “can’t allocate memory” error even once on this Mac.

So the catch here is that the virtual memory (despite having a SSD hard disk) is painfully slow, and it takes a much longer time for the program to read and write from the memory than it does with the main memory. This means that processes that need more than 8 GB of RAM (I frequently end up running such queries) execute, but take a really long time to do so.

This is like Chennai auto rickshaws, who never say “no” but make sure they charge a price that will well compensate them for the distance and time and trouble and effort, and a bit more.

Clusters, not champions

Recently, Nokia decided to shut its factory in Sriperumbudur, following a tax dispute with the Indian government and the loss of orders from Microsoft, who took over most of Nokia’s cellphone business (in fact, all of it apart from the Sriperumbudur factory). Consequently, it has had to let go of most of its workers in its plant. The results, as Mint has reported here, are not pretty at all. People who worked for years at Nokia assembling phones have suddenly found that in the absence of Nokia, there is no comparable employer in the Sriperumbudur area. There is little else in terms of mobile phone manufacturing in India – most Indian companies get their phones manufactured abroad. Only this April did MicroMax start manufacturing phones in India – in Rudraprayag, far far away from Sriperumbudur.

Around the same time that Nokia was shutting down its factory in Sriperumbudur, Yahoo! decided to shut down its software engineering operations in Bangalore. About 600 employees got sacked (with severance payouts), while a few top performers were given the option of either relocating to its California headquarters or exiting the company with an even better severance payout. In the days that have followed, there has been a massive scramble for the now ex Yahoos among software companies in India. Apart from those that are yet to make up their mind, most ex Yahoos are gainfully employed, most of them in Bangalore.

The reason for the differential outcomes in the above two cases has to do with geography. Nokia’s plant in Sriperumbudur operated in an island – in a place where there was little else in the way of mobile manufacturing units. Thus, employees of the plant (who were well paid, as per the report), came to depend wholly on Nokia for their livelihoods. It something happened to Nokia, as it did, or if for some reason their relationship with their employer or boss soured, they had nowhere to go. Even when the plant was fully functional and successful, employees of the plant were effectively “locked in” to their employers and thus had little choice in terms of their careers.

Yahoo!, on the other hand, had (and has; operations have not ceased yet) its office in Bangalore, which has emerged as an excellent hub for software engineering services in India. From the time Texas Instruments first set up its office in the city over thirty years back (apart from the weather, a guiding factor for that decision was the availability of engineering talent in Bangalore, thanks to the presence of defence PSUs in the city), Bangalore has only grown as a market for techies. Thus, there is a large number of IT companies that has made Bangalore its home, and there is now a significant market for techies of all levels and skills in the city. Thus, when Yahoo! decided to shut its factory, there was little problem in absorbing the now surplus employees.

The difference between Sriperumbudur and Bangalore was that Bangalore grew organically as an IT hub – companies invested in the city because it made economic sense for them to invest there, in terms of the ecosystem, and upward and downward linkages available in the city. Nokia, on the other hand, was “parachuted” into Sriperumbudur. The Mint story mentions that the Nokia factory was situated there after some kind of a bidding war between various states, and was essentially a political decision and not a business decision made by Nokia. Nokia was followed there by its suppliers (who are also facing an uncertain future now) but that was the end of it – there was no further growth in the mobile manufacturing ecosystem in the Sriperumbudur area. Thus when the one company that the tiny ecosystem there got dependent on decided to shut shop, the consequences were disastrous.

There are several policy lessons that can be drawn out of the above contrast. The first, and most important, lesson is that companies should not be “parachuted” into areas chosen by the government (Jane Jacobs has an excellent treatment of this in her book Cities and the Wealth of Nations). Instead, investment should happen “organically”, by means of decisions that make business sense to the investor without relying on artificial sweeteners such as tax breaks (this might mean that no investment might happen in certain areas, but that only means that such areas are overpopulated and people from there should move to areas where investment happens).

As a corollary of the above lessons, we need land acquisition laws that are friendly to both investors and landowners – the current law passed last year is friendly to neither, but simply enhances the power of the government. The key to the building of large industrial clusters (especially in heavy manufacturing industries that the current government plans to promote) is that companies will want to be situated close to other similar companies, or companies that form an upward or downward linkage. This means the company should have the ability to set up shop close to the existing cluster, and that means being able to purchase the land at a “reasonable” (both to the company and the land owner) price. A land acquisition law which empowers the government at the cost of the transacting parties prevents such organic expansion from taking place.

Thirdly, governments should stop promoting “champions”. The assumption is that once you have a “champion” of a particular sector set up shop in a particular area, it will automatically lead to an ecosystem for that sector in the area. As the Nokia story demonstrates, this is not necessarily the case. What the government should do is instead simply be an enabler for clusters, and stop looking for “champions” to establish such clusters. In fact, it is not necessary for a big investment by an existing company for a cluster to develop, and this needs to be recognised.

Conventional wisdom regarding the success of the software engineering industry in India is that it succeeded “in spite of” rather than “because of” the government – being a new sector it was lightly regulated that resulted in its rapid growth and success. While the contribution of the government in the overall success of the industry might be debated, what is clear is that lack of government intervention has helped the industry to build around a few “clusters”, and that has only made the industry stronger.

Market forces

This morning I refused to board an auto rickshaw since it had one of those old analogue metres. Most autos in Bangalore nowadays use digital metres, which is the regulation. Except a few like the one I saw in the morning.

Now, given that most autos have digital metres people have a choice to choose only such autos. I’m sure the driver I met this morning will realise soon enough that he’s not getting as much business as he can due to his old metre, and make the switch.

It’s similar with usage of metres. In some parts of Bangalore it’s the norm for auto rickshaws to ply by metre. In such areas any driver who tries to make a quick buck by negotiating a higher fare is likely to lose customers. When a customer knows that after letting go of an auto which asked for excess fare, he had a good chance of finding one that will go by the regulated fare, he is less likely to heed to the demand for excess fare.

You can think of this being a case of what Malcolm gladwell calls the tipping point – once markets have tipped to one side (let’s say using regulated fares for auto rides) there is positive reinforcement that leads to an overwhelming move in that direction.

To get back to the metre example, when the fares increased a few months back traffic cops in Bangalore ran a drive where they checked for auto metres and fined those who had not made the switch by a particular date. Maybe that’s led to about 95% of the metres getting recalibrated. The beauty here is that market forces will take care of pushing this 95% to 100% and cops need not spend any more time and energy on enforcing this! Similarly if cops want to enforce usage of regulated  fares they would waste time by doing this drive in areas where most rides are by metre – the focus should be on tipping the other areas over!

To summarise, some parts of regulation gets enforced by sheer market forces, and regulators should not be wasting their energies there. Focus should instead be given to those areas where market failure is extreme – for that is where regulation has a role to play.

Extension of nightlife in Bangalore

Finally the Government of Karnataka has bitten the bullet and announced that restaurants will be allowed to open till 1 am every night, and bars will be allowed to open until 1 am on Fridays and Saturdays. The catch, however, is that this is just a pilot move, and the government will take a decision on making this extension permanent based on feedback from various stakeholders after the pilot period.

The problems with having a three-month pilot period are several. For starters, the Bangalore city police is already over-stressed, and in three months (with only a pilot scheme) it is impossible to recruit and train more policemen who will be required to maintain law and order at the late hours. The result of this will be that the existing policemen will get over-stressed, with extended working hours, which cannot be good for policing in general. And as my colleague Saurabh points out, the police might vote to not extend the deadline beyond the pilot period.

The next problem is with the businesses themselves. A number of restaurants, I’m given to understand, were not in favour of the extension of deadline since they did not think the extended hours of working would be adequately compensated in terms of revenues. Take for example a typical “Family restaurant” (like Shanti Sagar). If most of the restaurant’s regular clientile is families, they are unlikely to provide any incremental business in the extended hours, and thus it doesn’t make sense for such restaurants to be open late in the night.

One constituency that should normally welcome later opening hours are the offshored IT and BPO workers, a large number of whom reside in Bangalore. However, in response to the early shutdown of the city, IT and BPO firms have adapted over the years, and arranged for in-house food and transport for their employees. While life should become theoretically easier for these workers with the extended hours (giving them wider choice for food and transport), three months, and that too a pilot scheme, is not enough for them to change their behaviour. So it is unlikely that these people will take advantage of extended opening hours.

Then, to be open for two additional hours in the night, bars and restaurants will need to make further investments in terms of personnel. However, if the extension in deadline is only for three months, there is no way any bar can realistically invest in the necessary personnel and infrastructure to be open in the late hours. This is likely to further mute the response to the pilot.

Finally, it needs to be noted that there are strong network effects involved in maintaining a night life. The streets of a city will be safe late at night only if they are busy. The streets of residential areas are unlikely to be busy at that hour (in fact, they are empty as early as 9:30 pm in some areas), but what we need for successful night life is a cluster of bars, restaurants, theaters and other “happening” places in small geographical areas that ensure large human traffic in those areas, which makes them safe. You wouldn’t, for example, feel safe traveling back on entirely empty streets from the pub to your home. What this implies is the need for organic growth of night-life. Abruptly shifting the deadline, and that too on a temporary basis, is unlikely to have an impact.

It appears that the three-month pilot for extension of deadlines is a policy that is designed to fail. Three months is too short a time for any of the stakeholders to make any realistic investments or behavioural changes, and given the network effects involved, it is unlikely that we are going to have a critical mass of establishments and people who will take advantage of the extended deadline for the policy change to be made permanent.

Expect the extension in deadline to be rolled back as soon as the three months of pilot are over.

Reforming Bangalore’s Public Transport Network

This is based on a twitter rant on the same subject a few weeks back.

Bangalore’s public transport network has traditionally followed a hub-and-spoke model, with three hubs – Kempegowda Bus Station (aka “Majestic”), KR Market and Shivajinagar. It can be modeled, however, as a two-hub system, for Majestic and Market are quite close to each other and thus quite well-connected. It was probably not originally meant to be that way – for bus number 1 (not sure it still exists) ran from Jayanagar 4th Block to Yeshwantpur – basically from the south to the north-west corner of the city. Of course, it passed through Market.

Over time, however, the bus system has moved to an increasingly hub-and-spoke model. The BMTC (Bangalore Metropolitan Transport Corporation) did one exercise a few years back, trying to rationalize routes (it was partly due to an effort led by Ashwin Mahesh of Mapunity). However, while adding useful additions such as the ring routes (the “big circle” and the “chikka (small) circle” routes) and one or two “trunk routes” (that run right across town), what this revised template does is to further increase the primacy of the hubs. For example, the much talked about Big 10 routes are essentially arterial routes running from a point in the middle of town to some place along one of the highways leading out of Bangalore (they are not strictly hub routes, though, since some of them stop a short distance from a major hub).

The increase in primacy of hubs combined with metro construction (the two metro lines will criss-cross each othe at – you guessed it – Majestic!) has completely overwhelmed the hubs. It is impossible (unless you sacrifice copious amounts of time) to change buses at Majestic now, for the amount of time it takes for a bus to get into majestic and for a bus to get out of majestic is too high a transaction cost.

Moreover, changing buses at a terminus is not efficient, given the waiting times involved and the extra transaction costs of getting out of the terminus. What works better is changing buses at an intermediate stop. To use an anecdote, for two years (1998-2000) I traveled to school in Indiranagar (east Bangalore) from my home in Jayanagar (south Bangalore). I would take a bus going to Shivajinagar (Jayanagar-Shivajinagar is well connected – being a hub route) and get off at Richmond circle, from where I would take a bus from Majestic to Indiranagar (again a hub route, so well served). I could change buses while standing at the same bus stop (made things easier), and the frequency of buses on the two hub routes meant I would get to school easily (again the traffic in the 1990s was nothing compared to what it is now). I had the option of changing buses at a hub, but eschewed it due to transaction costs.

Coming back, what we need in Bangalore is to reformat the bus network in a way that mimics the patterns in which people travel. Right now the assumption of the BMTC seems to be that they should connect every area to a major hub, and then let people take it from there. What they do not take into account is that 1. traffic has grown much worse and 2. People put a higher value on their time nowadays, because of which the transaction cost of the old hub-and-spoke model is way too high. What they need to do instead is to design the network based on people flows.

The first step of such reform is to understand the patterns in which Bangalore moves. One way to do this would be via smart ticketing. A few years back buses in Bangalore started introducing smart ticketing machines, and your ticket would be a printout. However, that didn’t take off. If that can be reintroduced (in all buses) and coupled with destination based ticketing rather than leg based ticketing (for example, if I’m going from Jayanagar to Indiranagar via Richmond Circle I get on to the bus in Jayanagar and buy a ticket to Indiranagar directly. The same ticket allows me to travel on any bus between Richmond Circle and Indiranagar. This introduces complexity but can be done). This will give the BMTC information in terms of the routes on which people actually travel. And once that happens, an effort can be made to reformat the bus network.

Analyzing Bangalore’s Growth

Banglore’s population has grown 20-fold and area 10-fold since 1941, going by this chart (via Gautam John on Facebook, photo taken at MG Road Metro station).

Bangalore Population and Area
Bangalore Population and Area

What would be interesting to see is when the spurt in Bangalore population actually happened. Checking that is quite simple. Using the population figures from the census, we can derive the CAGR (compounded annual growth rate) of the population in each decade. This is presented in the chart below:

bangalorepop

Conventional wisdom is that Bangalore was a sleepy little city until the “IT revolution” happened around the turn of the millennium after which the city exploded. The chart above calls that wisdom into question. While the annual growth rate of Bangalore’s population has been higher in the noughties compared to the earlier two decades, this is by no means the period of Bangalore’s fastest growth.

Bangalore grew fastest in the 1940s, perhaps because it was made capital of Mysore State after independence, thus leading to the arrival of a large number of government servants in the city. Interestingly, the next period of high growth in the city was in the 1970s, which was even before the seeds of the IT revolution had been sold (the setting up of the Texas Instruments office in Bangalore in the early 80s is regarded as the beginning of Bangalore as an IT hub).

What might have led to the perception of Bangalore’s growth being fastest in the noughties is that the strain on a city’s infrastructure is a superlinear function of the city’s population. And with a lot of the city’s infrastructure having been stagnant over the years, the strain started getting really noticed in this decade.

Landmark mismanagement

Yesterday’s Landmark Quiz in Bangalore was a major waste of time. No, I’m not talking about the quality the quiz here – the prelims was among the better Landmark Quiz prelims I’ve sat through, and given that we just missed out on qualification for the finals (AJMd, as we say here in Bangalore) I didn’t sit through the finals though I was told the questions there too were pretty good.

I’m talking about the transaction costs of attending the quiz. The overall management of the event left much to be desired. First of all, we had to show up at the venue at 11:45 for a quiz that was supposed to start at 1:45 pm. Teams with confirmed seats were let in at around 12:30 and only around 1 o’clock were us “waitlisted” teams let in. There too, the organizers did a major show of letting in waitlisted teams, calling them in order and taking over half an hour to let everyone in.

The point is that even after all the waitlisted teams had been let in, there was plenty of room in the auditorium. This makes me wonder about the wisdom of waitlisting so many teams, and then making such a big show of letting people in. Given that the total turnout was much smaller than the hall capacity, things would have been much simpler if people had been simply left in, with volunteers only ensuring that the seating was efficient (without leaving gaps).

Before the quiz yesterday i started writing a blog post on how the quiz registration process was itself flawed, and gave incentive to people to register zombie teams because the option of registering a team came free. So while the hall had been theoretically filled up many days ago, most of these registrations were zombie registrations thus leading to a long wait list and thus calling people early. Given that the quiz doesn’t have an entry fee, I can’t currently think of a good way to price this option.

But reaching the venue early was not the only waste of time. The written prelims of the quiz finished around 3 pm, including calling out the correct answers. The results, however, weren’t announced till close to 6 o’clock. In the interim time period there was the finals for school students, but that still doesn’t explain why they had to wait until 6 o’clock to announce the results of the senior quiz.

The way I see it, it was sheer disrespect on the part of the organizers of the time of the participants. Yes, Landmark might be a much sought after quiz, rated among the best in the country. Yes, most people come there for the questions and not just to win – and so stay on to watch the finals even when they haven’t qualified (it is indeed commendable that Landmark quizzes have managed to be great spectator events while not dropping quality). Yes, many participants have traveled from other cities and so having traveled the cost of their time might be “cheap” – in that they have little else to do in the rest of the day.

Even taking into account all these, the wastage of 5 hours of each quizzer’s time (2 hours for early reporting; 3 hours gap between prelims and results announcement; 4 if you consider that watching the Junior finals wasn’t a waste of time) is not a done thing. Given the quiz’s unparalleled reputation it is unlikely that market forces are going to tell the organizers that they are wasting people’s time, but the message has to go through.