Tag: time series

Statistics and machine learning

So a group of statisticians (from Cyprus and Greece) have written an easy-to-read paper comparing statistical and machine learning methods in time series forecasting, and found that statistical methods do better, both in terms of accuracy and computational complexity.

To me, there’s no surprise in the conclusion, since in the statistical methods, there is some human intelligence involved, in terms of removing seasonality, making the time series stationary and then using statistical methods that have been built specifically for time series forecasting (including some incredibly simple stuff like exponential smoothing).

Machine learning methods, on the other hand, are more general purpose – the same neural networks used for forecasting these time series, with changed parameters, can be used for predicting something else.

In a way, using machine learning for time series forecasting is like using that little screwdriver from a Swiss army knife, rather than a proper screwdriver. Yes, it might do the job, but it’s in general inefficient and not an effective use of resources.

Yet, it is important that this paper has been written since the trend in industry nowadays has been that given cheap computing power, machine learning be used for pretty much any problem, irrespective of whether it is the most appropriate method for doing so. You also see the rise of “machine learning purists” who insist that no human intelligence should “contaminate” these models, and machines should do everything.

By pointing out that statistical techniques are superior at time series forecasting compared to general machine learning techniques, the authors bring to attention that using purpose-built techniques can actually do much better, and that we can build better systems by using a combination of human and machine intelligence.

They also helpfully include this nice picture that summarises what machine learning is good for, and I wholeheartedly agree: 

The paper also has some other gems. A few samples here:

Knowing that a certain sophisticated method is not as accurate as a much simpler one is upsetting from a scientific point of view as the former requires a great deal of academic expertise and ample computer time to be applied.

 

[…] the post-sample predictions of simple statistical methods were found to be at least as accurate as the sophisticated ones. This finding was furiously objected to by theoretical statisticians [76], who claimed that a simple method being a special case of e.g. ARIMA models, could not be more accurate than the ARIMA one, refusing to accept the empirical evidence proving the opposite.

 

A problem with the academic ML forecasting literature is that the majority of published studies provide forecasts and claim satisfactory accuracies without comparing them with simple statistical methods or even naive benchmarks. Doing so raises expectations that ML methods provide accurate predictions, but without any empirical proof that this is the case.

 

At present, the issue of uncertainty has not been included in the research agenda of the ML field, leaving a huge vacuum that must be filled as estimating the uncertainty in future predictions is as important as the forecasts themselves.

Relationship Stimulus

This post doesn’t necessarily restrict its scope to romantic relationships, though I will probably use an example like that in order to illustrate the concept. The concept that I’m going to talk about any kind of bilateral relationship, be it romantic or non-romantic, or between any two people or between man and beast or between two nations.

Let us suppose Alice’s liking for Bob is a continuous variable between 0 and 1. However, Alice never directly states to Bob how much she likes him. Instead, Bob will have to infer this based on Alice’s actions. Based on a current state of the relationship (also defined as a continuous variable between 0 and 1) and on Alice’s latest action, Bob infers how much Alice likes him. There are a variety of reasons why Bob might want to use this information, but let us not go into that now. I’m sure you can come up with quite a few yourself.

Now, my hypothesis is that the relationship state (which takes into account all past information regarding Alice’s and Bob’s actions towards each other) can be modelled as an exponentially-smoothed variable of the time series of Alice’s historical liking for Bob. To restate in English, consider the last few occasions when Alice and Bob have interacted, and consider the data of how much Alice actually liked Bob during each of these rounds. What I say is that the “current level” that I defined in the earlier paragraph can be estimated using this data on how much Alice liked Bob in the last few interactions. By exponentially smoothed, I mean that the last interaction has greater weight than the one prior to that which has more weight than the interaction three steps back, and so on.

So essentially Alice’s liking for Bob cannot be determined by her latest action alone. You use the latest action in conjunction with her last few actions in order to determine how much she likes Bob. If you think of inter-personal romantic relationships, I suppose you can appreciate this better.

Now that you’ve taken a moment to think about how my above hypotheses work in the context of human romantic relationships, and having convinced yourself that this is the right model, we can move on. To simplify all that I’ve said so far, the same action by Alice towards Bob can indicate several different things about how much she now likes him. For example, Alice putting her arm around Bob’s waist when they hardly knew each other meant a completely different thing from her putting her arm around his waist now that they have been married for six months. I suppose you get the drift.

So what I’m trying to imply here is that if you are going through a rough patch, you will need to try harder and send stronger signals. When the last few interactions haven’t gone well, the “state function of the relationship” (defined a few paragraphs above) will be at a generally low level, and the other party will have a tendency to under-guess your liking for them based on your greatest actions. What might normally be seen as a statement of immense love might be seen as an apology of an apology when things aren’t so good.

It is just like an economy in depression. If the government sits back claiming business-as-usual it is likely that the economy might just get worse. What the economy needs in terms of depression is a strong Keynesian stimulus. It is similar with bilateral relationships. When the value function is low, and the relationship is effectively going through a depression, you need to give it a strong stimulus. When Alice and Bob’s state function is low, Alice will have to do something really really extraordinary to Bob in order to send out a message that she really likes him.

And just one round of Keynesian stimulus is unlikely to save the economy. There is a danger that given the low state function, the economy might fall back into depression. Similarly when you are trying to get a relationship out of a “depressed” state, you will need to do something awesome in the next few rounds of interaction in order to make an impact. If you, like Little Bo Peep, decide that “leave ’em alone, they will come home”, you are in danger of becoming like Japan in the 90s when absolute stagnation happened.