AT: What do you say to scepticism about the viability of complex AI systems in financial markets?
Marco: The debate between simple and complex can be framed in many different ways. One way of looking at this is that simple systems tend to not have the characteristics of having [an] adaptive and intelligent nature that we believe is critical for predictive systems. Most systems involved in systematic trading tend to identify trends as opposed to predict price behaviour and are based on simple rules that have been pre-determined by the portfolio manager.
Those simple systems tend to work very well in trending markets. The problem is that if the markets are not trending, for example when they are choppy or bound within a tight trading range, the signals are not as clear, and the systems get caught out. This also happens when the markets reverse very quickly when a trend ends.
AT: What is the difference between trend following and your own adaptive systems?
Marco: What we have been working on and been developing and using are systems that are intelligent in the sense that they are able to change their own rules by themselves, as a function of the changes in the market. If, for example, they see a market that is choppy, they behave more like an oscillating system that is more reactive to short-term changes, not stuck in a single (long or short) position as if there was a strong trend. On the other hand, if they detect trends in the market our systems can recognise this and keep their positions longer, so basically our systems are designed to continuously learn and adapt to changing market conditions based on what they see; and the way they do this is by changing their own rules at each prediction point, so every day. Our systems are also short-term focused. They focus on predicting price behaviour over the next 24 hours and have an average holding period less than four days, whilst trend-following systems hold positions for several weeks or months. Our systems are also market-specific, unlike most trend-following systems.
AT: What is the mathematics/science underpinning your technologies?
Marco: It is very well established in industry outside of the world of finance so, for example, it's similar to the type of technology that is used in weather forecasting or by the online search engines. If you think of Google when you start to type your name in, Google is giving you a prediction of what you are likely to be searching for, taking into account what you searched for most recently and also what the online search community that best matches your profile has been searching. These types of industrial applications use adaptive machine learning technologies similar to the ones we have developed for financial markets.
[The technologies] all have in common this feature of being able to self-generate the predictive rules at each prediction point, as opposed to being based on fixed or pre-determined rules, which is what the simple systems do.
Some of these technologies work better than others in financial markets. So - just one example - neural networks were a real fad several years ago. A lot of people were talking about neural networks as very promising, and they proved to be a bit of a disaster.
Essentially, if you want to know whether you need to go long/short and by how much, [neural networks provide] a very detailed map of the past that can be very accurate in predicting the future, if exactly the same conditions persist in the future. But if you get something that deviates from the past, which is the norm in financial markets, the systems can get confused and what they predict often amounts to wild guesses, with little or no value - could be right, could be wrong - but their predictive accuracy is not reliable.
Our systems use machine learning technologies that are more reliable as universal approximators. Rather than memorising the past, they attempt to model it in more general forms, capturing only its most essential characteristics, and in this sense they act much more like a compass or a smart sat-nav system in predicting likely future behaviour over a broad area, than a very detailed paper-based, static street-map focused on a narrow strip of territory. For this reason they tend to be much more reliable than neural networks at inferring likely future price behaviour based on unseen data, especially in changing market conditions.