The Gateway to Algorithmic and Automated Trading

The accidental quant

Published in Automated Trader Magazine Issue 32 Q1 2014

Ernie Chan says it was not by design but by chance that he ended up in the market. As accidents go, it turned out to be a happy one. A theoretical physicist by training, he's also that rarest of traders who likes to talk openly about his ideas rather than keeping them secret. Ernie tells Automated Trader Editor Adam Cox about his research into mean reversion and momentum strategies and his journey from science to finance.

Ernie Chan

Adam: You've done a lot of work on the issue of whether to use mean reversion or trend-based strategies. Let's start with what your current research is telling you.

Ernie: Yes. In my first book I've been focusing on mean reversion strategies mostly and that actually has been my focus in my own trading as well as the strategies we run in our previous fund. And the reason is that I was sort of obsessed with Sharpe ratio. I was obsessed with generating steady profits and consistency in those profits. Now, that has a drawback. Mean reversion strategies do produce consistent profits and do produce high Sharpe ratios, but with a hidden downside. And the hidden downside is the presence of the fat tails and the outliers. Many people compare running mean reversion strategies with picking up pennies in front of a steamroller. Most of the time you get those pennies but once in a while you get run over. And that has been my experience. Other people might compare that with selling options. You gain premium most of the days but once in a while you get wipeouts.

Unless of course you impose various risk management techniques such as stop losses. But stop loss itself is problematic for mean reversion strategies because a) if you are trading stocks there's no stopping your loss during the overnight gap. You can only really effectively use stop loss for forex and futures, and even those markets you cannot apply stop loss during the weekends. So stop loss has somewhat limited benefits, unless you're trading intraday. Another problem with stop loss is that typically with a mean reversion strategy a stop loss contradicts your trading signal. So if the price goes lower, you're supposed to buy more instead of stopping out. So in principle a stop loss for a mean reversion strategy should only be placed somewhere such that in the back-test it was never breached. Once a stop loss level is breached in a mean reversion strategy, you should give up that strategy, you should never trade it again. That is a very unique stop loss that we can impose.

Cumulative Returns for a Momentum Strategy

Cumulative Returns for a Momentum Strategy

This figure shows the typical cumulative returns profile for a momentum strategy. The strategy performed superbly during the financial crisis of 2008-9, reaching a high watermark around April of 2009. But when the stock market started to recover, this strategy's performance also started a relentless decline. This is a phenomenon called "Momentum Crashes", and many commodity funds suffered a similar fate as this strategy.

Adam: In other words, the simple rule is that whatever your back test shows for a successful mean reversion strategy, the stop loss needs to be outside of that range so that you don't cancel your own strategy prematurely. You need to be prepared to lose that much.

Ernie: That is correct, yes. If you place the stop loss outside of the lowest price in a mean reversion back test, so that it will never be breached, and if in live trading it is breached, that practically invalidates your back test immediately. One should give up. Your back test says it should never be breached, so how come in live trading it's breached?

Adam: Much then depends on the extent of the back testing, so in your experience in these strategies, how big might a loss potentially be?

Ernie: Well, it very much depends on your own risk tolerance and your investors' risk tolerance. That's a pretty arbitrary number. Some people can tolerate 10% maximum drawdown, so the stop loss should be less than or equal to 10%. Other people can tolerate 50% maximum drawdown. Okay, that's great then we should place the stop loss at 50%. But again one should very much bear in mind that the stop loss is only effective only during market hours, so if one is holding a position overnight or over a weekend, all bets are off.

Adam: Typically for mean reversion strategies, are you looking at intraday or longer term?

Ernie: Mean reversion can work at multiple timescales. It can range from seconds to months. I think that there's no specific timeframe for it to work. Actually, it can work for years -- that's what Warren Buffett is betting on. But the shorter your holding period, the more effectively you can control risk.

Adam: Does it make a difference in terms of the potential quality of your back-testing if you're looking at an intraday or longer-term strategy?

Ernie: Backtesting an intraday strategy has more statistical significance than backtesting a longer-term strategy because an intraday strategy generate more trades within a given period of time.

Adam: You do a lot in terms of writing and conducting workshops, but you still manage a fund. Where do you put the bulk of your time?

Ernie: My main job is to manage my fund, and I do spend a majority of my time on research and developing new strategies on my fund. On the operational side of my fund, I'm fortunate to have a fully automated system so it doesn't take too much time to actually operate the existing strategy. In terms of the technology development, I'm also fortunate to have a partner who is much more expert in software architecture than I am. So my role in the fund is to decide on the overall strategic direction and most importantly, in doing research, to create new strategies.

Adam: How would you describe the current market environment? There's a lot of discussion about possible regime change in terms of the US Federal Reserve tapering debate. Do you think that's happening now?

Ernie: What I see in the current environment is very similar to what many traders -- as opposed to investors -- see. This is a market that I'm sure will make a lot of investors happy. It's going up every month and there's low volatility -- what's not to like? But for traders this is not a particularly favourable market. It's a market marked by low volatility. Some people attribute that to the excess liquidity being pumped in by central banks, and there may be some truth to that. Whatever the cause, the low volatility and low volume and low liquidity all conspire to diminish the returns of many strategies. But low volatility does not mean low risk. A lot of people confuse volatility with risk. Actually volatility is not risk, volatility is profit. The risk is in what statisticians call kurtosis. It is the fourth moment of the return distribution. That has not decreased.

That's the quality that measures what is not normal. What we have in the market right now is not normal. It is a very fat-tailed distribution, and perhaps the tail is getting fatter every day. We have extraordinary events that wipe out lots of day-to-day profits and those profits are very easy to be wiped out because the low volatility makes those profits very small. And that applies to mainly mean reversion strategies.

Actually you might argue momentum strategies benefit from the fat tails. But you still get fewer opportunities there. In general, low volatility has been bad for most traders and that is my main complaint about the market.

If you look at the forex market, practically every year we have lower volume than the previous year. It really bodes very poorly for the market and for participants of the market, except if you are in buy-and-hold strategies. At some point it's going to impact buy-and-hold strategies because the traders withdraw. Look at FX Concepts.... Now that they've closed down shop it has had a big impact on forex market liquidity. Where are the Warren Buffetts of the world going to get their forex liquidity? Who is going to be on the other side of Warren Buffett, should he want to make a huge foreign purchase. Let's say they want to exchange US dollars with pound sterling to buy a British company. Who is going to provide the liquidity when a major forex hedge fund shuts down and it no longer provides that liquidity. Of course the banks are still doing that but the banks are not in the business of taking risk. They are in the business of market making, And they only take a certain amount of risk. They have to pass what we would call hot potatoes along to the people who are willing ultimately to take risk. Without those hedge funds, there are no risk takers out there, and sooner or later it's going to impact the Warren Buffetts of the world. There are people who are cheering the demise of trading, there are people who are gleeful that these traders are getting hammered and getting driven out of the market so everybody is going to be happy and the long-term investors win. That is a very naive view about how the capital markets work.

Adam: Do you think the current low volatility, low volume environment will continue or does your research suggest we might be due for a regime change?

Ernie: Well, everybody is waiting for the Fed to withdraw liquidity because as I said most people attribute this low liquidity and low volatility environment to the flooding of the market with cash by the central banks. So when central banks start to withdraw some of this liquidity then hopefully the volatility will pick up and perhaps the liquidity and volume might pick up as well. However, until the US economy and unemployment gets back to a more favourable level, certainly that is just a waiting game.

"What we have in the market right now is not normal. It is a very fat-tailed distribution, and perhaps the tail is getting fatter every day."

Adam: That makes mean reversion strategies that much more difficult because the lack of volatility means there are fewer opportunities that arise when everything is so stable. What do you do instead?

Ernie: In more recent years we have really started a major research effort into momentum strategies. And that, I'm pleased to say, does bear some fruit. Momentum strategies is the opposite of mean reversion strategies in many ways, no big surprise there. First of all, in terms of Sharpe ratio, you should expect it to be lower, you should expect momentum strategy to make many small mistakes and then all of a sudden hit a home run. That's the kind of risk return profile of a momentum strategy: many small loses and one big win to make up for the small loses. And risk management for a momentum strategy is very easy, because the moment you start to lose money, typically, it means that momentum has reverted and therefore you are required, by theory and by practice, to exit the position. So a losing position is not going to lose too much, because of that stop loss. A stop loss can be placed much tighter in a momentum strategy.

Momentum also can sometimes benefit from fat tails. For example in 2008, Lehman Brothers collapses and the market drops and your momentum strategy would be shorting the market and that would be a good thing because the market keeps going down a couple more months, so that illustrates momentum strategy would actually get those big events right, because it takes time for a big event to play itself out, and that would benefit this kind of strategy.

On the other hand momentum strategy does suffer from a problem that academics call momentum crash. That is in the aftermath of a financial crisis. For example, in the later part of 2009, prices started to revert and particularly the badly beaten stocks reverted the most strongly. And that's when a momentum strategy would get hammered. So one should not think that momentum strategy is always in good shape because they like crises. No, there are periods in the aftermath of a crisis when the markets start to recover, start to revert, start to move into a new, hopeful regime, that's when the momentum strategy gets completely wiped out, and I know quite a few hedge funds -- particularly futures trading shops -- that get completely wiped out after the crisis. During the crisis they were the heroes. They made so much money that people thought they were the gods of money. But right after the crisis when everybody else is recovering, they are completely wiped out.

Adam: Another issue for momentum strategies is when to take profits. In other words, what methods do you use to determine how long to ride that wave?

Ernie: For momentum strategies typically the only profit-taking mechanism is something like the trading stop. So you keep moving your benchmark, let's say, higher as the price goes higher. And once it starts to reverse from the high water mark since you got into the position, that then is a signal for a trading stop and in effect you are hitting a profit target. But it's expressed in the form of a trading stop.

"The details are what drive the difference between a so-so strategy or a strategy on its way to oblivion and a strategy that can still generate returns."

Adam: So you keep a moving stop and you can base it on whatever you like, technical formations or fundamental-based levels or what have you.

Ernie: That's right. There are multiple ways of defining what a trading stop is and they all would, hopefully with some back-testing, tell you which one is optimal for your particular strategy.

Adam: You mentioned you had started research in momentum strategies that was starting to bear fruit. Can you describe this a bit more?

Ernie: Sure. Some of my earlier research was already discussed in my second book, half of which is really about momentum strategies. Basically I extended a lot of that research in different directions. You know, event-driven momentum, momentum due to roll returns, trying to capture the roll returns in the futures, trying to capture a cross-section of momentum of stocks, and so forth. I might have had a few new ideas after the book was published, but a lot of the ideas were to extend and refine the strategies I've created already and published.

Adam: What got you interested in writing? Also, many people don't necessarily want to share their best ideas but when you're writing a book you are doing just that, and you don't mind.

Ernie: Writing the book wasn't actually my idea in the beginning. Wiley, the publisher, approached me and asked me to write a book because they read my blog and they liked it. So maybe the question can be re-framed as why I wrote my blog in the first place. First of all, a lot of the strategies that people think are so unique and so amazing -- even very profitable strategies, where they think their brainchild is something that has to be kept really secret -- are in fact well known out there. Despite the fact that it (a strategy) is well known, it's still profitable because there is a natural inefficiency in the market. You know, perhaps the other side you're trading on, the long-term investors, when they want to buy the stock they will buy the stock whether it costs them one cent or 10 cents per trade. They've just got to buy it because they're holding it for years. They're not shy about paying a bit of a transaction cost. So just because you talk about it doesn't mean that opportunity is going to disappear overnight. And the other point is that I of course do not trade the strategies exactly as described in my blog and in my book. I merely describe a broad outline and the proprietary tidbits are really in the details. The details are what drive the difference between a so-so strategy or a strategy on its way to oblivion and a strategy that can still generate returns. My trade secrets are the modifications, the refinements, the improvements and not really the basic skeleton of the strategy. So I'm absolutely not afraid to discuss them because it's well known. And you know, it may not be well known to every reader but certainly well known to professional traders. Now also, by disclosing and discussing the basic skeleton I happen to receive a lot of feedback from professional traders. And professional doesn't necessarily mean institutional, but also very serious researchers. They continue to give me new ideas, they continue to challenge my thinking, challenge my strategy. They give me their ideas. So I find that as a person who doesn't sit in a big bank, I continue to benefit from this kind of information, in fact better than if I were to continue to sit in a big fund or a big bank.

That's what scientists do in all fields of scientific research too. It's not just in finance. Scientists, before they publish a paper, have some idea and they keep it secret. Then they publish it, but by the time they publish it they're on to their next idea. But by publishing it they get lots of feedback and generate more ideas than they had in the beginning. I was trained as a scientist, and I view the whole process of quantitative trading as extremely close to the scientific research procedure. And that's what scientists do and it has been proven fruitful for the past few hundred years.

Q-Q plot of the 5-minute returns of USD.MXN

Q-Q plot of the 5-minute returns of USD.MXN

Q-Q plot is a type of chart used by statisticians to test whether the distribution of a quantity is normal (Gaussian) or not. Here we tested the 5-minute returns of the exchange rate US dollars vs Mexican Peso (USD.MXN). If these 5-minute returns were normally distributed, they would fall on a straight line (indicated by the red dotted line). Instead, we see severe deviation from a straight line, indicating that the returns of USD.MXN have very "fat" tails, which are very risky to traders.

Adam: You've also spoken about how comparing quantitative analysis to the physical world only works to a certain degree, saying that you can't treat markets in the same way as physics where there are laws that are immutable because market 'laws' are always affected by the people who are participating, so there are no iron-clad laws.

Ernie: Yes, I mean certainly one cannot bring 100% of what one learns from scientific research to trading. First of all, like you said, the market is not stationary. It does not repeat exactly the same patterns as it has before. So, unlike scientific research, you have to also be aware of the external factors driving changes in the system. For example, what the Fed is doing, what the regulators are doing, how the market structure has changed because of the high frequency traders. So yes, it's not a closed system, it is a system that is constantly being effected by external forces. You can still adopt a scientific attitude as long as you take into account those external forces and modify your model accordingly. But the scientific spirit still holds. That is to say, one has to form hypotheses of the market, and then build models to test whether those hypotheses are correct. That is the attitude that still works.

Adam: You did work in pattern recognition for IBM before working in the financial industry, including stints with some big names. What attracted you to markets from science?

Ernie: Actually, to put it in context that's what most physicists do, at least among my classmates from where I went to school. I would say that a rough guess is that half of my classmates in theoretical physics are in finance right now. It's totally in line with my peers, what I've chosen. But more specifically it was a lifestyle choice. I was working quite happily in IBM research in a suburb of New York at that time. But I'm more of a city person. I don't enjoy living in the suburbs, at least I didn't at that time. I wanted to live in Manhattan. Actually I lived in Manhattan while I worked for IBM -- it was a very unpleasant commute. I decided I couldn't take that anymore and I decided to find a job in Manhattan and practically the only the only jobs available to physicists in Manhattan were in finance, so I joined Morgan Stanley as a result. So it's quite by accident that I got involved in finance. But once you get into something your previous training comes into play and you adapt your training to your new profession just as well.

Adam: You hear about people who do rocket science or come from other scientific backgrounds and then get involved in markets, but I didn't realise the numbers were that high in terms of the percentage. Presumably the former classmates you know are all pretty successful?

Ernie: Yes, I think the majority of them have been employed by the largest investment banks and hedge funds and so forth. I frankly couldn't think of one example where they failed. You know, they succeed to varying degrees. Some are managing directors at Citibank, Deutsche Bank and so forth. Others are portfolio managers at UBS, Renaissance Technologies, and other places. Certainly they've had varying degrees of success but none that I can think of that you can call a failure.


Adam: Another big career change was not just moving out of science and into markets, but also moving from a big firm and setting out on your own. Was that a big shift for you?

Ernie: Yes, it's a major shift. I was never profitable working for large institutions because of various factors that I discussed in my first book. One main factor was that, in a big bank, one of the main tasks you have to do is develop something that pleases your boss, not necessarily something that makes money. Unless you are already head of a group, then you can decide whatever strategy to trade. Your main task is to make sure your strategy gets traded at all, and that depends on pleasing your immediate manager. And that process has been, I find, extremely detrimental to profits. So I only became profitable after I could basically direct my own investment strategy.

The first year where I become independent I had a down year, but after that, I never had a down year if you count all the investments. But before I became independent there was never an up year so that's a major change. That's why I disliked working in big institution. The goal is not clear enough and the structure is not simple enough to really do what I think is right. Plus, I find that it's incredibly distracting to work on a trading desk. There are many trading groups who are not aware of what is the ideal environment for a quantitative trader. The ideal environment for a quantitative trader is to put them in a quiet corner with an office, maybe a decent cubicle, so they can do their research quietly. Most places think that a quantitative trader is just another trader and they put them in the middle of noisy trading floor. I find that I simply cannot think, let alone think creatively, in the midst of hustle and bustle. So I find it very difficult to make any sort of contribution in a trading floor environment.

Adam: You have a partner at QTS Capital Management who handles all the technology. Technology is so much more a dominant factor in markets than it was a decade ago. Are there any developments that have made a significant difference for your fund's performance in, say, the post-crisis era?

Ernie: I would say that the cost of co-location has greatly decreased. I think that's the main benefit. In terms of software, we are already using all the available software and hardware that anybody else can use. We are not high frequency traders. We don't need graphical processing units numbering in the thousands to run our strategy. But we do benefit from the low costs of co-location.

Adam: Any specific markets you tend to like trading more than others?

Ernie: I like trading futures most of all. Futures are better than the equities market right now because the liquidity is much better. In the equities market right now, if you look at the top of book, liquidity has been practically drained due to high frequency trading. In terms of the advantage over forex, forex is in a kind of a turmoil as evidenced by FX Concepts. The forex market right now, you might say is being manipulated by central banks. So it's become more difficult to find patterns. But futures offers us the most opportunity and liquidity at the moment.

Adam: Final question. Any predictions for the year ahead?

Ernie: I don't expect much difference in the beginning of 2014. They (central banks) may be starting to withdraw liquidity later on in 2014, but certainly not in the beginning.