The Gateway to Algorithmic and Automated Trading

Slow down and smell the ash cloud

Published in Automated Trader Magazine Issue 20 Q1 2011

Last year’s ‘signature event’ was not the flash crash but the volcano eruption. This year, we’ve had apparently internet-led regime change in the Middle East that has destabilised the entire region, while a single ‘whistle-blower’ seems to have dumped most of the US government’s secret correspondence in the Wikileaks inbox. Oh, and the floods that devastated an Australian state have been followed by a cyclone. Is the world trying to tell us something? William Essex wonders whether ever-lower latency and ever-greater capacity are really what we need right now.

Do we need to stop and think? These are dangerously interesting times for auto/algo traders. Latency has been reduced almost to vanishing point, and a modern restatement of Parkinson's Law might reasonably suggest that 'Data expands to clog up all the available bandwidth - and then some.' There are still warm-bodied traders around (although doesn't that strike you as odd?), but automation has become embedded in the trading experience.

And yet - United Airlines. The 'flash crash'. If the 'data question' has ceased to be 'How can we know everything?' and become instead 'How can we isolate what we need to know?' - shouldn't the 'latency question' switch from 'Can we go even faster?' to 'What is an appropriate [safe?] speed for our strategy?' And what if - just imagine - human traders turned out to have qualities that machines just don't have? Can't we find a way to plug them in somehow?

Like Brian Tehako in this issue's 'Me and My Machine' (page 26), we all make use of human-generated ideas. We take those ideas and we make them into trading models. Then we trade them, and while we do so, we insist firmly that there is no discretion, no curve-fitting, no subjectivity. Sooner or later - and if you haven't read this month's cover story yet (page 18), now would be a good time - we have to contend with the gradual failure of our ideas - or rather, our models - to perform. Like a warm-bodied trader in the early stages of dementia or promotion to management, the familiar edge just doesn't seem to be there any more.

Intelligent machines?

But the curious glitch in the onward march of technology remains the same: whatever the machines can do, the human traders can do it - not necessarily better, but well enough to stay in business. As Paul Bowes, Head of Exchange-Traded Instruments, EMEA at Thomson Reuters, says: "The prediction of the end of the trader has been made for as long as I've been working [twenty years]. The number of traders has increased multiples of times, although there's no question that a greater degree of trading is done by machines." True - and not only that: they're everywhere. Bowes continues: "Even in equities, which is an incredibly automated sector, there are still people involved. Very sophisticated hedge funds using state-of-the-art technology; they are still people who do business with people."

Similarly, Kevin Covington, Deputy CEO and Managing Director for ITRS Europe, says: "You still have to have people adjusting the rules, because all the red lines are constantly moving, and you have to have people interpreting what's going on. You need intelligent people working with intelligent machines." But this is the point made above: every machine can be traced back to an idea that first appeared in a light bulb above a human head (even if, these days, the causal relationship is not invariably direct). We may get to machine-begets-machine, but let's look the other way. We are not quite at Rilke's dystopic vision of "self-regulating machines oiling themselves in silent factories" ('The Sonnets to Orpheus', 1922; my rough translation), and in any case, the threat is coming from the other direction.

We all think - don't we? - that more data plus faster delivery/reaction/execution gives us an edge. I'm not necessarily talking about HFT here; if a "slow-trading movement" grew up, on the analogy of, say, the slow-food movement, there would still be a case for not being last in line at the execution venue. You go to a slow-food restaurant, where everything was grown on the premises, prepared with love, etc., etc., burgers absolutely not 'flipped', and brought to your table by the heir(ess) to a long family tradition of waiting at table rather than by a disgruntled ex-student earning money to support an online gaming habit - and even there, in those relaxed surroundings, music playing, a fountain in the courtyard, you'll be more likely to complain if everything's brought too slowly than if it arrives too fast.

Kevin Covington

Kevin Covington

Smart machines?

Let's rewind the history and watch it speed up as we play it back towards the present. Look out for the logical flaw.

In the beginning, people shouted at each other. Some of them wore brightly coloured jackets and some wore bowler hats, but generally, they voice-traded at the speed of (a) a loud voice plus sign language, or (b) a sequence of Bakelite telephone connections. Settlement was T + the postal service. Somehow, they made money. Things fell apart every time the shoeshine boys got into the stock-tipping game, but - yeah. Wealth was generated.

Then trading became automated … systematic … algorithmic … and speed won. But the more interesting part was what happened next. We all got smart. Smart order-routing, pattern recognition, black boxes, very clever people with very clever ideas that programmed themselves into very clever trading models. Automated Trader burst onto the scene. Oh, and MiFID, dark pools, et cetera. The raw material - data - came faster too, and there was more of it. And then there was that whole industry that grew up around news. That United Airlines news release - remember? Scoring.

We were fast and we were smart and we were consuming ever-increasing volumes of data in the belief that either (a) the further we roamed in the data universe, the more edge-generating ideas we'd find, or (b) there's always one more tiny piece of data that will conclusively (in)validate our model, or (c) [Insert your own reason here. The argument can take it.]. The logical flaw - hardly a punchline, but we should acknowledge it - was that smart'n'fast didn't match. If speed was enough, why did we need to be smart? If smart was enough, why did we need to be fast? Yes, yes, I know, but the real question is: we've done fast; we've done smart; where do we go from here? What is the still-missing ingredient in the recipe for a competitive edge?

[By the way, wouldn't it be great if I answered that question? You flip to the last paragraph and there is is: the secret of staying ahead. But then you realise … you're not the only one reading it.]

So the point is, there's not a lot more we can do to remain competitive, if we decide to stay on this same old road towards knowing everything as it happens and reacting to it before it's finished happening. We are living in a world in which latency just goes on shrinking and the volume of available data just goes on expanding. That world has evolved around us, and perhaps we began to get caught up in it without realising what was happening - investing in more storage, smarter/faster execution, without considering the end-point (vanishing point?) of such a strategy. We can go on like this - buying ever-more storage and maybe just 'exceptions-processing' the data - but here's the funny part.

We've turned into a herd. We're all individualists working our own clever models but we're all chasing in the same direction. A very wise asset manager once concluded a short presentation (on a particularly quirky notion that amounted to a one-way bet) by saying: "I'm not saying this is a good idea, but I do notice that everybody else is betting the other way."

And think about this. There actually is a slow-trading movement. Simon Garland, Chief Strategist at Kx Systems, says: "Some people are still aiming to be the fastest and the best, but we have had a few people recently who have said, 'I can't keep up', or 'I can't be bothered', or 'It's costing me too much'. Just as the nuclear arms race bankrupted the Soviet Union, so the IT arms race is proving - well, expensive. And futile? Garland says: "When everybody's doing it, there's no gain in it. If you do find some clever trick, your closest competitors have got no choice but just to buy whatever you bought. You have the gain for three months, say, then you're all back on level pegging again." Yes, I suppose that is pretty much a description of the game we're in - if it works, buy it - but we're cleverer than that, surely?

Simon Garland

Simon Garland

Inconvenient facts!

There are - and this is healthy - exceptions/objections to just about every sentence in this feature. We're not a herd. Not everybody wants to get into HFT. Many traders get by without chasing speed and/or omniscience. Fine. I'm not saying that anybody's wrong, and nor am I really saying that I'm uneasy to find that we're all - how to put this? - correct in the same direction. I think we're all individualists betting our own talents in the pursuit of alpha, and if the impact of technology on that pursuit has made some of us behave like, er, a herd of individualists - well, there are enough predators in the jungles of this industry to restore a natural balance.

To take a holistic - as opposed to simply macro - view of markets, it is necessary to appreciate their full complexity. This requires some acknowledgement of the - to me, at least, in this job - inconvenient facts. There are still human traders making a decent living in largely automated markets. So there might be 'human strategies' that work in an automated environment. Stuff happens. Remember that volcano? Fantastically well-constructed models take on complex markets and win. But the creators of those fantastically well-constructed models, when they go to the beach, don't turn off their mobile phones. Why not?

The kicker, for me, is the volcano. Let's not be too Rumsfeldian about this, but last May's flash crash was a 'known unknown', in the sense that it happened in the market and the market made it and we could at least hope to attribute it to detectable market causes. You could program a machine to avoid the next one (and there's an interesting passage in Q4 2010's First Person interview with Bernard Orenstein on learning from Lehman - see also Bob Giffords on the SEC/CFTC flash-crash post-mortem at

BUT no machine could have glanced out of the window of its office in Iceland, seen those wisps of smoke on the horizon getting thicker, and shorted European airline stocks. To use the analogy that always crops up in paragraphs like this - you can't design a model to catch the moment when that tiresome butterfly starts flapping its wings on the other side of the world. You have to wait until the resulting earthquake shows through in the market.

burning pc

People-powered machines?

Referring to the volcano in the course of a long and fascinating conversation about data, latency and other matters currently arising, David Csiki, managing director of INDATA, laughs and says: "That's life, unfortunately. Life doesn't fit in the system sometimes, and that's where we see a large focus on disaster recovery, trying to plan for scenarios that you didn't envisage happening." Csiki also makes an interesting point about the current data explosion: enabled by technology, it might be, at least in part, generating itself. Csiki says: "In the past, you would have a block trade and you would have a limited number of executions. Now, what we're seeing is, for any given block trade you have thousands of executions." We're in a 'data spiral'. But the real 'lesson' of the volcano, to the extent that there is one, is not just that stuff happens, but that data doesn't equal a defence against stuff happening, and more data doesn't amount to a stronger defence.

As Csiki says, life doesn't fit into the system. Life doesn't fit. As this feature is being written, Egyptians are shooting at each other on the streets of Cairo, and Cyclone Yasi is passing over Queensland. Here in the UK, there's warm sunshine radiating through the office window but snow on the weather forecast. Unease about the weather is such that, wherever you are in the world, you are never more than a few kilometres from a conference on climate change. Osman Latif ('First Person', page 6) mentions seeing Wall Street when he was younger. How many of us saw The Terminator at an impressionable age? That's the one where the machines take over the trading systems - sorry, no, the weapons systems - and try to wipe everybody out because they see humans as the primary threat to trading success - no, sorry, peace (but the humans win in the end).

Given that they still exist in our markets (and, let's say, represent an exploitable resource), could we consider plugging the human beings into the system (in a way that doesn't introduce latency and spilt coffee into the network)? Could we build models where a rigorous exclusion of discretion wasn't necessary? Discussing heat maps and intelligent filtering - among other approaches to data overload - Kx's Simon Garland comes to a sobering conclusion. Garland says: "There's no substitute for smart people. You're better off having people who have a feel for when something strange is going on." Now he tells us. But this isn't a case of an IT company packing up and going home. Garland continues: "Years ago, working with a big bank, we decided we weren't clever enough to have algorithms working out where we should be looking for strange things going on. So what we monitored instead was what the traders were doing that was different from what they were usually doing."

Get that. They used the people to spot the anomalies. Cool, or what?