The Gateway to Algorithmic and Automated Trading

Lab in a box

Published in Automated Trader Magazine Issue 17 Q2 2010

In London, in late April 2010, in the bleak but soon-to-be-Olympic surroundings of the Docklands, an experiment took place that could significantly alter our understanding of how humans and machines interact in a trading environment. Six veteran traders and two scientists met in a small room above a trade show to re-enact IBM’s 2001 contest between human traders and adaptive trading technologies. But that was not all. The scientists were also planning to test something else on the human traders involved. Something new... John Howard, Automated Trader’s CEO, was one of those traders. Here, he tells his story.

If you cannot view this video then your Flash Plugin is out of date. Please click here to upgrade.

We were at Trade Tech in London. The usual mix of panels and presentations, and a lively exhibition hall full of familiar faces networking. The Automated Trader team was doing what it usually does on these occasions, along with a certain amount of work, and we were also getting to grips with the new challenge of putting together magazine content to be barcoded for access via mobile devices.

Then we were approached by Professor Dave Cliff of Bristol University's Department of Computer Science. Professor Cliff explained that he had an experiment in mind, for which he needed volunteers. The experiment would be painless, and good fun, and he was promising anonymity to all participants.

It was the immediate judgement of the entire Automated Trader team that I would be the ideal volunteer for Professor Cliff's experiment. And that was even before Professor Cliff had explained that the experiment would be an exercise pitching human traders against algorithms developed at Bristol and at Southampton Universities, and that it was designed to answer a number of questions. Somewhere near the top of that list of questions needing answers, was whether the human traders, with the benefit of experience and intuition, had any chance of beating the much faster robot - but also, Professor Cliff and his colleague Marco De Luca, also of Bristol University, were keen to establish just how well their robots could learn from the trading behaviour of their human guinea pigs.

As I was led away, the relevant part of my life flashed before my eyes. I remembered my decision, some fifteen years ago, to invest my intellectual capital in the development of systematic trading models. I had come to the painful conclusion that - while I often did have well-founded trading ideas, backed up by a reasonable 'feel' for the market - my fundamental character flaw [This should be interesting - Ed.] of impatience fuelled by a need for near instant vindication of my ideas rendered me a truly dreadful discretionary trader [He's just being modest - Managing Ed.]. A systematic approach allowed me to leverage the strengths of my trading concepts without my own character flaws [Do you think we should tone this up a bit? - Ed.] spoiling the party. So, here I was again fifteen years on, about to be reminded of my discretionary trading deficiencies [No. It's an effective build-up - Chief Sub-Ed.].

The trading environment that had been created was described by Professor Cliff as a "lab-in-a-box", and consisted of six low-cost netbooks as trading terminals, a mid-range laptop taking the leading role of matching engine, a few cables and not much more. There were two stages to the experiment. In the morning, a "Control Group" session had already established a set of benchmarks, and this was now being followed by an afternoon session using a different set of "real" traders - us - from which the ultimate conclusions would be drawn. The six 'volunteers' were divided into two groups of 'sales traders', with one group executing short trades and the others taking long positions in response to randomly generated order flow arriving on each trading screen. Responding to the incoming orders, the two groups would thus be trading not only against each other but also against the robot.

Following a short orientation by Marco De Luca, we then had a short familiarisation period before the real session started. The live trading experiment consisted of ten three-minute sessions, each intended to represent a single trading day. As with any market that's been closed overnight, the first moments after the 'opening bell' were where fair value was established and the greatest volatility and thus opportunity and risk existed. It became clear very quickly that profits were more likely to be accumulated through dexterity with the mouse and keyboard, than having any innate 'feel' for the market - miss the opportunities in the initial flurry, and you'd end up finishing the 'day' a few minutes later with little to show for your efforts.

As soon as the screen went live, I took an immediate - And now, for readers at home only, we interrupt John's account with this message from Professor Dave Cliff of the University of Bristol's Department of Computer Science. Because, you see, there were two robots trading that day…

Marco De Luca, University of Bristol

Marco De Luca, University of Bristol

Professor Cliff writes:

"As far as we know, this is the first time that anyone has attempted to repeat or replicate the results that IBM published in 2001 where they showed that human traders were consistently out-performed by autonomous adaptive 'robot' trading algorithms.

"We ran two experiments at TradeTech2010: one using the ZIP algorithm that IBM tested in 2001; and another using the more recent AA algorithm invented in 2008. These results are preliminary - we need to repeat the experiments to generate more data, and then do a full analysis of all the data, before we can say anything scientifically concrete; but on the basis of the data we have so far it is clear that the robot buyers consistently beat the human buyers, which confirms IBM's results.

"On the selling side, the data is more mixed - it seems that the robots encountered a small minority of human traders who were consistently underselling (offering give-away deals with almost zero margin) and the robots moved very fast to undercut the giveaways, but in doing so they allowed one or two human sellers to outperform them in P&L terms. This, to us, is novel and interesting - it seems that in these experiments the robots were exposed to a giveaway style of competitor that they hadn't previously been tested against.

"It gives us more to think about and more to work on, which is great. We're very grateful to WBR (organisers of Trade Tech) for the opportunity they gave us to run these experiments."

And now back to the experiment.

- as I hit the key, delighted with the success of my algorithm-beating strategy, the final "day" ended.

The big challenge for the creators of an experiment of this type is in providing as realistic an environment as possible. That realism applies equally to both market structure and the access to the market. In both respects, the Bristol University team did a good job - the trading terminal was very much like many basic order management systems, and the order book and ebb and flow of bids and offers changed in much the same way as one might expect in many real markets. However, I couldn't help feeling that the experiment would have had greater validity if the human traders were able to use whatever terminal they use on a day to day basis, as - even with the simplicity of the GUI provided - taking a trader out of his comfort zone will always be an impediment to performance.

My other observation is that the human traders were effectively handicapped in two ways. Firstly, the humans were only able to trade in single units, and secondly we were only able to trade long or short depending on which group we had been placed in - not both. (Professor Cliff explained that this limitation was introduced for consistency with previous experiments conducted by IBM in 2001, and will be relaxed in future). As a result, there was little opportunity for creativity in the way a trader sought to achieve his ultimate objectives from each individual trading session. Further to this point, the addition of a group of pure proprietary traders, able to capitalise on a rising or falling market would add to the realism of the artificial market structure, as well as provide potentially interesting results pertaining to the behaviour of traders with an entirely different set of motivations.

Finally, there's a lot of hearsay and several studies that suggest that women are under-represented in trading rooms [Uh oh - Ed.] primarily because the aggressive testosterone fuelled environment of the trading floor is not a particularly welcoming environment [WHAT is he saying? - Managing Ed]; and in actuality, freed of the shackles of the aggression and ego of their male counterparts, [He's due back soon, isn't he? - Managing Ed.], with cool objectivity and an ability to multi-task that most men can't hope to match, women make highly effective traders [I'll be out for the rest of the day - Ed.]. It would be interesting to discover whether Cliff's and De Luca's "Lab-in-a-Box" might end the conjecture and prove or disprove this theory. How different our trading rooms might be [And tomorrow - Ed.], if this theory turned out to be true.