The Gateway to Algorithmic and Automated Trading

Hired Guns

Published in Automated Trader Magazine Issue 12 Q1 2009

Hedging your talent risk could mean employing games enthusiasts with recent weapons training, says Helen Sanders

The technology world would seem to have come a long way since the first computer, as we would understand the term, was first designed. In 1943, Howard Aiken and IBM created the first universal calculator to calculate the data he needed to develop his theory of space-charge conduction in vacuum tubes. Harvard Mark I, as it was known, was 16m long and 2.4m high. It had over 765,000 parts, 3,300 relays, 175,000 connections and over 500 miles (800 km) of wire. At the time, the machine was unbelievably fast: it could do 3 calculations per second.

Hired Guns

Your first reaction to this may be slightly amusement and possible smugness. After all, look where we are now, surely this is ancient history.
But looking at today's trading technology, the challenges are actually very similar. How do we find ways of processing more data, more quickly? The volume of data continues to proliferate, servers are getting bigger, but not by enough to satisfy the inexhaustible craving for crunching all these figures. Every trader creates his or her competitive advantage by being microseconds faster than the competition. It is not only the technology which is struggling to keep up. Perhaps more significantly, it is a scarcity of skills with which to leverage the technology which could mean that traders will lose competitive advantage.
Barry Thompson, CEO and Founder of Tervela, says: "There has been an explosion in algorithmic trading requirements since the start of the economic downturn, as traders seek to change their strategy. For example, hedging strategies which may have been appropriate a few months ago are probably now too expensive and erode profitability." Taking the point a step further, Dr Paul Tolman, Founder, Beta Gamma Research, adds: "A successful trading tool requires a variety of different skill sets - IT, quant skills and trading experience. To be successful, you need to bring these together, and it can be complicated to get the combination to work. There is a skills shortage in some areas. The larger investment banks can attract the right staff, but tier two banks tend to struggle more, as they can't necessarily justify the cost of a full time quant, for example."

Barry Thompson

Barry Thompson

A skills shortage? Thompson says: "There is significant demand for algorithmic skills, both for financial modelling and implementing trading strategies. While there is a dearth of quant analysis skills, there is a greater dearth on the technical side." But surely, there are thousands of graduates coming out of college with well-sharpened programming pencils, so why should there be a shortage?

Key to the problem is that automated trading requires highly efficient programming to deal with massive volumes of records. Any latency means that both time and competitive advantage tick away. Simply adding new servers is not the answer. Simon Garland, Chief Strategist at Kx predicts: "Servers will continue to get bigger, and multiple core processing will evolve further, so it is imperative to be able to programme in a multi-threaded environment.

Banks and trading firms will not just be able to keep buying new servers, however, systems need to be faster and use server capacity optimally."

Dr Paul Tolman

Dr Paul Tolman

First, adding new servers is expensive. Secondly, many firms have stipulated that no new servers can be put in, unless others are taken out, mindful that energy
costs have soared and power availability is finite. In London and New York, for example, the problem of power is now acute. In areas such as Canary Wharf, the risk of electricity "brown outs" has become critical, which will be exacerbated as London's demand for power increases ahead of the Olympic Games in 2012.
A key point which Simon Garland makes here is the issue of multi-threaded processing. On a single processor, multithreading involves the processor switching between different threads, or tasks, storing the result of one process before moving on to the next, and then switching back again. On a multiprocessor or multi-core system, threads or tasks run concurrently, making processes more efficient and increasing the sophistication and speed with which calculations can be performed. True multi-threading, using multiple processors, brings huge benefits to automated trading, but only if banks, and the software applications they are using, take advantage of them.

As Thompson emphasises: "While programmers are plentiful, the individuals who can optimise a kernel [component for managing system resources, among other functions], for example, are few and far between." Having found the right skills, the problem then is keeping them. As Tolman puts it: "Keeping hold of intellectual property is a real challenge - once someone has come up with a good strategy, it could be attractive for them to go to another bank or set up a hedge fund. This is particularly an issue for some of the smaller banks." So if the technical skills required to support automated trading are not simply about developing functionality and calculations, but also about minimising latency and maximising the speed of processing, surely recent graduates should have these skills? One of the issues, as Thompson continues, is that: "College students graduating over the past 8 to 10 years have focused on the higher level programming languages: Java, C# or C++. There are only a small group of people who still count clock cycles and are able to look at the most efficient way of developing particular functionality, rather than simply the most efficient way of developing it in Java."

Simon Garland

Simon Garland

There would seem to be two sets of people best equipped to deal with this challenge, who at first sight might seem somewhat incongruous. As Garland illustrates: "Ultimately what is important is the ability to make programmes run extremely fast. A group of people who have experience of doing this are those who were programming in the 1970s and 1980s who appreciate the importance of highly efficient code. Recent graduates are often somewhat cavalier about CPU usage. However, if you're querying a billion records, and retrieving 100,000 records, the process will undoubtedly be slow if the programme is written badly and multiple hits are made on the data, slower retrieval has a real cost to the trader."

Secondly, perhaps rather than creating a vicious cycle of continually poaching staff from proven development teams, there is a case for looking beyond the traditional skills pot. Garland continues: "A company we know has found it difficult to find the right skills within the industry and have looked outside it to the gaming industry. There are some definite advantages to taking on people with this experience. They are accustomed to high speed graphics and can optimise programmes to run extremely fast on relatively slow machines. If you can take these skills and apply them to the stock exchange, the result could be very exciting indeed."


The idea may not seem as unlikely as it first appear. For example, the skills requirement for a recently advertised job for a racing game programmer read as follows:
• Strong math skills, including trigonometry, calculus and linear algebra

• Experience with vehicle dynamics is preferred but by no means essential.

• You will have been responsible for the physics implementation in at least one completed title

• Strong C++ programming skills.

• Experience of racing games is preferred but is not essential

• Strong written/verbal communication skills.

• Strong time management and organization skills.

• A passion for games and programming.

• A logical thinker with strong problem solving skills.

Of these, knowledge of vehicle dynamics and racing games may not be critical to an automated trading tool programmer (although i imagine that most programmers are hardly unfamiliar with whatever the Wii or PS3 can throw at them) but most of the other skills have very specific applicability to a trading environment. There are other possible sources of new recruits too. As Vivake Gupta, Lab49 explains: "We have had considerable success in finding people not sullied by the financial services domain who come from other environments. Those working in financial services for a long time get used to things, and forget the inefficiencies that they might have recognised to begin with. You need someone from outside to have a clear view of where efficiencies can be introduced."

One possible solution is to recruit specialists in military technology. Someone working in missile defence probably understands the imperative for rapid data analysis and decision-making far better even than most algorithmic traders. Similarly, the energy and aerospace sectors have the potential to provide the talent and skills required for trading. Gupta outlines other important skills: "Experts in data visualisation can also be important to the trading environment. In the past, there was little focus on the user interface. Someone experienced with visual design can look at how traders use the system and modify the user interface to make traders' role easier. This can make a huge difference, not just in the way that data is presented, but how it is used and what the sequence of tasks is. While this does not increase the sophistication of system, it can dramatically improve productivity and base level trading ability."

Not everyone is convinced by the potential for sourcing skills from outside the industry, however. Tolman warns: "While there are certainly useful skills that can be sourced from further afield, the problem with people from other industries is the lack of practical trading knowledge. Teams need to be driven by the needs of the trading desk, irrespective of where you source other skills.

Vivake Gupta

Vivake Gupta

Otherwise, it is easy to end up with something that either doesn't work or is vulnerable to changes in market conditions." A related problem is the length of time it takes for a recruit new to the financial services industry to learn the trading environment and become familiar with the vocabulary. It can take months, or even years for a new recruit to reach their maximum potential, which is a cost few organisations can support.

There are other issues, relating not to the background of individuals in the trading solutions team, but the ability for banks to dedicate the right investment to projects of this type at all. Tolman explains: "In banks in particular, projects can run for years with different IT and quant teams working together. The result can end up being a huge monolithic system which is very difficult to change and does not satisfy traders' needs. Conversely, what traders need is a solution which is quickly adaptable to market conditions. For example, recent increased volatility and marked movements in FX rates have caught some people out."

A question which management are asking increasingly is whether there is a need to maintain these skills in-house at all, or whether third parties could provide the necessary skills or technology instead at a lower cost. The idea should not be an anathema: in the 1980s, banks developed their own network routers, today, such a project would seem absurd.
With skills at a premium and easily poached, a potentially long learning curve for new recruits, and a far more rigorous approach to the cost benefit of IT projects, now would seem to be the time to look outside the firm to the tools which are available from elsewhere. Many people I have interviewed have been pleasantly surprised at what they have found.

The investment by specialist vendors in third party trading applications, specialist programming tools and consultancies who can optimise these tools vastly outweigh the investment that any individual bank would be in a position to make. SunGard's recent multi-million dollar acquisition of GLTrade, for example, illustrates the growing strategic importance of automated and algorithmic trading tools in the industry.

As Thompson summarises: "In the past, people used to build their own trading software, which was specific to a particular technical environment. Today, there is a movement towards pre-packaged software and away from in-house development and we expect to see a continuing trend in this direction."

The typical objections, that these solutions are not specific enough to the organisation, that competitive advantage can only be created by having in-house skills or that third party software inhibits the variety of trading strategies that can be adopted have long been disproved. Rather, high performance, the ability to integrate market data seamlessly, the flexibility to implement trading strategies quickly and greater accountability are all characteristics not of in-house systems, but of modern, packaged trading applications. Gupta emphasises: "Most people are focused on their legacy system and it becomes impossible to innovate. The fact that the financial services industry is not an early adopter of new technology shows that there is a skills shortage. Banks need to decide where to focus and what their core competencies should be."

Not every trading firm is in a position to take immediate advantage of third party systems, in which case looking for new skills from beyond the normal channels could be an important way of reinvigorating stale solutions. However, with budgets frozen and every bank seeking to realign their strategies with their core competencies, those who gain competitive advantage will be the ones who can redirect their investment into financial modelling and use their IT resources to implement and optimise, rather than develop, the tools to turn these strategies into reality.