The Gateway to Algorithmic and Automated Trading

Rapid Model Deployment (RMD): The new race for speed in systematic finance

Published in Automated Trader Magazine Issue 11 Q4 2008

These days, speed is of the essence in more than just order routing. Phil Perkins, Ben Van Vliet and Carl Ververs of the Institute for Market Technology explain how a rigorous process for rapid trading model development is becoming an absolute necessity.

It's been nearly one hundred years since Henry Ford pioneered the assembly line. Ford recognised that the auto industry had reached a new phase, one that would require more than just better technology. While his competitors were focusing on faster, sexier cars, Ford focused on faster, more efficient production and delivery, and he drove many of his competitors out of business. He proved that bringing together people, process, and technology could make the whole much greater than the sum of the parts.

Rapid Model Deployment
Figure 1: RMD Framework
Figure 1: RMD Framework

Fast Forward to Today

Many automated traders we've talked to report that the life spans of profitable trading systems have shrunk from years to months to even weeks. It appears that buying faster servers, contracting for co-location and PhD-level mathematics are no longer in and of themselves enough to sustain competitive advantage. There is increasing pressure to develop more new models on tighter and tighter timeframes; a firm needs an assembly line of models just to survive. The goal is a continuously evolving portfolio of trading systems. So, for firms that want to stay competitive:

1. Models must be proven and deployed faster.

When it comes to capturing competitive advantage, the ability to rapidly test and implement trading, pricing, and risk models is becoming as important as better analytics and execution speed.

2. Models must cost less to research and deploy.

To maintain profit margins, firms must identify opportunities to decrease the resources and time needed to research, back test and programme models, while at the same time increasing quality.

As the systematic finance industry matures, the firms that bring together people, process, and technology will achieve these two goals, create competitive edge, and like Ford, survive at the expense of their competitors. Of course, these three components-people, process, and technology-already exist in every financial organisation; the question is whether they work with or against each other. We have been researching business processes that will ensure they work together-research we call RMD.

Rapid Model Deployment (RMD)

The Institute for Market Technology (I4MT) is working to create an open standards reference architecture directed at faster deployment of better trading, pricing and risk management models. This initiative builds upon existing open source projects, such as QuickFix, to provide a foundation for new platforms for systematic finance. RMD is a framework that will help firms create their own assembly line of trading systems, just like Henry Ford. The RMD recommendations form the foundation for a firm's people, its processes and its technology.

RMD Recommendation 1: Align people and processes

Processes define how people interact. However, if formed ad-hoc, then informal networks and relationships will undermine organised work processes, leading to failed projects. The reference methodology, K|V, provides for cross-functional-team-based development and evaluation of trading, pricing or risk management systems through collaboration and process adaptability. K|V breaks down efforts into four stages for rapid delivery of high quality models:

Design and prototype the trading model and communicate it to stakeholders by outlining the business idea, competitive advantage, potential performance, technical feasibility, scalability, and risks to the project and model.

Back test the model, vetting it against historical market data to ensure the strategy meets investor specifications and to understand the process capability over different performance metrics.

Implement, perform quality assurance, program the model and deploy the required technology. Start trading the model at a low scale to verify performance.

Manage the model and associated risks within specified control limits as it makes live trading decisions. Plans for continuous model improvement should be fed into a redesign phase.

Figure 2: K|V Methodology
Figure 2: K|V Methodology (click for a larger view)

Each of the four K|V stages above is further broken down into plan-benchmark-do-check (PBDC) steps.

Plan - Benchmark - Check - Do

Plan. What do we need to do?

Benchmark. What's the best way to do it?

Do. Do the work.

Check. How did we do?

The PBDC spirals signify iteration. Iteration allows development teams to adapt to changing conditions and new knowledge. By focusing on small iterations, projects can achieve small increments of success with minimal planning, as opposed to lengthy planning that may lead nowhere. Iteration minimises the risk that bad models will proceed, while speeding development of worthwhile strategies.

At the completion of each K|V stage is a gate, a management meeting with development team members where projects are evaluated based on objective criteria against original performance expectations. Gate meetings must result in one of four decisions:

1.) Go. Go on to the next stage of development.

2.) Kill. Kill the project entirely.

3.) Hold. Hold development at the current stage for reconsideration at a future date.

4.) Return. Return to a previous stage for additional research or testing.

At gate meetings, bad models and bad projects should be weeded out, so that resources can be reallocated to more promising ones. For projects allowed to continue, management should define the deliverables expected and the criteria for evaluation at the next gate meeting. Gates predefine incremental releases of capital. A mandatory, open discussion allows management to control incremental investment in new ideas, rather than making a single large investment up front. At each successive gate, management makes a progressively stronger commitment to the project. In the end, gates allow losing projects to expire worthless, and allow worthwhile projects to proceed. This management involvement creates distinct ownership around the control decisions that lead to investment in projects, and determines proper conditions for success (e.g. scalability, proper use of leverage, timing, etc.).

Notice the hallmarks of Ford's assembly line in this methodology: aligned resources, time boxed production cycles, and critical decisions made with management involvement.

Figure 4: Groundswell Reference Platform (click for a larger view)

RMD Recommendation 2: Use standardised, interchangeable parts

With the complexity of trading strategies and instruments only increasing, any delay in deployment of new models introduces unnecessary risk to both the project itself as well as the trading strategy. Steps in the process must be constrained to only those that add value. It is critical to isolate those components that add value from the rest of the system.

In the implementation stage, every added line of code requires additional time and resources for planning, quality assurance, packaging, and deployment. Any handling or review of lines of code that do not directly contribute to competitive advantage (i.e. trading profits) is wasteful. Grouping non-critical infrastructure updates with model implementation (in K|V Stage 3) slows deployment and decreases profits. Using interchangeable components, such grouping is unnecessary.

A robust technology architecture can make deploying new trading models almost as simple as (1) coding a model, (2) changing a few clearly defined configurations, and (3) clicking a deploy button. Focusing on configuration over coding lowers operational risks by significantly shortening the project time box.

Note: Model code is rarely the primary bottleneck in technology platform development. The infrastructure that handles things like messaging, order management and connectivity is often much more important to the speed and accuracy of a system. Optimal configuration of performance related components and architecture can lead to faster trading systems.

RMD can be a foundation for an open platform for automated trading that addresses these issues. Open source solutions, such as QuickFix (FIX engine) and ActiveMQ (Messaging), often have a large community of contributors, and therefore, define the cutting edge of their various target solutions. Our architecture, which we call Groundswell, leverages existing open source initiatives to create a framework for systematic finance that can be quickly configured, assembled, and deployed using a workflow engine.

Within the Groundswell reference framework, vendors can create configurable components (such as ticker filters, indicators and trader filters), and plug them directly into the standardised architecture. The best methods for handling these components already exist, and re-inventing the wheel is wasteful.

Open standards such as Marketcetera and Groundswell will allow vendors to create tools that link architecture components together without compromising the proprietary parts of trading systems. These new initiatives will change the landscape of systematic finance by creating leaner, faster, more competitive technologies and trading strategies. Going forward, firms that can optimally balance strategic and non-strategic components of their architecture will see larger profit margins, just like Ford.

Layer Description
Tick Filter
The tick filter layer prepares tick data for usage by the indicator generator.
Different indicators may require filtering tick data in different ways.
Indicator/Theoretical Value Generator The indicator generator receives filtered tick data and generates indicators such as volatility, moving averages, RSI, or other custom indicators.
Trading Engine The trading engine receives indicators and identifies if the conditions
necessary for position entry have been met. The trading engine is 100%
Profit Target/Stop Loss Engine The target-stop detector monitors positions and generates exit signals. It operates separately from the trading engine to ensure exit logic does not
impede the execution speed of the trading engine.
Trade Filter The trade filter encapsulates business rules that are not specific to the trading engine. A trade fi lter may, for example, limit the number of trades
made within a specified period of time.
Order Management Adapter The order management adapter defines an interface that can be implemented by vendors for tracking activity. This allows for changing
brokers or software vendors for bookkeeping and performance management without changing code related to trading, risk, or pricing models.
Order Management System The order management system is responsible for ensuring orders are routed to, and managed correctly on, the appropriate exchange.
Session Manager The session manager is responsible for keeping track of fills, so the information is available across multiple layers.

RMD Recommendation 3: Coordinate staff with management

Staff execute processes. It is management's responsibility to ensure that processes are effective at generating profits and continually improved. A consistent, process approach to management provides tools and measures to control the process of trading strategy design, testing, implementation, and management. Further, it will ensure that stakeholders have the necessary input into decision making, without inextricably tying management to every decision; decision-making can be delegated and monitored, keeping the innovation assembly line moving.

Industry-wide failures, like the current credit crisis, are nothing new. From railroads to the Internet, over-investment followed by collapse of the speculative bubble is a hallmark of maturing industries as they enter a new phase. Failed ventures are scrapped for parts and merged in with survivors. This is when industries begin to understand risks inherent in earlier decision-making processes, and this is when new and better processes and oversight methods emerge.

Figure 5: Intersection of disciplines in systematic fi nance
Figure 5: Intersection of disciplines in systematic finance

Interest in measurement and governance has grown in the financial industry as firms have come to realise that project risk adversely affects the bottom line. Hallmarks of good-governance are:

Stakeholder involvement: Ensuring that stakeholders are identified with clear responsibilities and deadlines, and are regularly consulted (at gate meetings) on progress and goals. Stakeholder concerns are systematically documented and addressed.

Transparency: Highlighting of critical activities of the organisation to ensure management, investors, and regulators understand business processes.

Efficiency: Promoting efficiency by minimising wasteful activities. The output of one stage is the input to the next, so efforts should always be focused on mission critical, value-adding activities.

Manageability: Encouraging consistent and timely flow of information to decision makers, so projects have the resources to keep moving while top management has the information needed to guide projects in the light of the portfolio of development projects.

Coordination: Promoting cross-functional and self-organising teams regardless of existing corporate hierarchy or the functional roles of team members.

Separation of responsibilities: Focusing on standardisation of processes and technology that automate trading, pricing and risk management so that firms can safely separate strategic and non-strategic capabilities.

Top management is responsible for delegating authority and making sure that metrics are integrated with process to ensure these benefits. Traders, quantitative analysts, developers, and risk managers as well as top management can use the K|V reference model to facilitate proper interactions among their peers.


The story of the tower of Babel shows how difficult it is to coordinate different groups with different languages to work together to accomplish a task. The differing vocabularies between the functional areas involved in trading system development can doom projects in a similar fashion.

Different vocabularies and skills among traders, quantitative analysts, and programmers often inhibit collaboration and lead to failed projects. The risk of losses due to simple miscommunication is greater than most firms should be willing to accept.

To succeed, firms can be well served by identifying a common set of skills required of all members of these groups toward developing a core professional discipline, then building an assembly line of trading models around lean, well-understood processes.

Going forward, firms that achieve this goal will, like Henry Ford, succeed at the expense of those that don't.