Do Your Testing Methods Deliver?

Issue 08 Q1 2008
Automated Trader Magazine

Much of the underperformance of messaging systems can be attributed to inadequate measuring tools and testing conditions, according to Barry Thompson, Founder and CTO, and Dave Lauer, Senior Systems Engineer, Tervela.

The financial services industry is deploying so-called next generation network devices and software systems to combat the exponential increase in market data volumes and internally-generated messaging traffic. While messaging technologies, such as message-oriented middleware, have evolved, the means used to appraise them have not. This disconnect has led to ‘game-changing’ technologies failing spectacularly when moved from testing into production. Buy-side firms, in particular, are feeling the pain. For example, the ability to process and distribute market data relies heavily upon messaging systems, such as feed handlers and complex event processing (CEP) engines. For order execution, similar challenges exist for both order management systems and FIX engines. The sell side is not immune to this either and often leverages the same technology to service their buy-side clients. Though there are many components to a financial services trading infrastructure, the emphasis of this article is on the performance of messaging systems because they are so critical to other parts of the system. We also examine how replayed and live data yield completely different results when trading firms are testing competitive technology offerings.

Expectations and reality diverge

Recently, a large investment bank in New York walked us through their testing approach for new market data infrastructure products, which was created after a messaging system for market data distribution vastly underperformed against published test results. They lamented that even with their own lab they lacked the requisite methodology and framework to achieve a level of accuracy that would ensure seamless live deployment. So why does production reality not meet real world expectations given how well these technologies performed in the lab? ...

Subscribers Only Content

This article is for subscribers only, please either log into your account above or click here to purchase a subscription for Automated Trader Magazine.

click here to return to the top of the page
77