• Round-table participants
• Kevin J HoustounChairman, Rapid Addition
• Emmanuel CarjatCEO of Atrium Network.
• Hirander Misra COO of Chi-X Europe
• Frederic Ponzo MD, NET2S
• Scott IgnallCTO, Lightspeed Financial
• John OddieCTO, Celoxica
• Petrina SteeleVice President of Business Development in Europe, Equinix
• Shawn MelamedCEO, Correlix
• Rob CiampaVP Product Management, Tervela
• Ben NewtonWebSphere Front Office, EMEA Technical Sales, IBM
• Mark AkassCTO, Global Banking & Financial Markets, BT
• Alasdair MooreDirector, Fixnetix

Do you consider that an industry standard for latency would be useful?

Kevin J Houstoun: Yes, at the moment, building low latency systems is way harder and more expensive than it has to be, because each builder has to measure each components performance to determine how well it performs, the difference are vast yet all vendors claim to be the best. One "high performance, low latency" FIX engine we know of is lucky to handle 2,000 msgs/sec another 100,000 msg/sec yet there is no way of comparing them other than performing a Proof of Concept with each. {comment}Emmanuel Carjat: An industry standard for latency is inevitable. Given there is so much investment being made, there needs to be a way of "measuring" the success or identifying how much further you need to go. However the challenge starts with: who is measuring "latency" and what are they measuring?
Historically, connectivity providers have been a source of time-stamps. As the market move towards finer granularity of measurement (ie microseconds instead of milliseconds), connectivity providers have an opportunity to continue providing this service, using highly accurate time stamps.{comment}Hirander Misra: Yes, it is important to compare apples with apples as far as latency is concerned to ensure consistency. An independent standard to benchmark against will greatly assist this process and distinguish real performance from marketing hype.{comment}Frederic Ponzo: Indeed, it would be. However they are some basic impracticalities that get in the way. Latency measurement across multiple sites is notoriously hard to do, because of simple things such as clock synchronization, and harder aspects such a deploying measurement kits in somebody else's data centre.
Transparency of the latency in trading venues, or network links, will rapidly mean that clients (banks and funds managers alike) will start asking for SLAs on latency, which obviously creates a negative incentive on the providers. Also, latency is a relative quantity; it is about how fastER or slowER you are against your competitors. A standard benchmark might disclose too much information in the eyes of some players.{comment}Scott Ignall: Not particularly. The standard is defined by the customer and he or she will seek out the providers that can offer the lowest possible latency. A significant reason why we have excelled in this space is due to the speed of our automated trading interface and market data products. It is our belief that execution times over 1 millisecond roundtrip (from our clients to the market and back) is really not "low" latency - our clients would certainly agree.{comment}John Oddie: A number of companies are building businesses based on low latency trading, some are vendors of tools, others service vendors like Celoxica and of course the companies that perform automated trading devote significant dollars to achieving the lowest possible latency in their trading applications. Standards will probably emerge as a result of these investments as particular companies achieve dominance. As yet I see no concerted move to open standards in this space.{comment}Petrina Steele: Yes, an industry standard for latency would be highly useful. However, there are almost too many components to understand - as there is currently no generic standard for measuring, it makes latency almost impossible to measure and benchmark. Even the likes of Corvil, SeaGate, Correlix and TS-A, who are highly focused on the latency arena, define their measurements differently.{comment}Shawn Melamed: In high frequency trading, low latency is a critical factor to maximize profitability. Given the complexity of the entire chain of trade events from price discovery to trade execution, there are many points where latency needs to be managed and it is often outside the trader's control. By setting an industry standard for latency, traders would be able to pinpoint and identify latency bottlenecks to achieve best execution.{comment}Rob Ciampa: Yes, we believe there should be latency standards. And, the reality is there should be many - not just one. An industry standard benchmark for latency would be useful for out-of-the box evaluation of messaging performance, but this is only a start.{comment}Ben Newton: We'd need a set of standards, each representing a consistent piece of the total picture. If we could state exactly what each component absolutely needed to do then we could begin to make proper comparisons. A balance will need to be struck between concise standards and the F1 rule book which stretches into 100's of pages. Without careful thought a standard could be either unusable or meaningless. A well thought out set of standards would be universally helpful.{comment}Mark Akass: Some form of reference model would help in like-for-like comparisons and benchmarks. Otherwise there's a lot of claims made for performance that are hard to verify or assess. It's unlikely to be an enforceable standard, particularly if it adds materially to the costs. It might also be very useful to have agreed time source standards to provide a common reference point and allow data sourced from different elements to be integrated appropriately.{comment}Alasdair Moore: No. A latency standard whilst clearly a desirable benchmark to evaluate vendors, is highly improbable. There are areas where this could more readily be applied such as hardware, where there are fewer variables. However not only is latency non static the question is what are you measuring from where to where, what format, what environment and what throughput? Each vendor whether that be hardware, software or network provider would all have to agree to common metrics which is impossible.{comment}

How would you define such a standard? [As, for example, a 'basis latency' standard for installed components and architectures; or as an industry target; or as part of an initiative to move latency into the co-operative space?] What do you see as the biggest contributors to latency?*

Kevin J Houstoun: Two things, firstly there needs clear measurement and clear descriptions of what is being measured. Secondly there should also be overall benchmarks of who whole systems perform to allow for component incompatibilities.{comment}Emmanuel Carjat: Network - this is the easiest latency contributor to measure and on which to take action. Currently the biggest contributor of latency is distance.
For clients locating their engine in data centres that are at or proximate to the main trading locations, they can address immediate latency issues. However, with increasing fragmentation of the market place, traders need access to multiple venues. This means clients will need to access other venues from this location in the lowest possible latency as liquidity flows amongst the multiple venues.
We are aware of massive investments that our clients are making both in software and hardware. Our job is to ensure that their connectivity can maximise the return on this investment by removing as much latency as possible. This is achieved by being as transparent of possible and using verifiable latency performance.{comment}Hirander Misra: An independent industry standard measurement is important to enable consistent measurement. FIX protocol could possibly help deliver this based on industry collaboration. Firms will still be able to optimise their applications and the standard will just ensure that performance is benchmarked accurately.
Measurement should be possible in terms of nano second granularity and adherence to the same standard clock by way of synchronisation is also important to get an accurate measure.
Given that systems are disparate in terms of geographical location, end to end network latency becomes very important to allow message transfer at close to light speed. This will minimise the time overhead to cover distance. Provision of dark fibre connectivity is becoming more widely used for latency sensitive trading firms. Co location will also overcome this for a set of applications in the same data centre.{comment}Frederic Ponzo: The business requirement for low latency is not an absolute, but a relative quantity, based on the speed of the competition. When carrying out aggressive arbitrage activities, firms need to be faster than the competition. For traditional business, on the other hand, it is enough to be at the same speed as everyone else.
Latency is at its most important during periods of peak activity. These are crucial moments when the costs or the rewards of a firm's latency posture become concrete. The potential cost of latency varies across asset classes and lines of business. To calculate this, firms must review their requirements and capabilities for each of the main asset classes.
Two key latency figures exist for each asset class and type of business activity. One: a low threshold which must be reached to be considered a leading performer. Two: a high threshold above which firms are exposed to arbitrage, or clearly trailing the market.{comment}Scott Ignall: In our view, the most important measurement to consider is the round-trip time from our API to the exchange(s), including the order acknowledgement.{comment}John Oddie: This is a pretty big topic, the problem here is that there are a number of different use cases in the automated trading space. Probably I would try to define end to end latency for a useful component either at the business or technical object level. The definition would have to cover both latency and throughput. In many cases the latency experienced is subject to huge levels of jitter, so architectures would need to offer deterministic performance as well as ultra-low latency.{comment}Petrina Steele: Possibly a mixture of all of the above, as different levels of latency measurement are needed. For connectivity providers solely, they measure distance over actual end to end solution but for others they either measure through the latency of the actual trade or through the components and architectures.{comment}Shawn Melamed: To actually achieve an industry latency standard you need not only the collaboration of trading partners, but also an independent monitoring solution that can measure inter-party latency at a granular level across both networks and applications.{comment}Rob Ciampa: There are multiple forms of latency and many interdependencies that should be considered when endeavoring to measure them.
For example, we should be able to measure the latency of core messaging systems whether they use hardware acceleration or software - under all different load conditions. One place this is being done particularly well is HYPERLINK "http://www.stacresearch.com/taxonomy/term/5?page=1"the Securities Technology Analysis Center (STAC).
Also, we need to introduce and test for qualities of services such as guaranteed disconnected. And, let's not forget the latency inherent in applications … all middleware has an API component and these have to be efficient as well.
Perhaps most importantly, results have to translate from the lab to the street. As we develop more latency standards, we need to mitigate or remove the gap that exists between test scenarios and the real world.{comment}Ben Newton: The most important metric for messaging is application to application latency. This should be measured with a application sending messages across a network and switch to a remote machine which should bounce the messages back to the original sender. The latency can then be calculated across the setup without any issue with clock skew.{comment} Alasdair Moore: As I mentioned previously I do not think it is possible to define any meaningful standard. This would require co-operation either from the vendors, many of whom are unlikely to work towards exposing any inefficiencies they are aware of; or alternatively co-operation from the consumers. The problem with consumers working together is that it is a case of the haves and the have not's. Those consumers who have what they believe are compelling solutions are unlikely to share this knowledge, whereas those without them are less likely to pay for that knowledge.{comment}

What do you see as the biggest contributors to latency?*

Kevin J Houstoun: That will depend, but once you optimise your components often you are left with a system where the box to box IP hop is the dearest piece of the chain.{comment}Frederic Ponzo: Most financial institutions have tuned their networks or taken advantage of proximity hosting solutions. The network, on average, contributes to just 13% of a system's overall latency. This is now the second smallest contributor, below messaging (at just 2%). The new latency culprits are the applications at 65%, and firewalls at 20%; which together contribute to 85% of overall latency today.
Our research shows that the fight against latency in both the middleware and network layers is pretty much a won battle. The industry must now do battle with their applications if they are to finally win the latency war.{comment}Scott Ignall: The first stumbling block is usually network architecture that is not properly tuned for the traffic. The next one is usually application logic with too much complex processing. Less is more in the low latency business.{comment}John Oddie: Again this depends very much on the problem one is trying to solve, but obviously proximity, network bandwidth, message transports, message rates, hardware, operating systems and application architecture all contribute. What Celoxica have tried to do is to break the problems we solve into logical business components that can be highly parallelised so that we can configure our solutions such that each component in the chain gets no more data than it can handle. All of our solutions can be configured so that they are thread-safe without context switching.{comment}Petrina Steele: The biggest contributor to latency is simply a lack of real understanding of components: specifically their usage levels and complicated architectures.{comment}Shawn Melamed: Trading infrastructure is a complex environment and the smallest change from new applications and technology upgrades to new trading strategies can impact the entire trade execution and market data delivery process. Since order executions traverse multiple trading environments and networks, having real-time inter-party monitoring and analysis is the only way to identify the biggest latency contributors.{comment}Rob Ciampa: There are so many systems involved in the modern data architecture that latency is often created by the interdependency of the different components. This makes latency very difficult to measure and the results of testing it don't always manifest as quantifiable latency. The performance lag can masquerade as application issues like lost trades, price slippage et cetera.
Burstiness and volatility both accentuate the weaknesses of software-based systems. A data architecture must be both application-aware and network-aware in order to overcome the congenital deficiencies of software in a legacy environment. When one system fails it can create a domino effect and take down other parts of the architecture.{comment}Ben Newton: The biggest single contributor to latency that I have seen is from messaging. Especially for older systems this can contribute 30%-40% of the end to end time. There are many techniques to speeding this up, but they all roughly fall in two camps, tune what you've got or buy something faster. We have seen a combination of the two proving most successful. For obvious reasons the fastest systems have ditched information access through disk based databases in favour of in memory databases like SolidDB, where lightning fast access and coherent caches have cut many latency headaches. Database access can drop to 20 micro seconds with SolidDB, multiple orders of magnitude faster than a traditional remote database query.{comment}Mark Akass: Global WAN latencies can be 10s and 100s of milliseconds. Within a metro area the network latency falls to sub-millisecond on a specialist tuned network (e.g. Radianz Ultra Access on BT's network). So for low-latency trading you must first of all be able to route orders in the local market geography, e.g by proximity hosting with, your servers are in the same city with fast links to the venues (for example, by using BT's Radianz Proximity Solution), or in the same data centre as a specific venue.
After that, the next material latency issues focus on the application layer, such as the performance of the exchange's trading engine. Historically those were also in the 10s and 100s of milliseconds, but the best are now claiming sub-millisecond performance. For traders seeking absolute lowest-latency performance, the other material factors are the speed of processing the market data coming in, the performance of the computing stack for the algorithm and the performance of the smart order-routing tool.
A lot of this is now very well understood and fast becoming an entry qualification for being considered as being a provider in this market. But the application challenge is inevitably more complex and more variable.{comment}Alasdair Moore: For Fixnetix there are a number of areas of latency mitigation depending on whether we are solving for data delivery or trading connectivity. However in most cases the largest contributor to latency are the customers own requirements which define what is possible{comment}

Do you see latency as a 'moving target', in that it can always be improved? If so, please describe your approach to ongoing latency management.

Kevin J Houstoun: Yes, it's on going, and will probably remain so for quite some time. Ultimately we are all currently bound by the speed of light but are also all nowhere near that limit. At RA our approach is one of continuous research and then building the fruits of that research into our development process. We work very closely with Intel, Microsoft and a number of other high end technology companies to achieve this.{comment}Emmanuel Carjat: Driving down latency is playing "Whack-A-Mole". Once you hammer down one element of latency, other issues then start appearing. Reducing latency is an ongoing process of identifying the biggest source of latency, calculating the investment trade off required to remove/reduce it and, if appropriate, applying the changes. There is no point investing in multi-core threaded devices with parallel processing if it is located in the wrong place or not connected to the most liquid venues.
From the network perspective a typical approach is to target the most critical paths/venues, look at the density/location of customers and build the most direct route.
At Atrium Network, we are conscious of metro and long distance latency and we are always need to be looking for the most optimum path for our customers. Within the metro areas we believe there is no faster a route than using the direct dark fibre connectivity we have on our Exchange Ring. Long distance, there are still some areas for improvement as new fibre systems are activated.{comment}Hirander Misra: Yes, latency is a moving target as it depends on the volume of messages (throughput) in and out of the system at any time which will likely vary. There is no point having the fastest system in the world if it cannot handle the anticipated capacity as this will only lead to queueing and only serve to add to latency.
Latency can be improved but the speed of light is the ultimate constraint at 300 kilometres per millisecond.
At Chi-X Europe we launched more than two years ago with a round trip system latency of 10 ms and a capacity of 30,000 messages per second. This is now 400 micro seconds for co located clients and a capacity of 225,000 messages per second running a single fully redundant matching engine. This has been achieved through software and hardware optimisation. We continually measure latency of each individual component so that this can continue to be optimised further.{comment}Frederic Ponzo: Latency is all relative, as stated above. We still have a long way to go before hitting the limits of physics in more areas. The only place where we are getting close is transmission times over telecom links. Therefore, you need to work by successive waves of improvements, measuring and decomposing your latency in the trade flow and addressing the biggest contributor. Three years ago, messaging middleware where the biggest bottleneck, 2 years ago it was the network. Now the culprits are the firewalls and the applications.{comment}Scott Ignall: Yes, we do consider latency to be a 'moving target.' We measure, improve, ask for client feedback and repeat. This is a constant process for us and keeps our systems as fast as possible.{comment}John Oddie: Clearly there is a theoretical optimum latency level at which current technologies can perform. We try to achieve as close as possible to that level but we also expose all our solutions via an API so that as better hardware or tools become available we can then switch to those. We start with a good abstraction of the model, followed by a good implementation while making sure that as better implementation options become available we can offer those in our architecture. For example we currently offer acceleration of market data and messaging over 1gig ethernet but our architecture will support a firmware upgrade to 10 gig.{comment}Petrina Steele: Yes, up to a certain point. We're committed, along with our community partners within the collocation and proximity space, to understanding and managing the latency implications for our Equinix Financial eXchange customers. We have also enabled an Environment Testing Initiative, run by some of our community partners, that allows our financial customers to fine tune their low latency trading operations.{comment}Shawn Melamed: Yes, latency is a moving target as trading infrastructures are constantly changing. This constant evolution makes low latency management strategies and the sharing of latency intelligence a priority. By using a real-time latency monitoring and measurement service throughout the trading lifecycle, trading partners can collaborate to pinpoint where latency exists to cost effectively make improvements. {comment}Rob Ciampa: For the foreseeable future, latency and the benchmarks we use to measure it will remain a critical part of business performance. This is not so different from the bandwidth issues of a decade ago. When technology gave us access to more bandwidth a whole host of new applications emerged. As latency continues to drop, we envision a whole new set of trading strategies and trading applications. This will drive the marketplace.{comment}Ben Newton: Benchmarking once is a good start, but realtime monitoring ensures that the business gets the ongoing benefit desired. Insuring that each component is committed and able to continuously improve is important as you don't want to be left in the dust when the economy really gets going. The roadmap that we are following includes a few key technological advances, notably GPUs being integrated into the business world and the development of hybrid systems. Advances in CPU performance have generally followed Alasdair Moores law, but there is growing interest in outpacing these confines for specialist applications. With specialised architecture and focused applications we can begin to create hybridised systems that outpace the rest of the industry. One example of hybridisation is that Graphics card manufacturers have expanded the specialist GPU's (Graphics Processing Units) to other workloads. The emerging market is standardising around a few initiatives to ease the development of heterogeneous systems, one of these standards is OpenCL which is gaining rapid adoption around the IT industry. With the rapid progress of these standards businesses have just gained solid access to the unique latency cutting potential that GPUs can provide.{comment}Mark Akass: Some parts of the latency challenge are down to speed-of-light limitations. The latency debate is certainly now talking in the microseconds range rather than in milliseconds, but with a constantly changing market structure even location isn't a constant. Add into this the new venues, multi-asset trading models, new services and market data growth - then you need to be able to manage change and connectivity efficiently. So the market solutions are still evolving. A key area of focus is making latency deterministic - that means predictable and constant.
Where the BT products and services are oriented to precision low latency we apply very stringent engineering standards from inception, through design and validation, and then into production. For example, we inspect the low-level architectures of devices, disabling physical elements and functions and features to assure performance at the lowest latency, and to ensure performance is highly deterministic. It's a bit like tuning a real sports car - you throw out everything that adds weight you don't need.
Ultimately the approach we adopt is constant inspection combined with a complete and comprehensive understanding of end-to-end behaviour to identify the areas to focus on and where most benefit can be obtained.{comment}

Do you consider that you have achieved 'optimum latency' in your business? If not, do you consider (i) that optimum latency is achieveable, and (ii) that you have access to the expertise/tools to achieve it?

Kevin J Houstoun: No, we're a component supplier to firms whose businesses models often depend on having access to the best low latency technology available. We are hence trusted with supplying this to them, ours is therefore a continuous drive to lower and lower latency but we have the expertise and tools to continue to drive latency lower.{comment}Emmanuel Carjat: In London and New York, Atrium Network have identified the shortest path from key venues to client locations and installed its Exchange Ring. Clients are now taking advantage of this ultra low latency connectivity fabric.
The next step for Atrium Network is to extract the performance of the connectivity fabric into a machine actionable format. The aim is to present this to clients as a "Latency data feed" that clients can feed into their execution devices, to use as part of their decision metrics along with order fulfilment, liquidity, price, cost, et cetera.
This is a growing area of development, with standards needed on the format of the data being exchanged with clients as well as the points at which we start making measurements.{comment}Hirander Misra: There is room for continual improvement until you hit the speed of light constraint. As such we expect to be able to continually improve our system performance to come as close to the speed of light as possible. This is achievable by looking at each individual system component and being able to test performance to nano second granularity to enable us to squeeze every bit of extra performance possible from those components collectively.
We use performance measurements tools that are built in house in conjunction with those from a firm called TS Associates to allow us to achieve this level of oversight.{comment}Frederic Ponzo: I'm not exactly answering the question as I am not running an investment firm, but our experience shows that only few banks, maybe half a dozen, have really squeezed every millisecond out of their trading flow. The rest of the industry is playing catch-up.
The reality is that you cannot improve what you can measure, and less than 10% of the banks have the necessary tools to measure and decompose latency, despite the fact that at least four vendors propose adequate tools.{comment}Scott Ignall: No. Despite the fact that we are among the industry leaders that consistently deliver the fastest executions, we are always striving to optimize our systems. In this industry it is essential to continue to innovate and improve in order to stay ahead of the competition. It is this commitment that not only brings professional traders to our firm, but keeps them here.
We built our trading system from scratch using proprietary technology; therefore we are extremely confident that our in-house resources are completely capable of keeping Lightspeed Trading ahead of the competition with our available expertise and tools.{comment}John Oddie: We always strive to achieve optimum latency and to achieve deterministic latency. However nothing's perfect and we seek to improve our knowledge and tools constantly. We try to use a decent logical abstraction as a way of offering that improvement with minimum impact to our clients.{comment}Shawn Melamed: For high frequency trading firms, optimal latency is the ability to have the freshest data and act on it by rapidly executing an order ahead of the competition to secure the best price. Optimal latency management can only be achieved through inter-party latency visibility which enables trading partners to share latency views in a secure environment in order to troubleshoot latency bottlenecks in real time. {comment}Rob Ciampa: It is important to note that Tervela is in its third generation messaging architecture. We have an architecture that embraces this and we have set a benchmark for the industry. We will continue to evolve our products and blend different technologies together that set companies up to remain competitive. We can ride the latency curve with minimal impact on applications utilizing the same hardware-accelerated architecture we created when we first started.{comment}Mark Akass: I doubt that anyone has yet achieved the lowest possible latency. With constant technology enhancements of both hardware and software there's probably no real end to the possibilities, but the fractions-of-a-millisecond returns for the effort are getting smaller.
Optimal latency means different things to different people: for some of our Radianz Proximity Solution clients, optimum latency is not the lowest possible latency to any one venue - it's about low latency to a set of venues, and the ability to react efficiently to market changes without incurring significant latency penalties. For users of our Radianz Ultra Access product, optimum latency is about the low and deterministic network latency from their chosen location.
We also can't ignore the cost issue. With more venues arriving, the cost of connectivity rises, and choices are being made about where to optimise and access to which venue to optimise. Achieving physical proximity to two venues 10 kms apart means choices have to be made.{comment}Alasdair Moore: Ultimately latency is about efficiency so not only can no one ever achieve perfection, getting closer to perfection creates an exponential cost increase for a smaller and smaller uplift. There is however a point where an organisation can say that they have practically achieved optimum latency, if they understand the granularity they require and that this is achievable within their budgetary constraints.{comment}

What is the relative importance to your business of component and end-to-end latency?

Kevin J Houstoun: Only our clients can answer that question but it varies from absolutely the top priority to an important part, to actually high throughput is more important depending on which client you asked.{comment}Emmanuel Carjat: Driving out latency is incredibly important to us so we have built the best possible platform for the lowest latency connectivity fabric. While we cannot see the true end to end latency (e.g. a venue's matching engine to a client's execution engine), we can reduce the latency of the connectivity path data between the clients' and venues' devices.
We are now scaling vertically to invest in a fabric that allows the measurement of latency to be far more granular and then deliver this "latency data stream" to clients in a similar manner to market data.{comment}Hirander Misra: This is very important for our all our customers, most particularly for high frequency trading firms. Times is money and trading latency leads to missed opportunities. Having a high speed, high capacity system enables liquidity providers to update their quotes on the order book much faster than would be able to on most other platforms. In turn, firms with taker strategies can take this liquidity as quickly as possible. All this combines to create greater liquidity and ever greater trading volumes.{comment}Frederic Ponzo: In any institution we audited and compared against eachother (over 50 of them), we always found out that the only thing that matters from a business point of view is the end-to-end number. However, when we go about optimizing the infrastructure and reducing the overall latency, you need to tackle each component in order of decreasing contribution.{comment}Scott Ignall: End to end latency to the exchange and back is the key measurement. This is what the client sees and therefore what we are always trying to improve upon. End to end latency is paramount to our business.{comment}John Oddie: It is at the heart of what we do, our whole business is based on offering the lowest and most predictable levels of latency to the trading community.{comment}Shawn Melamed: End-to-end latency is most important when trading success depends on fast order execution and market data delivery.{comment}Rob Ciampa: It is important to look at component latency as well as end-to-end latency because the systematic view of latency is the most critical. This is precisely why we put both application awareness and network awareness into our messaging architecture. The real differentiator between Tervela and competitive solutions is that we look at so much more than just point-to- point latency. We look at application and network behavior and we make dynamic changes. {comment}Mark Akass: It's hugely important to us. BT is a service provider with specialist low-latency infrastructure services requiring the delivery of measurable, very low and highly deterministic performance. We reflect this commitment in our service level agreements and in the assignment of specialist resources to the development and support of these services. The provision of low-latency services requires precision engineering and constant investment in research and development for all of the component parts that make up the system being provided.{comment}Alasdair Moore: We believe the only true measurement of latency is end to end latency (i.e. from point of origination to application). Not only is this what a trader sees so any impact of a sub optimal solution can be evaluated immediately in terms of slippage or execution ratio but can there be little argument as to which offering is fastest. However in order to ensure we have the most optimal en-to-end latency we also measure latency across each component (where practical).{comment}

Where, and how, do you measure latency?

Kevin J Houstoun: We measure latency both holistically, e.g. from the wire to the wire, and also at a number of internal points within our components to allow us to determine where time is spent in each component.{comment}Emmanuel Carjat: Atrium Network is constantly monitoring our core network, which includes multiple POPs in the Americas and Europe. Clients can measure end-to-end latency across our fabric. An example is a venue that is providing measurement from its matching engine across our fabric to the client's execution engine.
A future development that we are planning to provide is the "latency data stream" out of the network on the performance of the fabric and report this back into clients in a machine actionable feed. The challenge for us, and our clients, is to identify the metrics that clients wish to consume and the standard way in which we can deliver it into the client location.{comment}Hirander Misra: Latency should be split into 2 key components: internal latency (the time it takes for a system to process a message that it receives and send an acknowledgement back out) and external latency (the times it takes for an external application to send a message to your system and receive and acknowledgement back).
These high level benchmarks should be further split out to identify latency across the chain in terms of network, hardware, security components and software. {comment}Frederic Ponzo: Ideally every step of the way, using passive measurement tools that can "sniff" the network for every packet, reverse engineer the application protocols, timestamp the business messages, and correlate them as the data flows and is transformed from one component to the other.{comment}Scott Ignall: We continually measure latency. Unfortunately, this is proprietary information so I can't offer any additional details on how we keep our systems as fast and as stable as they always are - regardless of market conditions.{comment}John Oddie: As much as possible we try to measure latency in line. Because we use hardware acceleration we can in fact measure operating performance in parallel to our normal operations without impacting service quality or performance.{comment}Petrina Steele: At Equinix, we don't measure latency ourselves - instead we work with our network suppliers and our latency engineering architects to better understand their measurements of latency. We then provide an overview of this to our customers.{comment}Shawn Melamed: Correlix offers inter-party latency visibility through its RaceTeam™Service allowing trading partners to manage Latency Intelligence in a secure shared environment to improve market data and trade execution processing. Correlix's unique technology enables trading firms to correlate and trace low-latency trading transactions as they traverse both internal infrastructure as well as external partner environments.{comment}Rob Ciampa: It is important to look at component latency as well as end-to-end latency because the systematic view of latency is the most critical. This is precisely why we put both application awareness and network awareness into our messaging architecture. The real differentiator between Tervela and competitive solutions is that we look at so much more than just point-to- point latency. We look at application and network behavior and we make dynamic changes. {comment}Ben Newton: We use: our own latency monitoring APIs embedded in WebSphere MQ Low Latency Messaging and WebSphere Front Office and good old stack dumps for precise measurements.{comment}Mark Akass: BT offer's latency measurement with the specialist services we provide. We also offer specialist consultative offerings using selected products to augment these services and for use measuring other elements of the full end-to-end solution. For both our core and specialist low-latency products measurement is provided end-to-end (to management demarcation) with different levels of granularity. For our Radianz Ultra Access product we are able to provide the resulting latency data in both online reports and as a formatted near-real-time feed for electronic consumption. Our specialist consultancy services can be tailored to meet customer requirements and are supported by industry specialists.{comment}Alasdair Moore: Where applicable Fixnetix measure latency across the technology stack for both hardware, software and network in real time. We measure this in a number of ways using both third party probes and in-house built monitoring technology. However for some customer deployments this is not practical due to either customer security requirements or the overhead impact of measuring to the microsecond at certain points in a production environment.{comment}

Do you quantify the impact of latency on your business? How, and what is that impact?

Kevin J Houstoun: Yes, low latency drives about 60% of our sales and is one reason our clients buy systems from us.{comment}Emmanuel Carjat: Absolutely! As a connectivity provider we are immediately judged on the latency performance of our network. We have responded to client demands by using fibre connectivity to connect our core together as well as connecting major trading venues and latency intolerant clients. Without this key differentiator we would merely be just another extranet.{comment}Hirander Misra: Yes. Average co location latency has been reduced from 800 micro seconds to 400 micro seconds. This means that you can get 2 trades done in the time it used to take to do a single trade. This has a very positive impact on trading volumes.{comment}Scott Ignall: We view latency as the enemy. It lowers the fill rates for our customers which can adversely affect their strategies. As a service provider, we do not quantify the end results with metrics, however we do recognize this factor as a major lynchpin in our success and, most importantly, in the profitability of our customers. As such, we do not quantify the end results of lower latency, but rather put all our efforts in creating and building the proper infrastructure for lower latency and develop and deploy our own custom operations tools to keep running at top speed. {comment}John Oddie: Low latency is our business! If we do not have a competitive offering we will fail as a business.{comment}Shawn Melamed: Our customers are using Correlix to quantify the business impact of latency as well as to support their latency management strategies. By understanding where latency exists in real-time, market participants improve the profitability of their trading strategies, service providers attract consistent order flow and execution venues see latency analysis all the way to the matching engine for application and network latency.{comment}Rob Ciampa: The impact of latency is something that our customers feel acutely. We work with the most demanding firms in the world when it comes to financial messaging traffic. Latency is first and foremost the most major element of our customers' trading strategies. We've worked with them specifically on latency modeling and how to anticipate measurement. This integrates directly into their trading model, which is their business plan for productivity{comment}

Do you 'benchmark' latency, however informally, and if so, what measure (if any) do you use? [If you 'benchmark' against your own urge to go faster, please say so.]

Kevin J Houstoun: Yes we benchmark latency; we benchmark against ourselves. We measure end to end messaging times. An important part of what we do is not just measure averages but every message; this allows us to calculate not just averages, but deviation and maximums as well. After all if most messages go through quickly but occasionally one takes a long time, that makes it very uncomfortable for traders. Bit like waiting for the "Gotcha" in the old Golfing joke.{comment}Emmanuel Carjat: The benchmark for a connectivity provider is the speed of light. We do everything that we can to identify the shortest path from the client location to the trading venue. In addition, we have implemented an ultra low latency fabric that will scale with the continually increasing market data rates.
We will continue to publish our latency and track our performance. All the while we will continue to strive to identify where we can remove further latency.{comment}Hirander Misra: We use performance measurements tools that are built in house in conjunction with those from TS Associates, the combination allows us to achieve a high level of oversight.
Our tools identify latency across the chain in terms of network, hardware, security components and software. At the system level this is measured in micro seconds and we look at clock synchronisation in nano seconds. {comment}Frederic Ponzo: This is one of the key things our clients are asking us to do: benchmarking them against their competitors.{comment}Scott Ignall: We continually benchmark against our current production latencies. We are always looking to shave more time off our current systems while simultaneously providing the customer with more functionality. This is the challenge we face as a low latency data provider.{comment}John Oddie: All our services come with their own measurement tools as part of our standard offer. We also supply tools that allow customers to benchmark competitive products.{comment}Shawn Melamed: Although an industry wide benchmark does not exist, we're seeing a positive response from buy-side firms to support the creation of industry latency benchmarks. To do this, trading partners need to collaborate through an independent monitoring solution that can measure and report latency statistics in real time.{comment}Rob Ciampa: We measure it formally. We have all of the tools and we view it as a science, not an art. What we end up doing is using statistical models. We look at distribution, average latency and outliers when we model latency. We end up using a distribution curve and we don't want it to be very wide. This would mean our systems were producing outliers which are devastating to traders using high performing algorithms. We've engineered our systems to eliminate outliers and this allows us to measure latency to the 99.999 percentile.{comment}Ben Newton: We continuously test all our products for a range of different speeds, markets, message sizes and volumes. It makes sense for us to benchmark the software that we write. Its important that we not only look for the lowest and average latencies but also drive down the standard deviation to ensure high consistency. Our most impressive figures are for MQ Low Latency Messaging's application to application latency of 5 micro seconds. For market data messages over Infiniband this is a ground breaking performance.{comment}Alasdair Moore: Yes we do. We utilise correlation analysis across multiple benchmarks.{comment} *Some contributors preferred to answer this follow-up to the second question separately, so it is listed twice.

What would be your personal best advice on reducing latency?

Kevin J Houstoun: It would be to look at the problem from an engineering perspective, consider the whole system and avoid unnecessary complications, to continue the engineering analogy a lot of systems that are trying to achieve low latency are build like the competitors to the Lotus Type 25. At the time most Formula 1 cars were evolutions of existing designs, yet Lotus were able to gain great success, against much larger competitors, by introducing technical innovation that made for lighter stronger simpler designs. Many of these innovations continue to define Formula 1 car design today. So in summary, and to misquote Einstein, makes your systems as simple as possible but no simpler.{comment} Emmanuel Carjat: Identify your target market/venue and get to it as close as possible, tap into additional sources using diverse engines and/or lowest latency access into other liquidity venues. This provides a flexible approach as trading strategies change.
We cannot offer advice to venues, but clearly there are venues which are starting to differentiate themselves on speed of execution. Traders will need to strike a balance amongst order fulfilment, price, liquidity and speed as well as the ability to reach multiple markets.{comment}Hirander Misra: Gain a thorough understanding of all the micro level components that make up your system as well as the performance of external delivery mechanisms. Chi-X Europe now offers co location access in the primary datacentre to allow you to achieve an average round trip latency of under 400 microseconds.{comment}Frederic Ponzo: Measure the whole flow, identify the biggest culprit(s), fix it (them) and start again.{comment}Scott Ignall: My best advice would be to follow the KISS method (Keep it Simple Stupid!). At times, networks and applications can have good intentions but are far too complex to handle the speed and throughput requirements for low latency data processing. Flatter networks and fewer applications threads are a good place to start improving the products.
Keep it simple....{comment}John Oddie: Everything starts and ends with a good quality abstraction and an understanding of the component throughput required. Get decent tools to measure actual latencies and measure those in hardware if you can. Design efficient interprocess communications avoiding context switching and unnecessary network hops. Measurement should be inbuilt so that it can provide operating feedback to help you continue to manage latencies in production as well as in development.{comment}Petrina Steele: My best advice on reducing latency would be to talk to some consultancy experts in this space. They can guide you through the different attributes of various suppliers and architectures - after all, every supplier will always pitch their solution as the best and it's good to hear it from an impartial party who will develop the best fit for you.{comment}Shawn Melamed: Trading firms should be using an inter-party latency management service to collaborate with their liquidity and service providers. This will help to improve execution performance and the effectiveness of their market making or liquidity taking strategies through real-time shared latency visibility. By identifying cross-party latency in real time, latency issues are resolved faster and expenses reduced.{comment}Rob Ciampa: Attack the big fish, i.e. take a serious look at how to accelerate messaging because this is where the bottlenecks occur. It is not just about hardware acceleration … it's about adding intelligence to the network and being able to dynamically make adjustments. You need to consider your entire messaging ecosystem as an integrated architecture instead of looking at the disparate elements.{comment}Ben Newton: A year or two ago I would have said 'keep it simple', now the industry has matured its not just about speed but also performance. This is especially important in driving new emerging needs to find fragmented liquidity, integrate greater risk management processes and to strengthen better execution. The need to save power, space and costs is reaching fever pitch, so the demands on the automated traders environment is growing all the time. We are seeing the advantages of hardware consolidation as you can tick a lot of boxes by cutting the footprint of multiple boxes and dropping the added latency by using MQ Low Latency Messaging's hyper fast shared memory interface. To enable these benefits and more, my best advice, come speak to the experts at IBM.{comment}Mark Akass: That would depend on your business, your budget, and your trading strategy (e.g. the target venues and the types of algos you want to drive). Reducing latency is an iterative exercise, so knowing your end-to-end delay and what is contributing to it are the essential first steps. Once you know that you can identify the elements that are contributing the latency.{comment}Alasdair Moore: The best way to reduce latency is ultimately to throw a large amount of money at the problem. The cost of reducing latency across multiple markets is increasingly becoming so costly that only the very largest financial institutions can justify building out their own solutions. Even then they have to rely on having a competitive latency advantage in order to justify this expenditure which they can only confirm once they have made the investment. Therefore if any organisation doesn't implicitly understand what latency advantage they can engineer and what the true TCO will be relative to any revenue opportunity, it makes sense to outsource to a third party.{comment}