The Gateway to Algorithmic and Automated Trading

Rooftop Parties: How extremely small waves are making very big waves in the markets

Published in Automated Trader Magazine Issue 27 Q4 2012

Fibre optic networks are far and away the dominant force in financial communications. But difficult technical challenges with microwave technology have been overcome and routes that are straighter and faster than ever before are now possible. James FitzGerald talks to some of the companies that are truly in the high frequency business.

Gordon Moller

Gordon Moller

When Spread Networks unveiled their newest New York-to-Chicago route in 2010, the fanfare was loud and the impact was immediate. Construction crews had blasted through rock and rough terrain to lay down an 825 mile (1,328 km) underground cable in the straightest possible way, although some concessions had to be made for Lake Michigan. The cost was enormous, but no one had doubts as to whether it was worth it. Spread Networks had lopped 100 miles off the previously fastest route and could offer a round-trip journey in a record 13.1 milliseconds, three milliseconds speedier than any previous path.

An awful lot of trades can be made in three milliseconds and some once-profitable firms could immediately see that they were being beaten to the punch. To make its heavy investment worthwhile, Spread Networks required clients to enter long-term contracts. There was no shortage of takers given the trading advantage they could have.

The ink on those contracts had barely dried before people began looking for alternative ways to connect the futures markets of Chicago with Wall Street. The solution that presented itself was, to use a well-worn expression, an oldie but a goodie: microwave transmission.

This technology, which has been around for decades, was found to shave precious milliseconds off trip times, once certain technological obstacles were overcome.

Microwave transmission has its drawbacks (more on that later), and it's not cheap. But in a world where a millisecond seems to last forever, anything offering a speed advantage was bound to draw fans. So much so that the trend has already spread from the United States to Europe and there is plenty of interest in using microwave transmission in Asia and South America.

Stephane Tyc, a co-founder of McKay Brothers, which has just launched its own Chicago-New York microwave pathway, sees this technology altering the trading landscape. "To compete with the fastest you will need to get a microwave line, or buy the fastest market data from a microwave vendor," Tyc said. "The availability of fast microwave data will bring back a level playing field."

Overcoming obstacles

Microwave has been used for point-to-point communication for more than half a century. The technology has become very well understood and developed over the years, so it can be adapted to many different requirements. As a sign of its longevity, the BT Tower in London only recently began removing microwave dishes that had been installed in the 1960s.

As the name implies, the wavelengths are very small, which means the frequency is very high. This has a few advantages. It allows a lot of information to be packed into a small space and it means the waves can be directed in exceptionally narrow beams. Occupying the radio spectrum between 1.0 and 30 GHz, microwave links over the last 40 years have been used primarily for telecommunication.

The speed of light travelling through glass is considerably slower - roughly 60 percent - than the speed of light travelling through air, since glass has a higher refractive index than air. But there's a catch to using microwaves. In order to transmit across long distances, a signal has to be repeated, and there needs to be line of sight between a series of repeater towers. Each time the signal is repeated, latency builds up as the digital signal is processed, regenerated and cleaned up. Over a route of many miles involving numerous towers, signal processing can take up to 500 milliseconds, taking away any benefits from the time that had been saved due to refraction issues.

It wasn't long before a solution to the problem was found.

"What we've done to reduce the latency of the system is to serially reduce the amount of digital signal processing in the modem unit," said Gordon Moller, CEO of Cielo Networks.

"Up to now, in every case at a tower microwave relay site, the radio would still do a full digital regeneration. As we got into designing and supplying these networks early last year, we decided to evaluate whether full digital regeneration at every site was really necessary," Moller said. "If the received signal quality was good, we might be able to skip remodulating it one or more times consecutively in a series of relay sites. Theoretically it looked doable, so we built and refined prototypes and the concept worked."

A modern microwave link consists of a radio (a transceiver), which is analogue, and a modem, which is digital. Cielo Networks markets a product that sits between the radio and the modem that has the capability to bypass the modem completely, so the signal stays in the analogue domain and its frequency shifted and retransmitted.

Stuart Little

Stuart Little

"It takes the relay site latency from seven microseconds down to well below one microsecond. So we're in the nanosecond range for site latency now," Moller said.

The next hurdle was setting up links.

There are two options when setting up a microwave link. One is to build a series of towers and set up dishes on each. Since point-to-point microwave links require line of sight, towers must be set up on the tops of mountains and hills. That can get costly. A second option is to rent space on existing towers, and install dishes. This is the most cost-effective option, and is by the far the most popular.

The same year that the ultra-fast route from Spread Networks went live, Tyc and Bob Meade co-founded McKay Brothers. They teamed up with microwave networking provider Aviat Networks, which already had microwave links set up across the United States that were used primarily for mobile backhaul. Their route has 20 towers and deviates from the straightest possible line by only four miles.

"We use existing assets - leasing space on existing towers primarily," said Stuart Little, director of global corporate marketing at Aviat Networks.

"We're using available microwave spectrum. You can go to the national regulator in each country and lease a slice of frequency. It's not expensive."

"You're looking at maybe a couple of hundred dollars a year for each link. Once you've got it up and running, the costs to operate them are very low as well. Overall, the cost of doing maybe a 1,000 km microwave link is a tiny fraction of the cost of building a fibre route."

In July of this year, McKay Brothers opened their microwave path from Aurora, Illinois, where the Chicago Mercantile Exchange technology hub is located, and Secaucus, New Jersey, a key data centre for Wall Street.

Tyc is confident about the speed. "Latency is not something that we disclose, but it's somewhere between eight and nine milliseconds from Aurora to Secaucus. Our path is very, very straight, and eventually we will be, if not the fastest, then certainly tied at number one. I doubt anyone can be faster than us."

Mark Akass

Mark Akass

McKay Brothers also does not disclose prices for a slice of bandwidth. But Tradeworx, another company that has set up a microwave link between Chicago and New York, charges a subscription of $250,000 a year for a 150mb spectrum.

Raindrops keep falling on my network

Microwave, however, does have its drawbacks. And those who have been cautious about the technology are not shy about noting them.

"There is mixed feedback from clients about the usability of this, and there are concerns about the error rates," said Mark Akass, chief technology officer at BT Global Banking and Financial Markets.

"The error correction is essential if you've got a slightly noisy medium, in order to make sure any data gets transmitted effectively. If you're sending market data and the transmission medium is inherently noisy, then the issue you may encounter is retransmitting data. Then you have the issue of making sure that the messages don't get out of sequence. One of the concerns for microwaves would be the quality of the path and the error rate. If you're going into microwaves you tend to have technologies for link-level error correction."

Large rain drops have been known to affect microwave transmission when the frequency gets higher and the wavelength smaller. Low microwave frequencies of six and seven GHz can support line-of-sight links up to 70 or 80 km, covering vast distances from one tower or mountaintop to the next. Those frequencies are immune to rain, but as the frequency gets higher - 18 GHz or more - rain attenuation becomes a significant factor. The industry has adapted by varying the frequency based on where a path is located.

Moller of Cielo said the company has decades of data on rainfall rate and type, from the small-drop, mist-like conditions common in Seattle to the heavy downpours that can drench the Gulf Coast. "So we know which frequencies to use for any desired path distance and reliability objective. For instance, if we're going across Montana, we'll use different frequencies than we would in Florida."

John Heflin

John Heflin

"Depending on the area's average rainfall conditions and the link reliability goal, typically 18 GHz can go 25 km, 23 GHz 10 km, and 39 GHz perhaps up to 5 km. Since most of these long haul networks use predominantly 6-11 GHz with quite large 6-, 8- or 10-foot diameter antennas, rainfall is really not an issue."

Little of Aviat Networks rejects the premise that microwave technology is not reliable enough for the demanding world of financial markets.

"The idea of microwave being a dodgy technology which drops in and out is just wrong," Little said. "I think a lot of people base their conception on their own experience with cell phones, the way signal drops in and drops out. We've engineered links 200 km across water between islands in different parts of the world where the conditions are significantly worse than in North America or Europe, and those links are rock solid."

"Microwave is used in all sorts of mission critical applications," he added, "emergency services networks, for example, where they can't tolerate even the slightest risk of failure - and microwave can provide that security."

But the biggest concern about microwave transmission is the amount of bandwidth it can handle. Fibre optic cable can offer firms near limitless bandwidth, more than enough to send over large volumes of market data. If you're using microwave in a metropolitan area over short distances, you can use millimetre-length waves. These have frequencies of between 30 and 100 GHz, so they offer higher bandwidth. But waves at these frequencies are much more susceptible to environmental conditions. To shoot across large distances, a longer wavelength is needed, which results in lower bandwidth.

Those offering a microwave link between Chicago and New York lease bandwidth in slices of 150 megabytes, which is too small to send across large volumes of market data. Once your data starts backing up, the likelihood of dropping packets increases. During a spike, this becomes even more likely. Since market data is made up of incremental, multicast messages, missing one causes serious problems.

Rob Walker, chief technology officer of xCelor, a low-latency technology vendor, said one way to deal with this issue was to process only the necessary data.

Rob Walker

Rob Walker

"Symbol filtering helps to overcome it [the problem of low bandwidth]. Our FPGA card discards symbols not specifically requested, so you're only sending across what you need. We also send a message a number of times. We measure in real time the condition of the microwave link, and depending on what's going on, we will send multiple redundant copies of a message, so that even in a spike it's unlikely you'll lose a message."

Still, the practical limitations of microwave technology in terms of data capacity are serious enough that both its detractors and proponents think more work is needed to make it a more viable alternative to fibre optics for many functions.

David Barksdale, chief executive of Spread Networks, said fibre optic and microwave technology should be viewed as entirely different beasts.

"You may see us integrating this with our fibre network in the future," Barksdale said. "Comparisons to fibre are pretty much an apples and oranges comparison. If you look at our fibre network, we've been operational for over two years. During that period we've had 99.999% reliability - it basically never goes down. And we provide virtually unlimited bandwidth to our customers."

That, he said, means clients can transport all their trade data and market data, with no serialisation delay. "The market data being transported via microwave is like a drop in a bucket of market data out there. Microwave is certainly fast, but you're very limited as to how much data you can send across."

London-Frankfurt in under five milliseconds?

Meanwhile, new routes are on the cards. New York-Chicago is fully operational, but other routes such as Chicago-Washington DC and Washington-New York are being developed. In Europe, microwave pathways that link London with data centres in Slough to the west and Basildon to the east are already up and running. The big prize is London-Frankfurt, and that is due soon. Routes in South America and Asia are also future goals.

"The microwave market is starting to settle and it's easier to identify the real players," said John Heflin, senior vice president of business operations and delivery at Interactive Data arm 7Ticks.

"From a supply perspective, some of the lowest latency critical paths and the infrastructure to support them have already been assigned, particularly in North America and EMEA. Between New York and Chicago for example, some of the best towers and spots for transmission on those towers are already spoken for."

As for Europe, work is already underway on a Basildon-Frankfurt link, despite challenges such as sending a signal across the English Channel. A company called Newgig Networks has begun construction of a Basildon-Frankfurt microwave route which is due to be activated soon. "Expected latency will be 4.85-4.9 ms RTT, with a sub 5.0 SLA," according to the company's website. It also said the link was 80 percent sold.

Tariq Rashid

Tariq Rashid

"We expect to go live in Q4," said Jake Zoldan, managing partner of Newgig. "We will not have a firm date until we are around 30 days from completion. The site acquisition process is fluid and specific tasks are completed in serial fashion. This process helps ensure the integrity of the design. The end-points at Basildon and Frankfurt are both secured. Based on initial bandwidth requirements from our clients, and combined with our EU business development plans, our network will support 7-12 total clients."

Meanwhile, in Basildon, NYSE Euronext has a microwave platform on the roof of its European Liquidity Centre which is currently being extended to cope with demand."Customers install their microwave equipment on the roof platform, and we then cross-connect them," said Tariq Rashid, managing director of European colocation at NYSE Euronext. "Firms using this service are member firms, buy-side and service vendors."

Rashid said the advent of microwave technology had presented a strategic choice for the exchange.

"Towards the end of last year we were approached by a number of microwave specialists who wanted to set up links in Europe connecting our liquidity centre in Basildon to other liquidity centres, primarily Frankfurt and other venues in London."

"We had to decide whether to build our own microwave network, or whether to facilitate firms building their own microwave networks into the facility. We went for the latter after consultation with our customers," he said.

And what about a link between London and New York? That, as the saying goes, may just be a bridge too far. The lack of suitable sites for repeater towers in the ocean means that a long distance, transatlantic link is impossible with the current technology.

But that hardly means enterprising companies won't be scratching their heads over ways to cross the Atlantic even faster.

"The interesting thing is, what's next after microwave?" said Heflin of 7Ticks. "People are looking at how to enable technology through the air, or across the ocean, in a way that's cost effective. Microwave is certainly not the end of the low latency line. It's just another phase in the race to zero."