href="http://streambase.typepad.com/.a/6a00d8341c63ff53ef0168e97221f0970c-pi"> alt="Error debug" class="asset asset-image

at-xid-6a00d8341c63ff53ef0168e97221f0970c"

src="http://streambase.typepad.com/.a/6a00d8341c63ff53ef0168e97221f0970c-320wi"

style="margin: 0px 0px 5px 5px;" title="Error debug"

/>The high-profile trading failure at BATS this week

sparked a rash of negative publicity, gracing the

front page of media outlets from the Wall Street

Journal to Forbes. In the software industry, more software

companies have proclaimed their techology would have sheltered

BATS from their storm than cheap umbrella sidewalk vendors on a

rainy day in New York City.

An especially

troubling rash of "thought leadership" came

from my own industry - real-time analytics and complex event

processing. Forbes, for example, published an article by SAP (who

sells a CEP product) href="http://www.forbes.com/sites/sap/2012/03/29/bats-hits-the-rocks-which-may-help-fill-its-sails/"

target="_self">BATS hits the rocks, which may have lifted its

sales that proclaims that the BATS incident portends

that "CEP technology is about to move from the

backwaters of data management technology to center

stage."

In Wall Street and

Technology, John Bates, the CTO of Progress Software was quoted

in the artice href="http://www.wallstreetandtech.com/electronic-trading/232700195"

target="_self">Flash Crash at BATS renews Market

Concerns to say "[BATS] clearly

lacked adequate risk controls such as real-time surveillance and

monitoring. Hopefully this incident serves as a wake up

call." What kind of software do you think Bates sells?

Drum roll please... real-time surveillence and

monitoring!"

My company sells these

kinds of solutions too. And I know BATS well (disclosure: BATS is

not a StreamBase client), and they are as technologically

advanced as any firm in the world. So this kind of ambulance

chasing is not only unbecoming, it's also dead wrong.

It's like suggesting Formula One racing will be safer if

the cars can be made to go faster (CEP) and we do a better job of

filming them as they go careening into a wall (monitoring).

Indeed, BATS quickly reported that they had a bug in their

system, so comments about risk management and surveillance are

irrelevent.

That said, there's an

issue here for the capital markets - but the issue is cultural,

not technological.

What's

happening in the capital markets is that we are in the midst of

real-time automation renaissance. 10 years ago, less tham 5% of

stock trading in the U.S. markets was done algorihmically; today,

as much as 70% is done on computer-implemented, and human

designed, algorithms.

Larry Tabb, the founder

and chief executive of the Tabb Group, said in the

Times that "The markets basically gutted their

high-cost, nonstop infrastructures for very fast, low-cost

infrastructures." Now we are talking - Larry is on the

right path!

In software, mistakes happen

frequently. Professor Neil Johnson at the University of Miami

a href="http://arxiv.org/ftp/arxiv/papers/1202/1202.1448.pdf"

target="_self">nalyzed 18,520 "ultrafast Black Swan

[trading] events" from 2005-2011.

In href="http://www.nytimes.com/2012/03/29/business/mishap-at-bats-stock-exchange-is-indicative-of-market.html?_r=1&pagewanted=all?src=tp"

target="_self">Stock Market Flaws Not So Rare, the

New York Times that just one of the 13 U.S. stock exchanges -

NASDAQ - declared 139 breakdowns at other exchanges last year

alone, up from 69 the year before.

Software

breaks. We have a word for it: bugs. But that's not new.

What is new is that the errors can do damage so much more

quickly, and, because the world is now so hyperconnected,

everyone knows about it more quickly than ever

before.

So what is the magic elixir

to making the capital markets fast, fair, and safe?

There are still voices out there

that seem to think that the answer is to discontinue the use of

computers for automation. Ha! The whining about the evils of

capitalism and sophisticated technologies continues to rain down

from dissenters posting their admonations from inside their pup

tents via shiny new iPad 3's. These dissenters need to

Get Over It - technology is here to stay, and its use is

expanding quickly from Wall Street to Main Street.

Making automation safe and reliable requires complex,

cultural change:

1. Continue to

refine reporting requirements. You

can't manage what you can't measure. In the

United States, exchanges didn't report outages on other

exchanges until 2007. Nasdaq reports that there have been 110

outages in the last year like the one that triggered the BATS

mishap. Continued measurement and transparency will lead the

industry toward greater reliability. With more analysis and

reporting, more accountability is possible, and

regulators could focus on correlating events among exchanges and

brokerages to look for systemic flaws.

2. Refine and lighten regulation;

don't just pile it on. Dodd Frank and

MIFID 2 have put regulation on the must-do agenda of all firms.

But it's still largely up to the reader to separate

what's essential from what's fluff, and

that's not effective. Exchanges like BATS are

extrodinarily regulated - so more, heavier regulation is not the

answer.

3. More systemic

attention to software

quality. Moore's Law

- the idea that the power of computing doubles

approximately every two years, does not apply to software

quality. The BATS crash shines a light on the issue

that is inbred in innovation; the rising importance of software

testing, quality, and simulation. Most firms have ad-hoc

processes for quality and should have continuous testing,

regression testing, and real world simulation. CEP can

make systems go faster, and surveillence helps spot problems more

quicky, but when more attention is paid to testing and

simulation, less errors occur.

These changes are complex and cultural, not

technological. Tools will help, but not solve, the challenge of

ensuring a safe, reliable, and fair market.