Tech Q&A with CEP Developers, and StreamBase
First Published Friday, 24th February 2012 02:30 pm from StreamBase Systems : Mark Palmer
The opinions expressed by this blogger and those providing comments are theirs alone, this does not reflect the opinion of Automated Trader or any employee thereof. Automated Trader is not responsible for the accuracy of any of the information supplied by this article.
Today StreamBase hosted an online roundtable entitled
"How to Empower Your Development Team with CEP and
Visual Programming" featuring developers from J.P.
Morgan, GAIN Capital, and Datasports. Combined, the panelists
have 35+ years of development experience, and years of CEP
experience, and they were kind enough to share those experiences
with us, discussing the adoption of CEP technology, developing
applications with CEP, and the best practices they've learned
along the way.
For developers, development
managers, architects, and business stakeholders this roundtable
tells an interesting story about how CEP technology can truly
transform the role of the developer. Take a look at some of the
highlights below, href="http://www.streambase.com/news-and-events/webinars/cep-developer-roundtable/"
target="_blank" title="Watch the replay">watch the
replay, then comment below and tell us about your
experiences with CEP.
Talk about your experience with
Phil Martin, Datasports: I
like using StreamBase because I like tools that let the computers
do stuff that the computers are good at, and let me do stuff that
requires thought. As a developer, you wind up doing some tasks
over and over again; the structure of the application, domain
specific plumbing, etc. Everytime feels like wasted effort.
StreamBase lets me focus on the stuff that requires thought, not
the details of implementation, connectivity, processing events,
and so on. Another comment is around the aggregate operator. It
is analogous to another API to learn. It's an incredibly rich and
flexible operator that I think almost qualifies as its own sub
language or sub paradigm. There's a lot there and a
lot of subtlety in its behavior, learn about and study
Mike Grandilone, J.P. Morgan Asset
Management: The shift to a message passing structure is sometimes
a shift in thinking. It's key to understanding how
StreamBase handles the life of a tuple; we gained that through
about the applications you've built with
Joseph Chua, GAIN
Capital: After 4 days of training and going through the examples,
I built my first StreamBase application in just three hours, and
it was fully deployed in a week. It connects to market data,
manipulates the data, and stores it in a high-speed database.
We've also built market making and real-time surveillence
applications, and many more to come.
I've built several applications with StreamBase. One is
an "indication of interest" (IOI) app that
monitors trading interest and compares it against a CRM system
and sends alerts to individual sales traders because they have a
customer who either has traded on the symbol, transacted above a
given threshold, or has a real-time order for it. The correlation
of these factors in real-time also includes real-time news
sources. Also, I've worked on two algorithmic trading
systems that host their strategies in StreamBase and perform
real-time risk checks on the algos, and an ETF arbitrage system
where there was only one strategy, but it was a very involved
Mike: We brought a smart order
routing system up and running in 6 months with 2 developers,
which we thought was very fast. We also got the business people
involved in the canvas and the development - this is a very
important factor in getting applications done quickly.
Talk about the benefits of StreamBase and
StreamBase helps facilitate communication. Whenever you want to
convey information about a system, you always draw a picture. And
they almost always look like StreamBase Event Flow. If you have
taken the necessary care to clearly delineate modules, then,
depending on the technical sophistication of who you are talking
to, you can choose how far to drill down. It's extremely
helpful in faciliating technical discourse; with the business
users, it saves time.
Mike: Visual development
has really improved the speed of our development. Especially when
you're doing a proof of concept. It helps you understand
the application's logic, and also communicate with the
business; in typical programming environments like Java,
you're not going to show many people the code, but you
can do it with StreamBase.
Joseph: The big
value is that all the technical details are abstracted away in
terms of connectivity and low-level code; we can then communicate
with the business owners, so development cycles are shorter and
more efficient. The other value we have found is they we can very
quickly build a prototype solution so the business can make
decisions more quickly with respect to requirements and decisions
to develop the applications.
Phil: One thing
that's helpful: with StreamBase, you can tell at a glance
that spaghetti code is spaghetti code. You can tell if an
application was thoughtfully designed without hours and hours and
hours of code review.
practices have you learned from your experiences with
Mike: You need to shift
your thinking, to think in terms of events, rather than
traditional static applications. And, just like any application,
build a testing environment at the same time as your application
Joseph: Identify the connection points
quickly, be open minded and willing to unlearn the more
traditional ways of development, use StreamBase interfaces, and
make sure you continuously refactor the application.
Phil: There are lot of resources for ramping up
StreamBase skills - invest the effort to develop your skills. The
speed of putting together something quickly is a great asset. The
thing to keep in mind is that although the development speed of
StreamBase is a benefit in the short term, it can, if
you're not careful, work against you in the long
Phil: While the principles
aren't different, the practice IS different. The first
thing that sold me is seeing the ease with which you can change
and tune your threading model, take things in and out of the
threading part of your system, with multiplicity and dispatch
methods. Scalability and performance is about taking more
advantage of the cores in a system, and careful management of it
becomes more and more important.
What was your experience integrating other
code into CEP?
Mike: We have
separate routing logic written in Java for routing logic that we
integrated with our StreamBase application. We've done a
lot of work with custom Java operators in StreamBase -
it's very straight forward - no problem in doing it.
There's lots of power in developing in Java for code that
isn't event driven that allowed us to bring in the power
of algorithmic trading as a drop-in to the StreamBase
environment. Also, we have a C# front end GUI, which uses the
.NET interface from StreamBase - this was easy, although the
challenge was how to take a GUI that isn't event-based
and make it work with an event-driven application. We're
in production with that now.
I've put two components onto the href="http://streambase.typepad.com/streambase_stream_process/
www.streambase.com/sbx" target="_self">StreamBase Component
Exchange - mostly adapters. The publishing process was
straightforward. One is an adapter that watches for events on the
file system and creates StreamBase events when files are created,
renamed, modified, or deleted, and another one listens for calls
on a web service interface and creates events for the
applciation. I suggest other developers look at the utility
classes - see what are other developers are doing, and pick up
some tips and techniques and helpful hands-on examples of how
they are building their applications.
StreamBase Resources for
Below are the
StreamBase resources mentioned in this
- Trial version of StreamBase:
- Product documentation:
- Personalized demonstrations:
- "How To" developer webinars:
- StreamBase Component Exchange:
- StreamBase University:
Training and Certification: href="http://streambase.typepad.com/streambase_stream_process/www.streambase.com/community/training-and-certification"
- Datasports blog for StreamBase CEP developers: