Persisting data in a Real-Time distributed system
First Published Thursday, 16th July 2009 12:27 am from Real-Time Innovations (RTI) : Jens Pillgram-Larsen
The opinions expressed by this blogger and those providing comments are theirs alone, this does not reflect the opinion of Automated Trader or any employee thereof. Automated Trader is not responsible for the accuracy of any of the information supplied by this article.
Most of the distributed systems
we deal with at RTI have performance constraints at their core.
Either the system is pushing the limits of the available
resources, or the action-reaction timing is critical for a given
event. In other words the constraint might be on throughput or
latency (or increasingly latency vs. throughput). In these kinds
of systems persisting data is a real challenge. In many systems
it is becoming a requirement that the distributed data is
persisted. Take for example flight systems and automated trading
systems, where persisting data is necessary to adhere to
regulatory demands. At RTI we have set out to make persisting
distributed data as minimally intrusive, performant and
configurable as possible.
Recording Data
The simplest case of persisting data in a distributed
system is recording the data to a file system. This can be done
locally for each node, or centrally on a
"recording" node. Each system setup solves
different problems. Persisting local state on each node allows
you to recover from a local crash by reading that state during
recovery. Recording all the data on a centralized node allows you
to query the entire set of data recorded (not just the subset any
node would see) to find patterns or anomalies, or do a
post-mortem on a system failure.
Locally each
RTI Data Distribution Service publisher or subscriber keeps
samples in queues. To allow for local persistence a mirror of
these in-memory queues can be kept on disk. After a crash the
publishers and subscribers can then be created with the same
state as they had before the crash.
Centrally
you can run RTI Recorder that can be configured to subscribe to
interesting data (or all data for that matter) and write it to a
file system. This data can then be queried either in real-time or
during off-peak hours. As an aside RTI Data Distribution Service
and RTI Recorder have countless flight hours on mission critical
systems!
Connecting the
Distributed System to an Enterprise
Database
In some cases simply
having the data on a file system is not enough - the
full power of an enterprise database is needed. At RTI we have
solved this by having RTI Real-Time Connect for ODBC enabled
databases. Now you can use RTI Data Distribution Service to
implement a distributed database, or utilize a database for your
persistent storage needs. This allows you to use any tool or
query you are familiar with to work with a distributed set of
data. The possibilities are literally endless!
No matter what the use-case - recovery,
logging, post-mortem analysis or database integration -
persisting real-time distributed data is here to stay. The
challenge is persisting the data with minimal intrusiveness on
the real-time aspects of the system. At RTI we work hard to make
that happen.
href="http://feeds.wordpress.com/1.0/gocomments/rtidds.wordpress.com/163/">
alt="" border="0"
src="http://feeds.wordpress.com/1.0/comments/rtidds.wordpress.com/163/"
href="http://feeds.wordpress.com/1.0/godelicious/rtidds.wordpress.com/163/">
alt="" border="0"
src="http://feeds.wordpress.com/1.0/delicious/rtidds.wordpress.com/163/"
href="http://feeds.wordpress.com/1.0/gostumble/rtidds.wordpress.com/163/">
alt="" border="0"
src="http://feeds.wordpress.com/1.0/stumble/rtidds.wordpress.com/163/"
href="http://feeds.wordpress.com/1.0/godigg/rtidds.wordpress.com/163/">
alt="" border="0"
src="http://feeds.wordpress.com/1.0/digg/rtidds.wordpress.com/163/"
href="http://feeds.wordpress.com/1.0/goreddit/rtidds.wordpress.com/163/">
alt="" border="0"
src="http://feeds.wordpress.com/1.0/reddit/rtidds.wordpress.com/163/"
/>
src="http://stats.wordpress.com/b.gif?host=blogs.rti.com&blog=7350090&post=163&subd=rtidds&ref=&feed=1"
/>
