Palo Alto, California - Hazelcast, the open source in-memory data grid (IMDG), has announced the 0.5 release of Hazelcast Jet - an application embeddable, distributed computing platform for processing of big data sets. New functionality in Hazelcast Jet 0.5 includes the Pipeline API for general purpose programming of batch and stream processing, and fault tolerance using snapshotting with the integrated Hazelcast IMDG.
The overall focus of the latest release is to increase developer productivity via a simple and intuitive API. Jet is a single library with no dependencies which can be easily embedded and deployed, removing the need for multiple systems. Typical application use cases include online trades, sensor updates in IoT architectures, real-time fraud detection, system log events, in-store e-commerce systems and social media platforms.
The Pipeline API is the primary programming interface of Hazelcast Jet for batch and stream processing, making it more appealing to a wider Java audience. This is a major enhancement to the Hazelcast Jet low-level Core API which uses directed acyclic graphs (DAG) to model data flow - allowing detailed DAG assembly of processing jobs. The new Pipeline API is easier to use and provides developers with tools to compose batch computations from building blocks such as filters, aggregators and joiners. The Java 8 Stream API is also available in Hazelcast Jet 0.5, a well-known and popular API in the Java community which supports functional-style operations on streams of elements. The key point is that ANY Java developer will find the new Pipeline API familiar and productive.
Also new is fault tolerance using distributed in-memory snapshots - in Hazelcast Jet 0.5 snapshots are distributed across the cluster and held in multiple replicas to provide redundancy. Jet is now able to tolerate multiple faults such as node failure, network partition or job execution failure. Snapshots are periodically created and backed up. If there is a node failure Jet uses the latest state snapshot and automatically restarts all jobs that contain the failed node as a job participant. No additional infrastructure, such as distributed file system or external snapshot storage, is necessary to ensure Hazelcast Jet is fault tolerant out of the box.