The Gateway to Algorithmic and Automated Trading

The kerf time series database system

Published in Automated Trader Magazine Issue 39 Q2 2016

Today's automated traders have many difficult data and infrastructure problems that revolve around large and complex time series. - Increasingly, they turn to specialized tools to address these challenges.

Capturing live data can be a challenge, though most sophisticated traders have some solution to this problem. Research on the captured data can be a real issue. Standard database technology does not generally scale well or provide the types of operations which are of interest to quantitative researchers. Integrating custom data stores with programming languages such as Python used in quantitative research has challenges.

There are a few systems on the market which assist traders with these kinds of problems. The two best known are Onetick and Kx Systems. A new entry in the market is Kerf. Kerf is an old English word meaning the width of a saw cut; the creators clearly hope it is a name which represents relentless and precise sawing through data.

Kerf is a timeseries database with an integrated vector programming language. Kerf Software was founded by Kevin Lawler and Scott Locklin in Berkeley, California in 2015. Kevin Lawler is a veteran programming language designer, author of the open source Kona language as well as of the iOS spreadsheet software Permanent. Scott Locklin is active in the array programming community and has been working as a data scientist and quant.

Kerf, like Kx Systems' kdb database, is related to the APL family of array-oriented languages. APL languages have powerful primitives for data analysis; the core operators of APL can be simply combined to create database queries, and vector-wise operations can provide very high performance.

While most APLs emphasize terseness at the expense of readability, Kerf emphasizes readability and provides more familiar landmarks to programmers who are used to conventional programming languages. One of the surprises that Kerf bestows on its users is its JSON-style notation for specifying data. Maps (hashes in Python) have the same notation as JSON. Equality is set with the ':' character rather than '=', which in turn is mostly used for comparison.

The remainder of this article is only available to Paid Subscribers

Click here to purchase a subscription to Automated Trader

  • Copyright © Automated Trader Ltd 2018 - Strategies | Compliance | Technology

click here to return to the top of the page