The Gateway to Algorithmic and Automated Trading

Software Review: ILNumerics - faster numbers

Published in Automated Trader Magazine Issue 27 Q4 2012

For the majority of Automated Trader readers, faster code is implicitly better code. When conducting research, anything that reduces the execution time of the often vast simulations or optimisations being undertaken is a 'good thing'. Even for those not at the cutting edge of HFT, the same applies to the execution time of code in live trading. Which is why in this issue's software review we're taking a quick look at the ILNumerics maths library.

One of the most frustrating juggling acts for developers of trading models is the trade off between ease of use and performance when it comes to choosing a programming medium. At one end of the spectrum are compiled languages such as C++, which are challenging to learn and require such things as declaration of all variables and types, but are compiled as machine code - resulting in slow creation time but fast execution time. At the other end of the spectrum are interpreted languages such as Visual Basic, which are relatively easy to learn, do not require the declaration of variables and types in advance, but have to be interpreted at run time - resulting in fast creation time but slow execution time.

Sitting somewhere in between these two extremes are languages such as C#, which are semi-compiled1, but can - depending on the exact circumstances - produce run time performance not far behind fully compiled languages. While most would probably accept that C# has less of a learning curve than C++, it is still demanding - and especially if one is attempting to express complex mathematical algorithms and has limited experience.

Changing demographic

This is particularly relevant right now due to a notable change in trader demographics currently underway. Especially over the last year, we have noticed a sharp uptick in the number of Automated Trader readers of a certain type. These are typically highly experienced manual traders who have found their existing trading methods are struggling to generate sufficient alpha in an environment where trading volumes in many major equity markets and their associated derivatives have declined substantially. As a result, they are looking for alternative opportunities in automated trading but lack the quantitative and programming skills to do this, which means they are looking for anything that flattens their learning curve.

This often results in them starting by using either a proprietary scripting language associated with a particular trading platform or something like MATLAB, R or Octave. But many of these traders are working in groups and are now considering implementing their ideas in applications with custom trading GUIs that can be shared across the group - a task for which C# is well suited. However, while there are plenty of maths libraries available for C#, the key question is how quickly existing trading models can be ported using these - therefore ease of use is paramount.

Syntax

Although it has many other more general applications, this ease of use is one of the stated objectives of ILNumerics, which claims to be the only mathematical programming library to combine the simplicity of a high level numerical scripting language (such as MATLAB, R or Octave) with the speed of compiled code.

Figure 1

Figure 1

It has to be said that ILNumerics gets off to a decent start on the simplicity front when it comes to its syntax, especially for those users who have done their model prototyping in something like MATLAB or Octave. While certainly not 100 percent identical, much of the syntax used is very similar or the same as MATLAB/Octave. ILNumerics provides a handy PDF that provides syntax comparisons with MATLAB for array handling, subarrays, cells, operators and functions. A quick glance at this reveals a raft of identical syntax for trigonometric, accumulative, algebraic, state and Fourier transform functions. In addition, quite a bit of the array handling syntax is also identical, such as horizontal and vertical concatenation, reshaping and replication.

Getting started

We installed ILNumerics using the quick start guide instructions on ilnumerics.net, which were very clear and worked well. However, it's worth noting that it is now also possible to install the library by using NuGet, which is a Visual Studio extension for installing and updating third party libraries in Visual Studio. This is well worthwhile, as it considerably simplifies the whole process of library management under Visual Studio, especially when updating existing libraries.

The quick start guide also includes essential directions on matters such as selection of the correct binaries to use for 32/64 bit project targets, plus the correct way to reference ILNumerics binaries and namespaces. It finishes with the code for a simple example of a console application that uses ILNumerics to solve a linear system of equations and display the result (see Figure 1).

Usability

As Aly Kassam mentions in his review (see below) ILNumerics itself is extremely accessible in terms of concepts and syntax, but the question marks for non-developers might relate to other issues instead. A non-developer member of the Wrecking Crew who nevertheless had some experience of using VBA and MATLAB kindly agreed to act as a guinea pig to test this hypothesis. He dutifully ploughed through several hours of C# and Visual Studio videos before having a stab at running some of the ILNumerics example projects (see below for more on these). This was then followed by an attempt at linking up ILNumerics with a third party graphics library for financial charting (candlesticks et al) in order to plot a set of historical data as an OHLC bar chart, with a linear regression line generated by ILNumerics superimposed.

The results were not quite as anticipated in that the mastery of Visual Studio and basic C# concepts didn't seem to present much of a problem at all. There was the odd minor hiccup, when building some of the ILNumerics examples, with remembering to reference the correct deployment platform, but in general the programming environment didn't represent a major hurdle. The second task of hooking up ILNumerics with the third party graphics library also wasn't an issue as regards Visual Studio, but it was a major roadblock as regards another potential issue that Aly raises in his review - the conversion of data types. For some reason, instead of also offering the common option of reading ASCII files, the graphics library would only accept price data inputs in MetaStock's binary data format. Our Crew member eventually managed to figure out a way around this by downloading Oleg Komarov's Metastockread MATLAB function to use as the most efficient way (from his perspective) of understanding the structure of the data required by the graphics library. While an experienced developer would have found a more direct solution faster, the rather convoluted process did serve to illustrate how much of an issue playing mix'n'match with third party libraries in Visual Studio can be for the tyro.

Figure 2

Figure 2

Examples

The help and class reference information provided with ILNumerics is comprehensive. The online help includes the source code for a number of simple example projects, but for those who purchase a commercial licence (as opposed to the alternative GPL licence) a complete set of twenty six example projects is provided that are ready to build and run. (Figure 2 above shows a simple example of a line graph, while Figure 3 below shows a more sophisticated example of a 3D graphic.) The help files also go beyond the basic explanation of functionality to include tips for optimising performance. One example of this relates to minimising memory usage, where lengthy loops are a common way that out of memory errors arise. The ILNumerics help files illustrate a very simple way of avoiding this issue; by enclosing the loop body with the 'using' block syntax mentioned in Aly's review. This ensures that the memory consumed is only that actually required for a single iteration of the loop. ILNumerics also supports in-place array operations, which enables the memory for an input parameter to be directly used for the output parameter of a function where possible. The help files demonstrate how to take advantage of this automatically through appropriate setting of the ILNumerics.Settings.AllowInArrayAssignments flag, which can radically boost performance.

Figure 3

Figure 3

In many ways, ILNumerics reminds me of MATLAB in its earliest incarnation - the only difference is the language used. MATLAB originally consisted of a set of C wrappers around the LAPACK linear algebra libraries, which gave those who didn't wish to engage directly with FORTRAN a means of doing so. In similar fashion, ILNumerics consists of a .NET wrapper around some highly optimised linear algebra libraries. However, in this case, the libraries concerned are both LAPACK and the Intel Maths Kernel Library (MKL).

Aly Kassam

Aly Kassam

In this edition of Automated Trader we welcome a new member to the Wrecking Crew. Unlike most members of the Crew (especially those from the buyside) who only appear under conditions of strict anonymity, Aly Kassam has graciously agreed to be identified. As can be seen from his biography, Aly's mixture of quantitative and programming skills, combined with his extensive experience clearly make him a valued addition to the Crew. For his personal take on ILNumerics, read on...

Optimisation

One of the most immediately obvious things about ILNumerics is that someone has clearly spent a lot of time thinking about the optimisation of these libraries, which is an important strength. The ILNumerics developer blog2 has an interesting and realistic performance comparison across C# (ILNumerics), FORTRAN, MATLAB and NumPy when evaluating a k-means clustering problem3, which provides an illustration of how effective this optimisation has been.

It is apparent that ILNumerics has spent a lot of time paddling around in the murky pools of garbage collection and memory (re)use. These are areas where many higher level languages that access LAPACK fall down with out-of-memory errors, even when plenty of memory is actually still available. The underlying problem is that LAPACK really needs contiguous memory and the errors arise because the memory has become fragmented.

Figure 4

figure 4

The way in which ILNumerics effectively forces you to write your code shows that considerable effort has been devoted to addressing these potential memory issues. For example, it takes advantage of the.NET 'using' construct (see Figure 4 below) which strongly scopes variables within its code block so that any variables created within the block are deleted when the block is exited. By compelling the use of this syntax when creating a function, ILNumerics is implicitly also compelling you to be far more efficient in terms of memory allocation.

In addition, a couple of further levels of memory control are available - such as the ability to explicitly clear variables in a function while that function is still running. Combining this good memory management practice with the highly efficient and state of the art LAPACK and Intel libraries delivers significant performance optimisation and extremely fast code execution.

MATLAB migration

The MATLAB-like syntax of ILNumerics is extremely straightforward and would not be difficult to learn - especially for those migrating from MATLAB itself. Common tasks, such as indexing into matrices to extract sub-matrices and other array manipulation activities are easy to do. There's also evidence of a clear intention to provide a migration path for MATLAB users. For example, ILNumerics has the ability to open .MAT files, as well as to save data from within ILNumerics as a .MAT file. (Though there are some limitations to this latter functionality as regards cell array variables.)

However, there are two significant obstacles to MATLAB finance users migrating to ILNumerics. The first is that while ILNumerics is itself easy to use, the environment in which one has to use it (Visual Studio) is not particularly intuitive to those who are not developers. The second is that although ILNumerics clearly bills itself as a maths library rather than a maths finance library, the finance-specific functionality it offers is still quite limited. While there is nothing to prevent one creating this functionality using the building blocks provided in ILNumerics, this is of course extra workload, which in some cases might also be rather challenging.

Who will use it?

The Visual Studio hurdle mentioned above also applies more generally to non-developers. Manual traders - especially those formerly more inclined to use spreadsheets - looking to up their game with a more quantitative approach will definitely need to either spend time learning or have training on the environment. While many of the tasks developers have to undertake in an environment such as Visual Studio are fairly formulaic and become second nature over time, they will still be unfamiliar to those who have previously only used VBA, MATLAB or a trading platform's proprietary scripting language. Chores such as adding references to other libraries and compiling for the right platform may not be intrinsically difficult, but they still have to be done.

At the other end of the spectrum, experienced quant programmers are unlikely to find ILNumerics particularly relevant because it doesn't cover all the bases they typically require. For example, as yet it only supports multivariate normal distribution and has only fairly simple statistical functions such as principal component analysis, means and standard deviation, but no facility for hypothesis testing, time series analysis, stochastic differential equations or nonlinear regression (though ridge regression is provided). So if you have a portfolio to which you wish to allocate weights and do some sort of portfolio optimisation then you don't really have the tools to do this in ILNumerics at present.

So who in the finance space might want to use ILNumerics? One possible category is those trader/programmers who have found VBA too restricting and have already migrated to using Visual Basic with Visual Studio. They will already have the necessary familiarity with the development environment and would find the ILNumerics syntax relatively easy to pick up. They would achieve a very appreciable performance boost, as they would no longer be constrained by Visual Basic's limitations as an interpreted language and would also benefit from ILNumerics' highly optimised access to LAPACK and MKL.

A further plus is that by operating in the .NET environment, these traders would also be able to take advantage of the various .NET trading APIs available, such as CQG and Trading Technologies. A further opportunity is the ability to connect with other complementary finance functionality. For example, combining QuantHouse QuantDEVELOPER and ILNumerics would deliver some interesting scope because the former has a lot of built-in specific finance functionality, while ILNumerics has the heavy lifting capabilities for large matrix problems.

Wish list

While ILNumerics is highly effective in covering the functionality that it provides, to build it out into a complete tool set for financial trading would currently involve adding various third party libraries. Financial considerations aside, this does raise the matter of library interoperability. This isn't necessarily a major problem in that these libraries should work together within the .NET environment, but there is the potential question of strong typing to consider. Each library will have its own set of objects and arrays and if these need to be available to other libraries, then some form of type conversion will be required.

How difficult or easy this is to achieve depends upon the individual libraries concerned, but in an ideal world it would be preferable to avoid the situation altogether by using additional native ILNumerics functionality instead. As mentioned earlier, it would be possible to build this from scratch using the building blocks ILNumerics already provides, but depending on the functionality required this could be quite time consuming and demanding.

So what would ILNumerics need to add to provide a complete financial trading toolkit? That obviously depends upon individual preferences, but the following would be my personal wish list:

• Optimisation: a decent optimisation suite would add a lot of value to this tool. It would also be nice to have some stochastic optimisation routines, such as a genetic algorithm.

• Nonlinear least squares and time series models: these have multiple uses in financial modelling across various disciplines.

• Statistics: the functions currently provided are the basics, but apart from the ability to handle non-normal distributions they could ideally be extended by adding features such as cointegration and VECM analysis, quasi-random number sampling from a variety of distributions, and a basic SDE solver.

Conclusion

ILNumerics is based upon several years of experience within a large open source community and appears to be a product close to the beginning of its commercial life, which as it continues to evolve would benefit from some of the additions mentioned above. However, its performance in terms of its existing functionality is robust and the considerable work that has been devoted to memory optimisation has clearly paid off.

Another useful aspect of ILNumerics not previously mentioned is graphics, which are already inherently built-in. This can otherwise be a major problem when programming in C# where you have to integrate a good graphics package to be able to see what's actually going on. ILNumerics also includes a number of plots that are useful for viewing research output, especially where it involves displaying very large matrix output (an area where some higher level languages struggle).

ILNumerics is also positioned interestingly when it comes to price - EUR999 for a perpetual commercial licence (a free GPLv3 licence version is also available) is not unreasonable for the functionality offered and is also significantly below the level of many other packages in this space.

Aly Kassam has a BSc in physics & astronomy, plus an MSc in nonlinear dynamics and chaos both from UCL, and a D.Phil from the Oxford University Computing Laboratory in Numerical Analysis. He worked for some years at Barclays Capital developing automated trading models in the fixed income world, and has also worked for several years in the finance group MathWorks - the company that produces MATLAB. He has recently started a quant modelling consultancy company - Quantitative Support Services - where he continues his meandering explorations through the always interesting and sometimes explosive labyrinth that is quantitative finance.