# Assessing the Risk and Return of Financial Trading Systems

## We apply large deviation theory to assess the probability that a trading system performs below or above a certain threshold. Our technique does not require that the distribution of the performance criterion obeys a closed-form equation, and can accept as input empirical distributions given under the form of frequency histograms obtained by backtesting or from prior use of the trading system. A nice property of the technique is that it can be easily automated and integrated into a trading platform. Furthermore, the approach is not limited to a single trading system but can be applied on portfolio of trading systems. By Nicolas NAVET and Ren SCHOTT

1 Introduction

Large deviation (LD) is a theory of rare events that is focused on the analysis of tails of probability distributions. Large deviation is classically used to study how random processes deviate from their expected value. If upper bounds on this quantity can be obtained through Chernov, Markov and Tchebychev inequalities, LD provides the exact rate of convergence, instead of an upper bound that is often not tight enough for real-world applications. LD has been a very active field of investigation over the last 10 years with numerous practical applications, for instance for evaluating performance of algorithms or telecommunication infrastructures. Another body of literature applies large deviation to risk analysis (see Ref. [7] for a recent and comprehensive survey), and this is actually the field of applications from which originates LD theory.

This paper belongs to this latter line of research and focuses on assessing the risk of financial trading rules. This work is aimed at giving answers to the practitioners, and thus the techniques provided can be applied with as few technical assumptions as possible. In particular, the approach developed here does not require closed-form distributions, and is able to deal with empirical distributions obtained from the backtesting step, or from experience gained using the trading rules. The interest of this approach with regard to Monte-Carlo simulations is three-fold. First, simulation is not well suited to estimate rare events because of the size of the sample that is needed to achieve reasonable error bounds. Second an analytical approach does not suffer the uncertainties of simulation (e.g., quality of the random number generators). Finally, this analysis can be integrated into a broader Value-at-Risk analysis.

2 Characterizing the Performance of a Trading System

A trading system is comprised of one or several trading rules that define entry and exit conditions, and decide the size of each position taken. A trading system can be seen as an algorithm that is implemented either as a computerized Automated Trading System (ATS) or executed, in a consistent manner, by a trader. The typical way to select trading systems is to evaluate a set of candidate systems on historical data and keep the best system or several top-scoring ones. This is the "backtesting" procedure, which is a feature nowadays available on almost any technical analysis software package.

The candidate systems can be selected by exploring the parameter space of a candidate trading system, for instance, a technical analysis trading rule or a system based on short term autocorrelations. More evolved schemes are not only concerned with the problem of setting the parameters but have also in addition the ability to induce the trading system itself. In particular, Genetic Programming (GP) has often been used for that purpose (see, for instance, Ref. [2, 6]).

It is to stress that backtesting is most often positively biased because of slippage and delays (in quotations and transmissions of orders) that are hard to model, and because of the data mining bias. Indeed, when trying a large number of systems, some are likely to have benefited, to a certain extent, of luck, for instance, by taking advantage of noise in the data. Thus, the performances achieved by backtesting should be artificially reduced taking all these parameters into account. The reader might refer to Ref. [4, 1] for good starting points on this subject.

In our context, the most important characteristic of a trading system P is the distribution P of its performance criterion over a chosen reference period1. We consider the most general case where the distribution of the performance criterion is given under the form of a discrete probability distribution directly obtained through the frequency histogram. However, it is of course required that the frequency histograms are meaningful from a statistical point of view, which implies that they have been obtained from a sufficiently large sample. In the following, our approach to assess risks through LD will be illustrated considering that the performance criterion is the profit/loss distribution, but it could be any other performance criterion of interest.

3 Risk Induced by a Trading System

Here we assess the risk of a given trading system using large deviation theory. Implied is the assumption that, roughly, the trading system will keep on performing as it did on historical data. This assumption that the market conditions are "homogeneous in time" is not questioned in this paper but can be revisited in future work by considering time-changing probability distributions (see Ref. [3]).

#### 3.1 Risk over a given time interval

The risk induced by trading system P is a function of n, the length of the time interval during which P is used. We assume that n is a multiple of T, the reference period of the performance criteria distribution because, in practice, it is usually meaningless to consider fractional reference periods. For instance, the meaning of considering one half of a day for an intraday trading system is dubious given that usually all positions are cleared at the market close to avoid overnight risks.

Let (Xn), n be the sequence of mutually independent, identically distributed random variables modeling the value of the performance criterion over successive period, and let the mean Mn = i=1nXi. Cramer's theorem (see Appendix 5 for details and notations), which is central in LD theory, enables us to state that P(Mn G) satisfies a rate deviation principle with rate-function I: P(Mn G) e-n inf xGI(x) where G is any subset of . In our case, G is the subset we want to assess the probability that the performance criterion belongs to. For instance, if we are interested in P(Mn k) with k E[Xi] then G = [k,+) and we estimate the decay rate of the right-hand tail of the distribution. From LD theory, we know that

(1)

If there is a closed form for the law of the Xn, or if the Xn form a finite Markov chain, it is possible to obtain an explicit expression for the rate function. In our case, where the law of Xi is given by a density histogram, this is not possible and a numerical method has to be employed. As I(x) is a supremum of affine functions, it is a convex function and it is enough to compute the point x* where I(x) reaches its minimum to obtain the asymptotic behavior

 (2)

x* is the point where the first derivative of I(x) with respect to t is equal to 0 (see equation 1). This point is reached for τ0 s.t. x k=-∞+pke0 = k=-∞+kpke0 which can be rewritten as k=-∞+(k - x)pke0 = 0. Let u = et and

 (3)

the problem consists in finding numerically u0 > 0 s.t. F(u0) = 0. This can be solved with Newton-like methods, which are available in any numerical or symbolical computation software.

#### 3.2 Applications

Let us consider a typical intraday trading system where positions are closed before the end of the trading day. The chosen performance metric is the profit/loss (P&L in K\$) and Fig. 1 shows its empirical distribution obtained over a sufficiently large timespan.

Risk induced by a trading system. A first information that can be derived is the risk induced by the trading system, which comes to evaluating the probability that the average return over n days is lower than a given threshold: P[Mn x]. Of particular significance is the risk of losing money which corresponds, below a certain loss threshold, to the risk of ruin. If the market conditions do not change over time, and if the expectation of the P/L is positive, we know that this risk decays exponentially and the higher the decay rate, the less risky is the corresponding trading system. This leads us to formulate the following definition.

Definition 1 The uncertainty of trading system SP to achieve a performance x over n time periods is (x,n) = P[Mn x∕n] e-n inf yx∕nI(y).

Definition 2 Trading system SP with performance objective x over n time periods is less uncertain than trading system Sp with return objective x over n time periods if (x,n) (x,n).

Considering our example, we evaluate the probabilities to achieve an average P&L lower than x for x {0,-0.5,-1}. For each x value, we replace x by its value in Equation 3 and obtain numerically u0 s.t. F(u0) = 0. The value of t s.t. Equation 1 is maximized is t0 = lnu0. Then, we compute I(x) over the interval G of interest (here (-∞,x]) and the infimum is the decay rate I(x*) we are looking for (see Equation 2). The upper bound on the probability that P[Mn < x] for x {0,-0.5,-1} is shown in Fig. 2. For instance, over 10 trading days, the probability to get a negative return is less than 0.42 while the probability to get less than -0.5K\$ and -1K\$ are respectively lower than 0.04 and 0.001.

Probability of profits above expectations. The same procedure provides us with a way to estimate the probability that the trading system outperforms a given threshold x with x E[Xi]. On the same example, we derived an upper bound on the probability that P[Mn x] with x {1,1.5,2}. The probability decays exponentially with rate 0.06 (x = 1), 0.28 (x = 1.5) and 0.64 (x = 2) and we obtain curves similar to the ones shown in Fig. 2.

Detecting when market conditions have changed. In an indirect manner, LD provides us with a way to detect that the market conditions have changed since the empirical distribution of the trading system's performance has been collected. Indeed, if a trading system experiences a performance that is way below what was foreseeable, if it achieves a performance level that has been evaluated as highly unlikely, this gives us evidence that the market conditions have changed, and thus suggests to us that our assumptions about the market should be re-examined. We believe this is important because markets's dynamics change over time and it is crucial to be able to detect when trading systems stop being efficient.

### 4 Risk induced by a Portfolio of Trading Systems

In practice, investors may prefer to employ a set of trading systems instead of a single system, because diversification provides additional profit possibilities, and also helps to limit slippage and reduce risks. If the probability distributions of the performance criteria of the trading systems are independent2, it is possible to assess the risk of a "portfolio" of trading systems in the same way as done in previously for a single system. More formally, this comes to evaluate

where the Xij is the sequence of random variables giving the performance of system j over time.

The probability distribution of the sum of two independent discrete random variables X and Y with probability distribution f and g is given by the convolution f ⋆ g. In practice, the most efficient way to compute a convolution is to use a Fast Fourier Transform (FFT) : f ⋆ g = FFT-1(FFT(x) FFT(y)). In our case, if Pj denotes the probability law of the sequence Xij, one has P1..m, = P1 P2 ⋆ ... ⋆ Pm = FFT-1( i=1mFFT(Pi)) and P( i=1nPi1..m, G) e-n inf xGI(x).

### 5 Conclusion and Perspectives

In this study, we apply large deviation theory to assess the risks of trading systems. Our approach is practical in the sense that and it can cope with empirical distributions given under as frequency histograms, can be easily automated and integrated into a existing trading platform and is able to consider the case of portfolios of several trading systems. The information provided by the analysis is twofold: first a precise estimation of the probability to perform below or above a certain threshold, but also a way to detect, with a chosen certainty level, that a trading system has stopped being efficient.

We assumed that market conditions do not change over time, precisely that a trading system will keep on performing as it did in the past. This assumption can be relaxed, for instance by considering that the performances are likely to decrease with the passing of time because of changing market conditions. Techniques developed in Ref. [3] in the context of dynamic random walks may provide us with the theoretical foundations to address the problem.

### References

[1]David R Aronson. Evidence-Based Technical Analysis: Applying the Scientific Method and Statistical Inference to Trading. John Wiley & Sons, 2006.

[2]S.-H. Chen, T.-W. Kuo, and K.-M. Hoi. Genetic programming and financial trading: How much about "what we know". In C. Zopounidis, M. Doumpos, and P. M. Pardalos, editors, Handbook of Financial Engineering. Springer, 2007. Forthcoming.

[3]N. Guillotin-Plantard and R. Schott. Dynamic random Walks: Theory and Applications. Elsevier, February 2006. ISBN-13: 978-0-444-52735-6.

[4]D. Jensen and P. Cohen. Multiple comparisons in induction algorithms. Mach. Learn., 38(3):309-338, 2000.

[5]J. T. Lewis and R. Russel. An introduction to large deviations for teletraffic engineers, 1997. available at url http://www.statslab.cam.ac.uk/%7Errw1/ld/LD-tutorial.ps.

[6]N. Navet. Genetic programming for financial trading: a tutorial. Tutorial at the 5th International Conference on Computational Intelligence in Economics and Finance (CIEF2006), 2006. Slides available at url http://www.loria.fr/~nnavet.

[7]H. Pham. Some applications and methods of large deviations in finance, February 2007. available at url http://arxiv.org/abs/math.PR/0702473.