As the competition to produce and quickly deploy profitable trading models continues to increase, many participants are starting to pay more attention to refining their model development process. In the second of three excerpts from their book "Quality MoneyManagement", Andrew Kumiega and Benjamin Van Vliet explain how research methods for quantitative strategies fit into the overall development process.

Researching Quantitative Methods from "Quality Money Management" by Andrew Kumiega and Benjamin Van Vliet

CHAPTER 9: Research Quantitative Methods

Research is the driving sector of the financial markets. Those who cannot reliably do market research, in any of its forms, or evaluate the research of others will find themselves on the sidelines in an industry that increasingly depends on innovations based on good data and produced by sound research.¹ While Stages 1 and 2 of K|V can be thought of as an empirical research methodology, concerned with the process of research, they are together better thought of as a process to manage ideas and combine innovations into working systems. This step, K|V 1.2, addresses the process of research most specifically. Each encounter with the previous step, K|V 1.1, should have defined a clear goal for the research step.

We recommend that, at the start of each loop through the research step, the product team defines the problem by restating the research goal as a question. For example, “ what is the best way to price options? ” This will help form well-defined boundaries for research. In general:

9.1. Benchmarking Quantitative Methods

Benchmarking is a comparative process. To be specific, over the course of this step, researchers compare and benchmark the following:

1. Trade selection algorithms
   1. Valuation and forecasting calculations
   2. Optimization routines
   3. Entry characteristics
      1. Signal definition (position rationale)
      2. Quantity
      3. Order type
      4. Entry price
   4. Exit characteristics
      1. Profit targets
      2. Trailing stops
      3. Time till exit
      4. Rebalancing exit
2. Trade execution and trade cost analysis calculations
3. Cash management procedures
4. Risk algorithms
   1. Risk calculations
   2. Hedging algorithms
   3. Drawdown procedures
   4. Cash and borrowing procedures
   5. Credit risk exposures
5. Performance monitoring and reporting
   1. Performance metrics
   2. Simple metric calculations
   3. Shutdown triggers.

The drive to uncover best practices will reveal what really works, including the conditions under which it works. Successful teams work from the general to the specific, understanding the entire strategy prior to modeling the pieces. Most complex trades can be broken down into simple concepts that should be understood before research starts.

In cases where many practices exist, researchers must be able to recognize the best practice, the one that will, either on its own, in collaboration with other methods, or through extension of the theory with proprietary calculations, produce a better, more stable trading/investment system. According to Robert C. Camp the best practice is one where: ²

• Performance is clearly superior. For example, a comparison of several forecasting methods may indicate that one clearly proves better than others.
• The quantified opportunity is large. For example, one strategy for exiting positions may prove to be more profitable than others.
• Endorsed by expert judgment. For example, financial engineers, either internal or external, may widely agree that one method is superior.
• The same practice recurs. For example, a survey may find that one method is used by all firms.
• Leadership position has been attained. For example, one execution algorithm may prove time and time again to provide superior performance relative to a VWAP, arrival price, or implementation shortfall, or pretrade, benchmark.

The research step permits the product team to survey all the relevant mathematical and logical models. The goal of the research process is to speed the path to the design or application of the best practice algorithms. Any quantitative method will fall into one of four categories:

1. Usable in a production system.
2. Usable only in the absence of a better model.
3. Totally unusable.
4. Model in inventory. Maybe not appropriate for the problem at hand, but worthy of further research for other, later problems.

Benchmarking trading/investment strategy processes is a proactive search for superior performance that includes gaining a concrete understanding of the competition as well as trying out new ideas and proven practices and technology. Process benchmarking should be approached on the basis of investigating industry practices first. ³ Capability maturity advances when proven best practices are incorporated in trade selection, order management algorithms, and risk management algorithms.

9.2. STEP 2, LOOP 1: Research Similar/ Competing Systems

The first loop of research begins to refine requirements into defined inputs the trading system will receive, actions taken by it, and defined outputs it will produce. Rarely, though, do firms dream up completely new trading/investment strategies. More often, they build on ideas of the past or of others, adding a twist or two here or there to enhance performance; many trading ideas are simply copies of ones that were successful in the past. The goal of a new system may in the first year be to simply imitate a competitor or index benchmark, matching its performance. Down the road, in successive years, the goal could be to improve on the results and outperform the benchmark. At some firms, the goal is to make each new trading system or product team cash flow positive within some timeframe. After that, algorithms are refined. Finally, the steady-state goal may be to consistently perform in the top 10% of the peer group.

The SEC ’s general push toward increased disclosure of holdings by mutual funds (and inevitably hedge funds) allows one fund manager to mimic another. Such funds are called copycats. In fact, research by Mary Margaret Frank et al. suggests that copycatters can even outperform the copycattees. The study found that hypothetical funds copycatting the 100 largest equity mutual funds in the United States, updating the portfolios semiannually based on publicly available information, generated returns “ statistically indistinguishable, and possibly higher ” than the returns of the copycatted funds. The disadvantage is that copycats ’ access to information is delayed. The advantage is that copycats pay lower research costs.⁴ Information can be gleaned by:

• Observing positions that the competitions ’ trading/investment system takes.
• Gathering information about the strategies and technologies the competitor uses.
• Searching for quantitative methods that replicate the competitor ’ s performance.
• Comparing the performance of a copycat system to the performance of the competitor’s system.

Information will come from within the organization and external to it. Traders, quants, programmers, and even human resource and sales and marketing professionals deal on a daily basis with vendors, news media, other traders, professional associations, patent attorneys, and academics and will hear about impending competitive pressures. Mutual funds publicly disclose their portfolio holdings periodically, revealing which stocks or bonds the fund manager believes are undervalued. That is useful information to competitors. Also, public pensions regularly hold meetings where money managers compete for investment dollars. Fund overseers usually grill managers about strategies and expenses at these meetings, which are often by law open to the public. Attending these meetings can reveal a lot about what your competition is doing and how they sell.

The Vanguard Group stopped reporting information about the net cash flows into its funds because third parties were apparently using this information to trade ahead of Vanguard funds, thereby raising Vanguard ’s effective cost of executing stock transactions.⁵

Though not as simple, information can sometimes be had on the Internet, at trade shows and conferences, and by interviewing industry experts, even by talking to or interviewing the competitions ’ customers and employees (for some firms this is part of their routine), and by reading marketing materials and prospectuses. Sometimes vendors will brag about how your competitors use their system. Whatever the case, information can be filed on paper or in a database so that when new information comes along, it can be quickly linked to similar information that had previously been found.

9.2.1. Reverse Engineering

Reverse engineering is the process of capturing the specifications of existing trading/ investment systems and then using the information as a foundation for designing a new system. The new design could be a replica of the original or an entirely new adaptation of its underlying strategy. Reverse engineering can be viewed as the process of analyzing a system to:

• Identify the system ’s components and their interrelationships.
• Create prototype models of the system in another form or a higher level of abstraction.
• Create the technological implementation of that system.

Reverse engineering includes any activity a product team may engage in to determine how a trading/investment system works, or to understand the strategies and technologies that make it run. Given, for example, a competitor ’s returns, the product teams perform regression and principal component analysis to understand the system. Reverse engineering is a systematic approach for analyzing the design of existing trading/investment systems.

Most trading and investment firms are highly secretive, protecting proprietary methods. Even identifying the best competitors can be difficult. Nonetheless, information should be pursued. (Although it is tempting to use illegal or unethical ways of gaining an advantage, quality and its customer-focus prohibit such short-term thinking. 6 ) Consider the following ethical uses involved in reverse engineering:

• Do not reverse engineer components of a trading system if a licensing contract prohibits it.
• Remember to perform reverse engineering using only information that is not proprietary to the firm you are scoping.

If you intend to perform reverse engineering, be sure that:

• The firm does not have access to proprietary information.
• The firm does not obtain information from disgruntled employees who work or very recently worked for the competing firm and/or who are under contractual obligation to refrain from releasing proprietary information. (Employees at firms that use quality money management should not be disgruntled!)
• The firm maintains complete documentation of each component it reverse engineers so there is a record that will stand as proof in court that it performed its reverse engineering lawfully.

Reverse engineering initiates the redesign process, wherein a product is observed, tracked, analyzed, and tested in terms of its performance characteristics. The intent of the reverse engineering process is to fully understand and model the current instance of a trading strategy in order to compress the new product development time.

9.3. STEP 2, LOOP 2: Research New Methods

Because the best practice is in fact unknown (and theoretically unknowable), benchmarking is really a misnomer. We are not simply copying a competitor ’s system deemed to be “ best practice ” and implementing it in-house. Product teams use a benchmark as a reference point against which to compare its own proprietary calculations. Without a reference point, the team cannot know if its calculations are in the top rank of its peer group or not.

Over the course of their research, financial engineers investigate mathematical models and logical constructs according to best practices, taking notes, writing up, and critiquing what they find, comparing alternative methods in an attempt to best describe the interaction between data and the desired outcome. Successful researchers calibrate their methods by first applying them using known inputs and outputs and documenting the results before applying the methods to unknown inputs. In practice, the accumulation of evidence for or against any particular quantitative method involves a planned research design for the collection of empirical data. The best method is algorithm benchmarking, which will increase the probability of success.

We recommend the team keeps research independent of data and testing (our methodology forces this separation). This will help avoid, among other things, spurious correlations.

Best practice also requires that teams separate the process of specifying the formulas from building the prototypes (in the following step, K|V 1.3). In this step, team members should document the logic behind the trading/investment system in an unambiguous statement of how prototypes will calculate the results. Clarify a quantitative method first; replicate second.

Researching quantitative methods means doing some leg work. Team members assigned this task, namely, the financial engineers, will either derive some proprietary algorithms or more likely go to the library and Internet, find white papers, published papers, working papers, and books, and assemble resources and articles. All of the resources, information, and articles should be organized into folders and the folders cataloged by title, concept, and author. (Building an organized library of quantitative methods is a key to the long-term success of the firm.)

We recommend that researchers standardize the formats of equations. Beta in one paper may not mean the same thing as beta in another. Some notation conversion must take place to standardize all the equations into a common format. Also, calculations should be labeled with journal names and page numbers. These documents should each have a cover sheet and be placed in bibliographical order.

We recommend product teams do a complete survey and identify all appropriate methods in order to exhaust all of the information that may exist in the research data, including intercorrelations and potentially all of the relevant projections. This process will start with empirical research, followed by an attempt to improve results with additional, theoretical research. (Research is often an imitation, or replication or corroboration, of the research of others, done with little understanding of what lies beneath the surface or consideration to the process of discovery. The search for better, though not perfect, performance must be the ultimate objective of research.)

We know a major trading firm that uses a model for a common tradable instrument that is known to be incorrect. We figure if they had performed a full survey of the body of knowledge, they would be making even more money. One day the market will punish them.

All equations and algorithms used in trading/investment will inevitably fall into one of four categories:

1. The model will be used successfully in a production environment.
2. It will be used in desperation for lack of or inability to implement a better model that will invariably cause losses and additional cost of rework sometime down the road.
3. It will be totally unusable and discarded.
It will be placed into an inventory of models for future research.

9.4. STEP 2, LOOP 3: Consolidate Trading/ Investment System Design

The final loop of system specification is to assemble the component quantified methods and business rules and fully define the outputs that the trading system will produce. Also in this final pass over Step 2, team members must agree on what information team members and management will require in the form of GUI and reports to oversee the fully operating trading system, that is, in Stage 4, including what is the purpose of each report and what will be the appropriate level of detail.

The final loop through this step should also define what the appropriate risk tools will be. A discussion of these tools should then be added to the description. This will prepare us for Stage 4, Step 1—Monitor Portfolio and Statistics as well as Stage 4, Step 2— Document Profit and Loss Attribution.

Very few firms pay attention to defining what will constitute abnormal or nonstable performance relative to a benchmark or peer group once they implement a system. Those that do will have a competitive advantage once certain indicators lose relevance.

9.5. Summary

The world of trading and investment has changed from one of place and time to one of algorithms and computer speed. Good research is no longer a luxury; it is a necessity. Simply adding more analysts to produce more standard research does not work. It takes too long. Ideas must be generated and vetted quickly. Nevertheless, clean, well-documented trading/investment rules lead to better automation of trading processes and scalable systems. To speed up research the firm should consider building or buying automated research and/or strategy evaluation tools. Research is applied, not theoretical, and our spiral methodology followed by a gate meeting should focus researchers on business deliverables instead of elegant equations. The advantages of doing research properly are many and cannot be overstated. Good research forms the foundation of a scalable trading/ investment business.

9.5.1. Best Practices

• Before each loop through this research step, the team should form a hypothesis as to what they expect the research to accomplish. Be sure to have well-defined boundaries.
• Work from the general to the specific.
• Keep research independent of data and testing (our methodology forces this separation). Likewise, separate the process of specifying the formulas from building the prototypes.
• Keep records of research as you go. Translate notation into a common format as you proceed.
• Identify several potential solutions and evaluate them, as well as their respective sources carefully, and review your work weekly.

You can buy "Quality Money Management" at a 10% discount in the Automated Trader Bookshop

• Copyright © Algorithmic Media Ltd 2009

• Popular Searches
• Privacy Policy
• Sitemap
• Designed by Vibrant Web Design Brighton
• Hosted by Enhancion