• 
• 
• 

• Performance
• Services
• Company
• Learn About
• My Attain

Attain's Top Fifteen ! (What's your BEST system?)

January 10, 2005

 

There's some mysterious force that draws humans to make lists at the end of the year (the best movies, the best cars, the best dressed, the best song, the best artist, and so on), and before 2004 grows too small to notice in the rear view mirror, we wanted to share with readers of this newsletter the BEST systems at Attain through the end of 2004.

But how do you calculate the BEST Systems? It is a timeless question, asked by novice investors and professionals alike. It seems no matter how hard system developers and brokers try and tell you there is no "holy grail" system - the harder the public searches for one.

This eternal search for trading's "holy grail" leads investors to quickly weed through hundreds of trading systems by asking every developer, broker, or investor knowledgeable about trading systems to show them their BEST system. But what are you really asking when putting this question to these "experts"? What is BEST supposed to mean, anyway? Best this month, this year? Best for all time? Best risk adjusted return? Best in terms of lowest Drawdowns?

The American Heritage dictionary defines best as:

1. Surpassing all others in excellence, achievement, or quality; most excellent: the best performer; the best grade of ore.

2. Most satisfactory, suitable, or useful; most desirable: the best solution; the best time for planting.

Definition 1 would have us looking for the system whose performance surpasses all others. This is unfortunately the measure most investors use to determine what system is best for them, and the reason the year's hot system is usually regarded as the BEST system. The downside to this analysis, of course, is that it ignores risk. A high return is nice, but at what cost. The BEST performers in 2004 were:

While Blue Wave Zones appears heads and shoulders above the rest, a simple change to looking at total return over the life of the system quickly puts two other system atop the BEST list - Compass (despite losses of -19.4% in '04) and R-Mesa 5. BWT Zones drops down to the third slot. The BEST systems by Total Return have been the following:

Its easy to play devil's advocate when looking at the total return table and say how it unfairly treats newer systems. It admittedly takes a while to build up significant total return numbers, and for that reason looking at the average annualized rate of return (ROR) makes sense. This measure is more of a "what to expect" than a "what has happened" measure. The BEST systems by Avg. Annualized ROR are:

But what if we use definition two above, and look for that system which is most suitable. The question in that case should not be, "What is your BEST system?" The question should be: "What is MY BEST system?", or in a more grammatically correct form: "What is the best system for me?" or "Which one is the most suitable for my needs."

To find what system is the BEST for you, a little soul searching is required. Are you interested in the absolute highest return? Lowest drawdown? Best mixture of the two, perhaps? Or perhaps you think the best system is the one which has been around the longest. There is surely something to be said for longevity. You will quickly find that different systems head many of these lists, showing that the BEST system is an elusive target indeed.

To begin to filter things down, we must incorporate the riskiness of each system. Many investors look at Drawdown to get a feeling of the risk involved. But concentrating solely on drawdown is just as bad as looking only at return. For starters, a system could have a very low drawdown because it has only been trading for a few months (note Axiom eMD only started trading in Oct. of '04) The BEST system for 'lowest' maximum drawdown have been:

The next logical step is to evaluate which systems have the BEST return per unit of risk. This is accomplished through the use of several risk adjusted ratios. The first of these is the Sharpe ratio, which measures returns divided by risk (as measured by the standard deviation of returns, or volatility). The formula actually uses the amount of return over the risk free rate. Attain uses 2% as the risk free rate of return in its calculations. The systems with the BEST Sharpe ratios have been:

One of the problems with using the Sharpe ratio is that it punishes trading systems for having a high upside volatility profile. It can be argued that upside volatility is of no concern, as that means large positive monthly gains in the distribution of returns. Does it mean an investment is more risky if it has a huge monthly GAIN. We don't think so. There is a risk measure which eliminates the upside volatility skew from the Sharpe ratio by using the volatility of negative returns only. This measure is called the Sortino ratio. The BEST systems by Sortino ratio have been:

The Sharpe and Sortino ratios have a flaw, however, in that they view the volatility of returns as the main ingredient of risk. This speaks nothing of what sort of drawdown had to be encountered to get the return. As many trading system investors can attest to, it is the drawdown period which represent the most risky part of the investment, not the volatility of returns. The Sterling ratio measures returns divided by risk (as measured by drawdown). The BEST systems by Sterling Ratio have been:

One last piece if information it is important to take into consideration is the length of track record. The above tables have looked at systems with at least three months of data, but measures such as the Sharpe ratio are usually computed on at least 3 years of data. The shorter the length of a track period, the greater the margin of error in the statistics. Thus a system such as Axiom eMD, which looks very nice atop many of the BEST tables above, could have a very large margin of error given its relatively short three month track record. The BEST systems for length of track record are:

So what system is the best overall? It again depends on what you are looking for, but the overall picture does have some clues. We attempted to take a stab at defining the BEST system mathematically, and came up with a simple formula for scoring systems based on their ranking among all 35 systems we track with actual customer fills. For each of the above eight categories, we ranked each system 1 through 35, with a BEST in category equaling 1 point on down to a worst ranking of 35 . The sum of all scores was computed to get the following standings for the Top Fifteen BEST systems at Attain: