• 
• 
• 

• Performance
• Services
• Company
• Learn About
• My Attain

Attain's Semi-Annual 'Top Fifteen System Rankings'

January 9, 2006

 

dried on last week's annual review newsletter than concerned developers, clients, and other weekly readers called or emailed in asking: "what we were thinking listing top "performers" for the year using only 2005 percentage return as the measuring bar."

"If I've learned anything reading this newsletter", one subscriber wrote; "it is that there is much more to performance than simple annual return figures". Well - we couldn't have said it better ourselves - as there is no doubt much, much more to performance than mere annual numbers. And along those lines, this week's newsletter highlights Attain's real performance rankings.

We release our rankings twice a year to give investors a concise summary of the systems we believe have the best chance for success moving forward. We are not content to merely show you the best performing systems this year or a list of the top performers of all time, however; as the best system for one investor may be anything but best for a second investor. For this reason, our rankings have developed over the years into a comprehensive tool which ranks systems across 8 different statistical categories.

It seems no matter how hard system developers and brokers try and tell you there is no "holy grail" system - the harder the investing public searches for one. Many investors are in search of a "holy grail" system which has low risk, high average returns, a multi-year track record, and of course - impressive recent gains, and this eternal search for trading's "holy grail" leads investors to quickly weed through hundreds of trading systems by asking every developer, broker, or investor knowledgeable about trading systems to show them their BEST system. But what are you really asking when putting this question to these "experts"? What is BEST supposed to mean, anyway? Best this month, this year? Best for all time? Best risk adjusted return? Best in terms of lowest Drawdowns?

Aren't we being short sighted when asking to see the BEST system in one category or another? The real question should probably be more along the lines of: "What system is consistently among the top systems across all of the different statistical measures important to this type of investment?"

This is exactly what our rankings have been designed to do: See which trading systems are the BEST in each of 8 categories, then see which systems are consistently among those on each list - and therefore the BEST overall.

We begin by looking at which systems are the BEST by last year's (2005) annual return. This is unfortunately the measure most investors use to determine what system is best for them, and the reason the year's hot system is usually regarded as the BEST system. The downside to this analysis, of course, is that it ignores risk. A high return is nice, but at what cost. The BEST performers so far this year are:

THE FOLLOWING TABLES SHOW PERFORMANCE STATISTICS OF HYPOTHETICAL MODEL ACCOUNTS COMPILED USING ACTUAL CLIENT BUY AND SELL PRICES. ALL NUMBERS ARE INCLUSIVE OF COMMISSIONS AND THE COST OF THE SYSTEM. PLEASE SEE IMPORTANT RISK DISCLAIMER BELOW.

While Compass and Axiom eMD appear to be very much the BEST this year, a simple change to looking at total return over the life of the system quickly inserts two other system into the top 5 list - R-Mesa 5 and Tzar eRL (despite losses of -50% in '05 for R-Mesa). Mesa Bonds and Fusion also drop out of the top 5 altogether, either due to losing years before '05 or not enough history to build up a significant track record. The BEST systems by Total Return have been the following:

Its easy to play devil's advocate when looking at the total return table and say how it unfairly treats newer systems. It admittedly takes a while to build up significant total return numbers, and for that reason looking at the average annualized rate of return (ROR) makes sense. This measure is more of a "what to expect" than a "what has happened" measure. It is interesting to note that R-Mesa is on this list despite a big losing year in 2005. Compass was in the same situation last year - and we expect a similar comeback from R-Mesa in 2006. The BEST systems by Average Annualized ROR are:

But what if we think of BEST not as the one that surpasses all other, but rather look for that system which is most suitable. The question in that case should not be, "What is your BEST system?" The question should be: "What is MY BEST system?", or in a more grammatically correct form: "What is the best system for me?"

To find what system is the BEST for you, a little soul searching is required. Are you interested in the absolute highest return? Lowest drawdown? Best mixture of the two, perhaps? Or perhaps you think the best system is the one which has been around the longest. There is surely something to be said for longevity. You will quickly find that different systems head many of these lists, showing that the BEST system is an elusive target indeed.

To begin to filter things down, we must incorporate the riskiness of each system. Many investors look at Drawdown to get a feeling of the risk involved. But concentrating solely on drawdown is just as bad as looking only at return. For starters, a system could have a very low drawdown because it has only been trading for a few months (note the Seasonal ST systems only started trading in October of '05) The BEST system for 'lowest' maximum drawdown have been:

But as nice as it is too see a low drawdown, low risk doesn't really help if there is also no return. We can always invest in treasury bills if we want zero risk. The next logical step, therefore, is to evaluate which systems have the BEST return per unit of risk. . This is accomplished through the use of several risk adjusted ratios. The first of these is the Sharpe ratio, which measures returns divided by risk (as measured by the standard deviation of returns, or volatility). The formula actually uses the amount of return over the risk free rate. Attain uses 2% as the risk free rate of return in its calculations. The systems with the BEST Sharpe ratios have been:

One of the problems with using the Sharpe ratio is that it punishes trading systems for having a high upside volatility profile. For example, the Compass system had a one month gain of +74% in 2002, which caused the volatility reading for the system to skyrocket. But it can be argued that upside volatility is of no concern, as that means large positive monthly gains in the distribution of returns. Does it mean an investment is more risky if it has a huge monthly GAIN. We think a huge monthly loss sure has to do more with risk. There is a risk measure which eliminates the upside volatility skew from the Sharpe ratio by using the volatility of negative returns only. This measure is called the Sortino ratio. The BEST systems by Sortino ratio have been:

The Sharpe and Sortino ratios have a flaw, however, in that they view the volatility of returns as the main ingredient of risk. This speaks nothing of what sort of drawdown had to be encountered to get the return. As many trading system investors can attest to, it is the drawdown period which represent the most risky part of the investment, not the volatility of returns. The Sterling ratio measures returns divided by risk (as measured by drawdown). The BEST systems by Sterling Ratio have been:

One last piece if information it is important to take into consideration is the length of track record. The above tables have looked at systems with at least six months of actual trading data, but measures such as the Sharpe ratio are usually computed on at least 3 years of data. The shorter the length of a track period, the greater the margin of error in the statistics. Thus a system such as Bounce eRL, which looks very nice atop many of the BEST tables above, could have a very large margin of error given its relatively short 12 month track record. The BEST systems for length of track record are:

So what system is the best overall? It again depends on what you are looking for, but the overall picture does have some clues. Our rankings define the BEST system mathematically by using a simple formula for scoring systems based on their ranking among all 42 systems we track with actual customer fills. For each of the above eight categories, we ranked each system 1 through 42, with a BEST in category equaling 1 point on down to a worst ranking of 42. The sum of all scores was computed to get the following standings for the Top Fifteen BEST systems at Attain: