How to "Guarantee" you're Not trading a Curve-Fit System.
May 1, 2006
The regulators hate the word guarantee, and there are definitely no guarantees as far as profits or losses are concerned in investing. That's for sure. But an investor can guarantee certain things. They can guarantee they are working with an NFA member firm and broker by checking the National Futures Association website, for example.
With a little bit of ingenuity and creativity, investors can guarantee they are not trading a curve fit system, as well. The secret - get on a different curve! Let us go back to basics for a second, however, and answer the questions of 1. What is curve fitting? and 2. Why should you be worried about it?
To the first question, one of the greatest appeals and advantages of mechanical trading systems is the ability to evaluate their historical performance by "backtesting" the strategies on historical price data. While we may have just a handful of months of actual performance data available, computers and backadjusted data make it possible to see what a system "would have done" going back years and years.
The problem, of course, is that the system has been designed on this very same data. Whether intentional or not, because systems are designed on past data, they are often the victims of what we call "curve fitting", making the ability to backtest results one of the biggest disadvantages of trading systems also. Because the future may look nothing like the past in a particular market - the "fitting" of parameters onto the past "curve" of data may cause big problems on the future data curve, causing the system to be out of phase and potentially causing investors losses. .
The easiest way to understand "curve fitting" is through a simple example. Imagine a system that buys or sells Soybean futures on a breakout above or below the market high or low for the past X number of days. When testing the system on the past data, the testing may show $5,000 in profits when using a 10 day high/low, $10,000 in profits when using a 20 day high/low, and $20,000 when using a 30 day high/low.
If you were the developer, which value would you use in designing the system, 10, 20, or 30? I would guess most people would use the 30 value, as it gives the highest profit. Now a developer will look at more than just profit, and test for lowest drawdown or most winning months, for example; but whatever your goal for the system, it is human nature to design a system whose parameters produce results as close as possible to those desired. The problem is, just because one parameter worked on the past data does not mean it will work on the future, unknown data.
Creating a "Curve-Fit Free" Trading System:
We noted above that the secret to creating a non curve -fit system is to change the curve. What does this mean? This is truly a simple idea, but as is often the case, the simplest ideas are usually the best ones. Changing the curve simply means changing the data the system is operating on, and changing the market you trade it on.
If a system was curve fit, whether intentionally or not, because it was tested and is now run on the same data - let's use Soybeans as an example; then one way to make sure it's not curve fit is to trade the system on a completely different market, like Crude Oil, instead of Soybeans.
There is simply no way for a system whose parameters were fit to the Soybean data curve to also be fit to the Crude Oil data curve. They are two completely different markets, in different sectors, reacting to different supply and demand dynamics.
One issue comes to light pretty quickly when actually putting this theory into practice, however; and that has to do with a system's dollar based stops and profit targets, if they have any. It's a lot easier to hit a $250 stop in the S&P market than it is in Soybeans, for example, with the market only having to move 1 point in the S&Ps, but 5 points in the Soybeans.
The natural inclination is to look at the dollar based stops being too close in the new market as a problem, and switch the stop level to be more in line with how the new market moves. But doing that is curve fitting again, by making the logic make sense on the data you intend to trade it on. It is extremely logical and perhaps even the right thing to do, but to insure you are not trading a curve-fit system - you must leave the stop alone and trade the system "as-is" on the new market.
The end result of this dollar based stop issue is that it is better to look for systems with dynamic stop and target levels which are percentage or indicator driven, and not based on set dollar or point amounts specific to a certain market. With dynamic indicators, a system can be applied to any market without worry of the stop being hit on the next tick or a target within the bid/ask spread of a new market, and so on.
Because we endeavor to trade a system "as-is" on a new market, we feel it's best to keep the developer out of the picture as much as possible when making a switch to a new market. The list of systems below contains all systems Attain implemented on different markets without the developer's prior knowledge. That's not likely to please many developers, but to truly know that you are curve-fit free, the developer needs to be completely out of the picture, his or her hands completely off the system. The client has to pay the purchase or lease price for the system, of course, but they can choose which market they wish to trade it on.
| Original System | Non Curve-Fit Variant |
|---|---|
Compass SP | Compass eRL |
Delphi eRL | Delphi F/X GBP, Delphi F/X EUR |
RC Success ES | RC Success eRL |
Tzar ES, eRL, NQ | Tzar FDX, Tzar HSI, Tzar eMD |
While the backtetsing on many of these systems may not be the perfect 45 degree up angle chart you are used to seeing on many hypothetical reports, the performance numbers will represent a truly out of sample test and possibly give a much better picture of what to expect from the system in either market moving forward.
There are of course limitations to this approach, and the old saying "junk in, junk out" should dictate your actions, meaning if the system is no good to begin with, putting it on a different market is not apt to help out a lot. I can also hear a whole slew of multi-market system developers like Dean Hoffman hollering that they have been saying exactly this for years, and Dean has been correct for all these years. A system operating on a different market with the same parameters is, by definition, not optimized or curve fit.
But this approach takes it a step beyond what long term, multi-market developers do - by putting the logic onto a market it has never been applied to and well after the testing phase. So for good, robust systems like those above that have stood on their own, but also look good on different markets - you are making sure you're not on a curve fit system while at the same time proving the robustness of the system's logic. A win/win in our book.
The chart and table at the end of the newsletter show the performance of one of the "non curve-fit variants" we have implemented for clients since September of last year. The performance has been very good, and far better than that of the original system, RC Success ES.
- Jeff Malec
IMPORTANT RISK DISCLOSURE
Futures based investments are often complex and can
carry the risk of substantial losses. They are intended for sophisticated investors and are not suitable for
everyone. The ability to withstand losses and to adhere to a particular trading program in spite of trading
losses are material points which can adversely affect investor returns.
Feature | Week In Review | Chart of the Week |
Feature | Week In Review | Chart of the Week |
***Overview***
Metals markets continued to be red hot last month as investors continue to search for ways to get involved in the metals rally, by bidding up prices on anything that has metals or mining in its name. Adding to the rally was another commodity based ETF was launched last month, this time in Silver.
Leading the way in April was Copper, with an unbelievable 37% rise in April. Trend following system Trend Simplicity luckily had a long Copper position on during the month, and made approximately $15,000 per contract on the position. Elsewhere in the metals, Silver futures climbed 16% and Gold futures were up nearly 12% for the month.
Energies also remained in the news last month as prices at the pump were increasing as Unleaded Gas futures climbed over 11% while Crude Oil moved 5% higher. Natural Gas went the opposite direction, however, losing 12% in April.
Foreign currencies also picked up steam towards the end of the month with the US Dollar beginning to look very weak again. Eurocurrency futures climbed +3.9% against the greenback, while the US Dollar Index futures dropped -3.9% for the month. US Bonds also moved lower with US 30 year bond futures dropping -2.1%.
***Day Trading***
It was a mixed month for day trading programs as equities hovered around the same levels as the month prior, despite one big up day. The first half of the month S&Ps tested the area under 1290.00 only to rebound and trade above the key 1320.00 level.
Compass SP had a stellar April, making +$3,320.67 for the month and is closer to breaking into positive territory for the year. The system started the month of April with six consecutive winning trades only to cool off slightly towards the end of the month. AG Xtreme made +$1,375 for the month and is attractive to investors because it usually is one of the first systems to enter on high volatility days making for some huge payouts. BetaCon 4/1 Dax made +$1,303 on only five trades for the month. R-Mesa eRL had some big winning trades in April that amounted to +$1,133 for the month. Epsilon Bund made +$1,101 for the month thanks to some huge downside moves for Eurex bonds.
Impetus eRL chose its entries wisely and came out on top with profits of +$555.50 per contract. Compass eRL didn’t have quite the success of the SP program but still made +$542.50 for the month. Kappa Dax profited +$290.77 for the month. Tanker CL welcomed the higher volatility in energy markets and made +$100 on four trades.
Elsewhere, Rayo Plus Dax ended up on the losing side of the coin down -$144.77 for the month. Beta V2 Dax followed a similar path with losses of -$265.65 for the month. RC Success ES had a small loss of -$357.50 as did RC Miracles eMD -$390. Beta Con ESX lost -$587.52 in the EuroStoxx 50.
Moving on, SPMD was very active for the month but lost -$654.55. Theta Dax underperformed for the month with a loss of -$1,406. RC Success eRL pulled back after a red hot start to the year with a setback of -$1,860. Phi Plus Dax lost -$2,152.94 for the month.
***Swing Trading***
Topping this months returns among swing trading systems was the Tzar system which recorded gains of +$9,767.50 across its 4 market portfolio. Tzar eRL earned +$6,340, Tzar eMD added +$2,320, Tzar ES gained +$657.50, and Tzar NQ pulled in a gain of +$450. Tzar has a counter trend entry technique that has simply been on fire this year – the combined 4 market portfolio was up +$20,691 for the year as of 4/28/06!
Other b index showings for the month included the Seasonal ST eRL and Seasonal ST ES systems which earned +$2,050 and +$1,197.50 respectively on 3 trades, AG Mechwarrior ES +$1,360, Targets eMD +$1256.90, and Delphi eMD +$627.50.
Beyond the indices investors experienced an impressive run up in the FOREX market, including a long profit target last night in SC Forex GBP that locked in a total of +$6,414.36 on only 2 actual trades taken for the month. SC Forex GBP is the most recent addition to the Attain suite of live Forex systems and has a recommended $20,000 account size.
Other FX trading included gains of +$1,915 and $940 from Delphi F/X EUR and GBP respectively. The only system not to catch the big run up was Hurricane FX which was stopped out at breakeven just before the move; the system lost -$1,015 on the month. While Hurricane missed this run it will surely look to get involved in the next move when and if it occurs.
Systems that struggled the most in April were the short term trend followers which experienced a whipsaw of market action day after day. One system feeling the pain was the Ping Systems portfolio which lost -$6,655 on the month; however still remains one this years biggest winners with YTD returns of +$11,952.90 per 30k account.
Also struggling this month was the Axiom 4 market portfolio which gave back -$9,685.70 across a total of 4 markets – the hardest hit was the eRL which lost -$4,280.70, followed by the ES which lost -$3,582.50, the eMD lost -$1,292.50, and the NQ lost -$530. Also having a difficult month was Hourly eRL which ended down -$3,430.25 per 3 contract unit.
Finally the bond markets continued to be active which offered an opportunity for the short term investors to jump in and out – Gettess US had a gain of +$461.25 and Jaws Narrowneck Bonds gained +$150 per contract in April. On the longer time frames Mesa Bonds and Notes are currently long the market on a counter trend entry – for the month Mesa Notes lost -$609.375 and Mesa Bonds lost -$2,143.75.
***Long Term***
After two years of suffering, it is sure starting to look like market conditions are favoring long term trend following systems again. Trends have finally returned to the commodity markets - led by metals and energies which are seeing historic rallies. The trading activity has become infectious with markets that were stagnant for long periods of time becoming active again.
One sector that has caught fire of late is the foreign currency or FX markets. Attain offers systems for the more traditional foreign currency futures markets along with Interbank foreign exchange markets that have become popular over the past few years. This section will concentrate on the foreign currency futures positions held by long term trend following systems.
As we mentioned above the foreign currency markets continued to trend in favor of long term systems last week with Eurocurrecy futures climbing +2.13%, Swiss Franc futures jumped +2.71%, and Japanese Yen futures climbed 2.55%. Long term systems benefiting from the moves include Aberration which is long in the Dollar Index for open trade profits of +$1890.00 per contract and in the Canadian Dollar for open trade profits of +$860.00 per contract.
Andromeda is in three markets including the Swiss Franc for open trade profits of +$2175.00 per contract, the Dollar Index for open trade profits of +$1810.00 per contract, and the Japanese Yen for open trade profits of +$550.00 per contract.
Axiom LT is also in multiple markets including the Dollar Index for open trade profits of +$2480.00 per contract and in the Swiss Franc for open trade profits of +$2400.00 per contract. The system is losing -$625.00 per contract in the Japanese Yen.
Brix has caught a large portion of the trend in the Swiss Franc for open trade gains of +$3212.50 per contract and has also fared well in the Japanese Yen for profits of +$962.50 per contract.
Trend Simplicity is the only system long in the Euro and it has paid off with open trade profits of +$3537.50 per contract. Finally, SEMA4 Symmetry has just joined the party with a long position in the Canadian Dollar and a short position in the Dollar Index.
Closed out long term trades from last week include Brix losing -$775.00 per contract in Soybeans. Meanwhile Trend Simplicity had a great week in the metals making +$4700.00 per contract in Copper, however it struggled in the interest rate markets after losing -$225.00 per contract in Eurodollars.
Please Login to: http://www.attainaccess.com for the latest updated statistics.
IMPORTANT RISK DISCLOSURE
Futures based investments are often complex and can
carry the risk of substantial losses. They are intended for sophisticated investors and are not suitable for
everyone. The ability to withstand losses and to adhere to a particular trading program in spite of trading
losses are material points which can adversely affect investor returns.