Sign Up

Managed futures newsletter


Sign up now to receive our   free newsletter

Click here to gain free access. Build portfolios, set watchlists, and view more data and statistics.

Call us at 800.311.1145 to speak with one of our alternative investment specialists. We answer the phone in One Ring. Try It.Sign up to view performance on 100s of Managed Futures Programs, Trading Systems, and Managed Forex Programs. Sign up FREEWhat are Managed Futures? Is this the same as CTAs? How do I invest? Click here  to learn all of this and more on our extensive managed futures education pageHow to set watchlists? Build portfolios? Find correlations? and more. Click here to take a tour of our advanced toolsUse our most popular tool to create custom multi-program portfolios. Click here to get started today by signing up for FREE ACCESSClick below to learn how attain can assist your CTA in everything from back office creation and trade execution to finding a lawyer to create your D-DocNo upfront fees for managed futures funds is one of the unique benefits of a managed futures account at AttainOur alternative investment books list includes some of the most thought provoking and interesting books on alternative investmentsLearn how family offices outsource the managed futures research, due diligence, data collection, and ongoing monitoring of accounts to AttainWhat is a trading system? Who develops them, and how are they executed for client accounts? Our trading system education explains this and moreWe assist talented traders in getting their trading ideas into an automated trading system, do testing, marketing, and more

In Defense of Managed Futures Indices- Part Two

January 2, 2013

 

 

A couple of weeks ago, we began a long overdue defense of managed futures indices. Financial indices are useful, albeit imperfect, tools for understanding asset class performance, but managed futures indices are criticized far more frequently than traditional indices are, and we'd had enough of the poorly founded saber rattling.

However, the defense wound up much longer than we'd anticipated, and rather than bombard you one enormous piece, we decided to split the defense into two parts. Christmas last week delayed the send of the second part, and New Years yesterday would have delayed it further, but we have too much planned for the coming weeks to delay any further. So, without any further ado, the continued defense of managed futures indices.

Miss the first part? Click here for the overview of managed futures index criticisms, and the first part of the response.

Survivorship Bias

People like to talk a LOT about survivorship bias in CTA indices. They'll argue that IF ONLY we could see all the programs that HAD NOT done well, we would know just how terrible and corrupt the managed futures space is. We'll frequently see people cite "evidence" of a 3 to 5 percent performance drag when this bias is incorporated. However, what those same people won't tell you is that the studies in question are attacking aggregate performance of a database, versus the performance of an index. We'll get into why that's a silly comparison in a minute.

This idea makes us laugh every time because the blows-ups are in the indices already. But before we go there, let's make sure we're on the same page regarding what we're talking about.

Traditionally, survivorship bias has referred to the exclusion of performance of failed or failing CTAs. It’s easy to understand that the winners are the ones who stay in business and survive while the losers are the ones which likely lost money and went out of business. And it’s easy to understand that if you don’t include the losers, any look at performance will be overstated. That’s all common sense, but how would one go about actually doing that calculation of excluding the losers?

An index demonstrating such a bias would have to actively exclude the performance of a failed CTA after the fact, changing all past index returns as though the program never existed in the results to begin with. That is definitely a problem and would definitely create a huge bias in indices, we couldn’t agree more. But here’s the thing – no managed futures indices are calculated in that way. Nor hedge fund indices that we know of. None of them.

In more colloquial terms, survivorship bias has come to mean that indices will not include the blow-up of a firm in the performance of an index. This is patently false. The DJCS and Newedge Indices deal with such large CTAs that they do not encounter this concern, and their inclusion criteria is based on AUM, which cuts out their ability to discriminate based on performance. In fact, in a 2012 interview with Opalesque, Newedge Senior Advisor Galen Burghardt stated:

As things have proved out, only one CTA has ever dropped out of the index in mid-year, and that was Bridgewater, and that was certainly not because they were not successful.

But the BarclayHedge CTA Index, with over 600 constituents in the year 2012, does, on occasion, see one of its index constituents close up shop. This was particularly true in the late 1990's. A 1998 paper by Spurgin and Schneeweis revealed:

The data set consists of 1171 funds that reported results to Barclay Trading Group for any period from January 1988 until June, 1997, a total of 114 months.There was a high degree of turnover during the period.290 funds were tracked at the start in 1988, 881 new funds entered the database, 737 funds dissolved prior to June 1997, and 434 funds were active at the conclusion of the test period. In all, there are 60,590 monthly observations, or an average of 51.6 months per fund in the sample.As seen in Figure 1, there was a large influx of CTAs in the early 1990's and relatively few departures, which caused the number of CTAs to increase to over 700 in late 1993.From that point forward, the number of CTAs exiting the market has remained above 30 per quarter while the number of new CTAs dropped to near zero.The total number of CTAs has declined each quarter after the end of 1993, reaching 434 at the end of June 1997.

Yikes. That's not a small attrition rate, and certainly enough to make any investor quake in their boots. But the numbers alone do not tell the whole story. Not only is this not reflective of the attrition rate within an index, but it neglects the most important questions of why they stopped actively reporting, and how the performance was treated within an index.

The question of why a fund stops reporting is a difficult one to ascertain. A frequent reason for a cessation in reporting is a lack of assets. As Agarwal, Fos and Jiang explained, for young funds, performance reporting is about marketing. But as anyone in this industry will tell you, reporting to a database is rarely going to cause money to start walking in the door; you have to work for it. If a firm does not attract enough assets, the operating costs of running the business may overwhelm the trading, killing the cost-efficiency of continuing the program. These programs don't necessarily blow up, but they don't have a reason to continue. Furthering this point, for the period referenced above, Spurgin and Schneeweis found that the bulk of CTAs closing up shop during that period had much smaller assets under management.

On the flip side of this, some larger funds or programs at capacity may find they no longer have a need to report to these databases. After all, if you can't take on any more money without impacting the performance of the program, or you're doing well enough with the amount of assets under management you have, then why spend the time updating a performance database anymore? You no longer need the attention. While this is likely more common in the hedge fund space than the CTA world, it's not unheard of.

But these aren't the only reasons a CTA will stop reporting. To prove this point, we reached out to CTAs from across the industry who had stopped reporting to the BarclayHedge over the course of the last year. The point of the outreach was to determine 1) why they had stopped reporting, and 2) whether they had continued trading after they stopped reporting.

The reasons for the cessation of reporting were incredibly varied. Some of what we heard included:

·         Their admin hadn't had a chance to update the performance numbers yet.

·         They offered a variety of programs, and they had decided to consolidate down to one program.

·         They had changed the name of the program for marketing purposes.

·         The program in question was a duplicate of a program they were already regularly updating with a small difference in the name (spacing, punctuation, and so on).

·         They had been forced to close up shop due to the freezing of assets in the MFGlobal of PFGBest bankruptcies.

·         They are in the process of preparing to retire, so they are streamlining their business affairs.

In other words, the world doesn't have to be ending for a CTA to stop reporting to a database. But what about when a program shuts down because of performance? We know there have been a few of those this year (for instance, FCI). What happens to their track record?

Again, BarclayHedge is the index most likely to face this concern, but the important thing about the rules of their index is that they only rebalance their constituent composition annually. This means that CTAs are not removed from the index except at that time. That means that if a CTA blows up in March, they will be included in the index until the next rebalancing. As Waksman pointed out in our conversations, the BTOP 50, explicitly, contains not just one blow up from famed Turtle Trader Richard Dennis - but TWO.

But what does this really mean? After all, in our experience, most CTAs don't trade to zero. They will stop trading before all the money is gone. In fact, of the CTAs we spoke with who HAD closed up shop, not a single one had kept trading after they stopped reporting. So if there's no trading in a program, how is performance generated and reflected? As a zero. They're still included in the index, but when trading stops, their performance, for the rest of that annual period, is recorded as 0 and calculated in with the rest of the CTA performance in the index.

In other words, the blow ups you want included in the indices? They're already there.

This is uniquely impressive, if you consider the period of accelerated closures referenced by Spurgin and Schneeweis. In the 1993-1997 period, there were four years of positive performance, including three double digit gain years, and the only losing period was only -0.65%. If the rate of attrition was as high as they say, and they were being calculated into industry performance on a monthly basis as turning in 0%, think about what the performance of the rest of the industry must have been to turn in those numbers. Of course, past performance is not necessarily indicative of future results, but it's certainly food for thought.

Hold on, you might say. I thought Simon Lack said that there was 3-5% drag on performance once survivorship bias was calculated in. You are correct. He said that. The author he attributed the data to, however, did not say that. In fact, in our survey of all the criticisms of managed futures indices, we could not find a single study that was able to quantify the impact of survivorship bias. In fact, Fung and Hsieh found that it was next to impossible to accurately identify the extent of survivorship bias in any data set without thorough manipulation... manipulation which would, in turn, call into question the integrity of the results.

The only way we see survivorship bias happening is going back and changing index results after the fact. If there were survivorship bias, we would see the BarclayHedge CTA index change from up 18% to up 22% in 2008, for example; or a Newedge press release that they were removing the losers from 2011 so the index went from -4% to up 1%. This simply doesn’t happen, and won’t happen as the indices are constructed.

Indeed, to us, an index is the antidote to survivorship bias, not a victim of it.While an investor looking at any single program today may realize some survivorship bias because he can’t see the single programs that failed – an index holds the history of its constituents old and new. The index is the historical record of the successful and failed programs.

Selection Bias

The last hurdle in terms of bias in financial indices is that of selection bias. In more traditional financial literature (read: pertaining to more traditional asset classes) selection bias more frequently refers to the idea of voluntary participation in an index (for example, you could choose to list your new company on the Nasdaq or NYSE, essentially allowing you to voluntarily choose whether to be included in the Nasdaq index or NYSE index), but here we'll take a look at the other way selection bias is utilized- index composition.

Each index has its own set of rules. For our purposes, we tend to rely on third party indices that are created in an effort to represent the industry. In our experience and opinion, despite the wide array of indices offered, the most useful and reliable indices have been the BarclayHedge CTA Index, the DJCS (Dow Jones Credit Suisse) Managed Futures Index, and the Newedge CTA Index. Their goal is very similar, but their execution is rather different:

 

Index

BarclayHedge CTA Index

Dow Jones Credit Suisse Managed Futures Index

Newedge CTA Index

Inclusion Criteria

4+ Years Advisor Record
2+ Years Record for New Programs

Minimum $50mm AUM
1 Year Track Record
Audited Financials

Top 20 Managers by AUM
Open to new investments
Daily reporting

Number of Constituents

602

34

20

Reporting Frequency

Monthly

Monthly

Daily

Constituent Rebalancing
Frequency

Annually

Quarterly

Annually

Weighting

Equally Weighted

Asset Weighted

Equally Weighted

Inception date

Jan-87

Jan-99

Jan-00

AUM Represented

No target

At least 85% of industry AUM

Top 20 Managers by AUM

 

 

As the saying goes, there are multiple ways to skin a cat. And as you can see, each index has its own criteria for index composition. That criteria, to bring things full circle, represents their selection bias. Given the differences in how they approach industry representation, one might expect that they would present wildly differing performance. We might even expect it; after all, we just wrote a piece about the shrinking returns of the titans, and that's who's in the Newedge CTA Index.

Except, then you look at their correlation levels (starting at Newedge inception)...

That's not exactly low correlation. And it's been consistent. Year after year, these very distinctive indices track each other fairly closely. There are months where they will deviate from one another, but in the aggregate, the performance stays pretty tight. This leads us to believe that, in a world where selection bias is calculable, given the lack of significant deviation in the performance of these indices despite the different methodologies, the impact of such bias is minimal... at best.

But that doesn't mean selection bias doesn't exist. So the critics AT LEAST have that in their quiver... right?

The Case for Intentional Bias

Not so fast. Just because we'll say that the managed futures indices include selection bias doesn't mean that we think it's the end of the world. In fact, in this instance, we think it's probably a pretty good idea.

The managed futures industry is sprawling. There are over 1000 managers registered with the NFA, and many more globally with no need to register. In no world would we recommend investing with all of them. In fact, we can't think of a single broker worth their salt who would tell you that's a good idea, and you would have to have a few billion to even accomplish it. While the cost of entry today is a little higher than it used to be, starting a CTA- generally speaking- is not difficult. It's delivering as a CTA that's the hard part.

None of these indices represent those young upstarts. Sure, you can report to the BarclayHedge database, but you won't be considered a representative part of industry performance until you've proven you're going to stick around for a little bit. And if they're not at a point where they could be considered for inclusion, we're not sure you should be including them in your managed futures portfolio. This is why the cross-application of database criticisms to indices is silly. These indices are not meant to represent the ENTIRE universe of an asset class. They are DESIGNED to exclude options not yet ready for investment - just like the S&P 500 doesn’t include penny stocks or companies listed on the pink sheets.

This kind of filtering, in our opinion, is good for investors, because it provides a glimpse of performance a little more representative of the kinds of managers they should be willing to give a second look, versus the performance of managers we'd tell them to ignore until they've grown up a bit. So yes- we're all for a little selection bias, particularly if the selection bias provides a more honest representation of investable options. And really, it could be a lot worse. No- correction- it is a lot worse on the equities side. When people complain about index bias, take a look at the way the S&P 500 Index is maintained. This may be the most widely cited stock index in existence, but they're the poster boy for bias. As their methodology PDF explains their paradigm for deletions as such:

Companies that are involved in mergers, acquisitions, or significant restructuringsuch that they no longer meet inclusion criteria. Companies delisted as a result of merger, acquisition or other corporate actionare removed at a time announced by S&P Indices, normally at the close of thelast day of trading or expiration of a tender offer. Constituents that are halted from trading may be kept in the index until trading resumes, at the discretion ofS&P Indices.If a company is moved to the pink sheets or the bulletin board, thestock will be removed. Index changes are announced with one to five days’ advance notice.

Companies that substantially violate one or more of the addition criteria S&P Indices believes turnover in index membership should be avoided whenpossible.At times a company may appear to temporarily violate one or more ofthe addition criteria.However, the addition criteria are for addition to an index,not for continued membership.As a result, an index constituent that appears to violate criteria for addition to that index will not be deleted unless ongoing conditions warrant an index change.When a company is removed from an index, S&P Indices will explain the basis for the removal. S&P Indices:S&P U.S. Indices Methodology TMIX constituents that acquire S&P 1500 index constituents but do not fully meet all inclusion criteria may still be added to an S&P 1500 index at the discretion of the U.S. index committee if the committee determines that the addition could minimize turnover and enhance the representativeness of the index as a market benchmark.

Yes, you read that correctly. Not only are changes made in an arbitrary timeframe, they're made on an arbitrary basis by a board. No, really. There is no structure beyond that. They even go so far as to define their parameters for change timing thusly:

Changes to the U.S. indices other than the TMIX are made as needed, with no annual or semi-annual reconstitution.

Selection bias? Survivorship bias? Yep, the case could be made. The advantage here is that there is no self-reporting bias... until there is. Like when the largest financial institutions in the world misrepresent their liabilities under the cover of the law. But that's a discussion for another day.

The point is, despite the complaints from critics about the amount of bias found in managed futures indices, the bulk of their laments have no basis in reality, and the ones that do, we would argue, are either minimal or actually beneficial. And the ironic part is that the bulk of these critics are the same ones that worship the cult of equities and the indices that represent them.

Conclusion

The number one reason that none of this matters is that you don't invest in managed futures indices. Not the big ones, at least. The indices that were designed to represent the industry were designed as just that. They were not created to represent the ideal, balanced portfolio. When you, as an individual investor, go to consider allocations, you will not consider the Newedge Index v. the DJCS Index. You will look at the performance of individual programs. There, you'll find no selection bias in their performance; they have to report all their performance, not just a selection of it for their track record. We believe you’ll find no self-reporting bias – and you can request an audited report of the performance to insure their self reported numbers in their disclosure document match client statements.

About the only bias you encounter is a version of survivorship bias – where it is important to recognize that not every program has been as successful as the one you are looking at. The survivor you are considering has likely performed better than the non survivors, but isn’t that the point? Isn’t that what you are looking for? And as a practical manner, the survivors are the only programs you can invest in. You can’t invest in a program that has been shuttered.

The slamming of hedge funds and managed futures performance data as biased, and subsequent implications for the indices that track their performance is not going to go away. It's going to come up again next year or month or week, and you'll probably see the same, tired arguments trotted out. It's an interesting intellectual debate, especially when considering that some bias could be beneficial. But for the investor... as long as you can put those index numbers in context, the rest is really just noise.

IMPORTANT RISK DISCLOSURE


No Yes Was this article particularly interesting or helpful to you?

No Forward this email to a friend who might find it useful.

Not on our mailing list? Sign up now to receive this weekly newsletter.

Feature

IMPORTANT RISK DISCLOSURE
Futures based investments are often complex and can carry the risk of substantial losses. They are intended for sophisticated investors and are not suitable for everyone. The ability to withstand losses and to adhere to a particular trading program in spite of trading losses are material points which can adversely affect investor returns.

Past performance is not necessarily indicative of future results. The performance data for the various Commodity Trading Advisor ("CTA") and Managed Forex programs listed above are compiled from various sources, including Barclay Hedge, Attain Capital Management, LLC's ("Attain") own estimates of performance based on account managed by advisors on its books, and reports directly from the advisors. These performance figures should not be relied on independent of the individual advisor's disclosure document, which has important information regarding the method of calculation used, whether or not the performance includes proprietary results, and other important footnotes on the advisor's track record.

The dollar based performance data for the various trading systems listed above represent the actual profits and losses achieved on a single contract basis in client accounts, and are inclusive of a $50 per round turn commission ($30 per e-mini contracts). Except where noted, the gains/losses are for closed out trades. The actual percentage gains/losses experienced by investors will vary depending on many factors, including, but not limited to: starting account balances, market behavior, the duration and extent of investor's participation (whether or not all signals are taken) in the specified system and money management techniques. Because of this, actual percentage gains/losses experienced by investors may be materially different than the percentage gains/losses as presented on this website.

Please read carefully the CFTC required disclaimer regarding hypothetical results below.

HYPOTHETICAL PERFORMANCE RESULTS HAVE MANY INHERENT LIMITATIONS, SOME OF WHICH ARE DESCRIBED BELOW. NO REPRESENTATION IS BEING MADE THAT ANY ACCOUNT WILL OR IS LIKELY TO ACHIEVE PROFITS OR LOSSES SIMILAR TO THOSE SHOWN; IN FACT, THERE ARE FREQUENTLY SHARP DIFFERENCES BETWEEN HYPOTHETICAL PERFORMANCE RESULTS AND THE ACTUAL RESULTS SUBSEQUENTLY ACHIEVED BY ANY PARTICULAR TRADING PROGRAM. ONE OF THE LIMITATIONS OF HYPOTHETICAL PERFORMANCE RESULTS IS THAT THEY ARE GENERALLY PREPARED WITH THE BENEFIT OF HINDSIGHT. IN ADDITION, HYPOTHETICAL TRADING DOES NOT INVOLVE FINANCIAL RISK, AND NO HYPOTHETICAL TRADING RECORD CAN COMPLETELY ACCOUNT FOR THE IMPACT OF FINANCIAL RISK OF ACTUAL TRADING. FOR EXAMPLE, THE ABILITY TO WITHSTAND LOSSES OR TO ADHERE TO A PARTICULAR TRADING PROGRAM IN SPITE OF TRADING LOSSES ARE MATERIAL POINTS WHICH CAN ALSO ADVERSELY AFFECT ACTUAL TRADING RESULTS. THERE ARE NUMEROUS OTHER FACTORS RELATED TO THE MARKETS IN GENERAL OR TO THE IMPLEMENTATION OF ANY SPECIFIC TRADING PROGRAM WHICH CANNOT BE FULLY ACCOUNTED FOR IN THE PREPARATION OF HYPOTHETICAL PERFORMANCE RESULTS AND ALL WHICH CAN ADVERSELY AFFECT TRADING RESULTS.