Here’s Your Data, Mr. Ritholtz

We came across the following line from one of our favorite bloggers, Barry Ritholtz, last week:

“Lately, it looks like the Equity Futures predict little more than the Open,” he mused. “Are they that easily pushed around? Do they contain less and less information? I would love to see a correlation study looking at their ability to forecast that day’s close.”

Never ones to back down from a challenge, we looked to delve right into this project and calculate just such a correlation study. But on the mathematical journey to those results, we had a heck of an internal debate as to what exactly Barry meant by this line.

Does he really think that what is happening at 8:30 AM should have any ability to forecast what will transpire over the next 7 hours or so into the cash equities close at 3:05 PM?  What about news, economic reports, Fed meetings, earnings reports, and so on during the actual trading day?  (Side note: being Chicagoans, we’ll be using CST throughout this post, and still can’t believe you New Yorkers get to sleep in and stroll in to a market open at 9:30 AM.)

Being avid readers of Barry’s, we found this to be somewhat out of character with his standard operating procedures, which looks at hard data, links to Freakonomics: the folly of prediction, doesn’t believe in tea leaves as a method of predicting stock movements, and, generally, is grounded, logical reading.

And being professionals in the futures markets, we know that futures (confusingly) don’t really predict anything about futures prices. The point of futures markets is to allow market participants to lock in a future price TODAY, not to predict what is going to happen TOMORROW (or even 7 hours from now).  The point of selling futures today is so that you don’t have to sell lower tomorrow, while buying today is so you don’t have to buy higher tomorrow. It is therefore not predictive of anything except what people are willing to lock in future prices at right now.

So, confused as to whether Barry actually believes that futures are predictive of the future (which they aren’t), we gave him the benefit of the doubt (he is putting out a few thousand words a day, after all) and proceeded assuming he meant something more along the following lines:

Wow, back in the old days – a pre-open equity futures indication of say,  down -2%, meant we were destined to have a bad day, but these days it seems the pre-open is nothing more than noise, with the sentiment represented there meaning very little to the real trading day. I wonder how the cash close correlation to the pre-open futures indication has moved over time.

And that interpretation, we can test.

The Background

U.S. stock markets open at 8.30 AM CST, yet there is overnight, or electronic trading, in the stock index futures markets, from 3.30 PM CST the previous day through the open of the cash stock market, all the way through the close of the cash market, and 10 minutes extra, to 3:15 PM CST.

For those following the stock market closely, it’s fairly common to wake up in the morning and check what the futures are doing (because the cash market isn’t open yet). You can see that Barry himself incorporates this into his routine, commenting on it here and here , and of course CNBC is always putting up quotes of where stock index futures are all morning until the cash open.

The normal reaction when these morning quotes are viewed is something along the lines of – wow, we’re in for a rough day- or the quite opposite.  And what we think Barry is in effect asking is – are these reactions justified anymore? Or are we just as likely to end up 1% after being down -2% in the pre-open as we are to be down -3%?

The Testing

To test this, we measured how much equity futures were up or down in the pre-open (we used 8:15 AM CST), and compared that to how much the cash equity index was up or down at the 3:05 PM close that day.

We used the S&P 500 cash prices and eMini S&P 500 backadjusted futures data back to October of 2001, and, because of that, dealt mainly in how much of a point move (instead of percentage move) there was, as percentages can get thrown off when looking at backadjusted futures data because the prices (the denominator) get adjusted to account for the rolls from contract to contract. We created custom sessions in order to see what the price was at 8:15 AM CST every morning.  Any percentages we did calculate are based off of the average S&P cash closing price over the 10 year period (which, interestingly enough, has been just above recent prices at 1164).

The Results

Without further ado – the data:

First, it is worth noting that, despite appearances, where it seems everything happens in the overnight session and the day session just plays catch up, the bulk of activity is still happening during the day session. Or, if you view your glass as half empty – about half of the market’s movement is coming from the night session (4.86 points out of 8.53 points). Below you’ll see the absolute values of the moves, meaning the average move was either up or down the amounts listed.

Next, we looked at how well the 8:15 AM futures indication did at predicting the cash close that day.

• Cash closed that day in same direction 64% of time.
• But was, on average, 8.5 points (0.73%) different than the futures indication.
  • 6% of time was within 10% of pre-open
  • 10% of the time it was within 20% of pre-open
  • 26% of the time it was within 50% of pre-open
• The cash market day session was able to extend the move seen in the pre-open futures 48% of the time (i.e. futures up 1% at 8:15 AM, market closes that day up 1.5%), Meaning it retraced the pre-open move 52% of the time (i.e. futures up 1% at 8:15 AM, market closes that day up 0.6%).

Next, we came to the data Barry was looking for, plotting the rolling 30 day correlation of the price change (in points) at the 8:15 AM futures indication and 3:05 PM cash close that same day. The correlation shows that the 200 day average has been rising since putting in a swing low in September of 2008. You can see with a glance that the relationship is highly volatile, and oscillates quite regularly between there being a relationship (assuming 0.6 correlations as statistically significant), and there not being one (assuming correlations below 0.40 as not significant).

Finally, we looked at all of the point differentials on one chart, which seems to support the non-correlated, or, “the-pre-open-indication-is-just-noise” camp. The following chart shows the point differential between the 8:15 AM futures indication and that day’s close. For example, if futures are showing S&Ps up +8.50 points, and we then close the day down -10.00 points, the differential would be -10.00 minus 8.50, or -18.50 points. If futures are showing the market down -5.00 points and we close up 10.00 points, the differential would be 10.00 minus -5.00, or +15.00 points.  The blue on our graph below means the market closed better (meaning higher for those long stocks) than the 8:15 AM futures indication, while the red on our graph means the market closed worse/lower than the 8:15 AM futures indication.

Two things jump out here. One, the difference between the cash close and 8:15 AM futures indication gets larger (on both sides) as volatility increases, which is unsurprising, as the point ranges are expanding. You can see the larger spikes in 2002, 2007, 2008, and recently.

Second, there is no clear-cut direction or bias for these differences, with the negative differentials appearing as a near mirror image of the positive differentials. This tells us that that, not only does the market frequently get the 8:15 AM futures indication wrong, but, in some lights, seemingly tries to make up for it in the near future by pushing the cash close beyond that indication.  The rationale? Perhaps some form of volume weighted averaging by overnight HFT (how will they defend providing liquidity in the illiquid overnight market?), or the old fill the gap mentality of traders where a gap open lower begs for prices to retrace to the previous close, or profit taking?

Conclusion

From what we’re seeing, the futures are not containing less and less information– they just never really contained all that much information to start with.  The correlation is positive and rising-which could mean it contains more information or that we’re reacting more to global moves in the growing global economy- but this correlation oscillates pretty wildly. And while the 8:15 AM futures indication does predict the direction of the cash close 64% of the time, that does little good when the magnitude could be off by a factor of 10!

The takeaway as we see it? Ignore the pre-open as it relates to what will happen in your portfolio over the next few hours. It could get worse, it could get better, but the only real certainty is that it won’t stay right there for too long.  As Barry would be likely to say, investing is proactive, not reactive – so don’t react too violently to the futures when you wake up… it’s likely just noise.