Long Backtests and Madoff’esque Returns

25Oct10

One last thought on this month’s series re: Tactical Asset Allocation (TAA).

Consider this backtest of our TAA model (red) vs the S&P 500 (grey) from 1971.


[growth of $10,000, logarithmically-scaled]

I fear that when investors look at a backtest like this one that covers such a long span of time their first thought is “wow, look at those smooth Madoff’esque returns, I’ll never have another sleepless night again”.

That would be the wrong conclusion.

The problem is that all the wins and losses that can make a strategy so gut-wrenching to trade in the real-world get lost when viewed from 30,000 feet up. Heck, even the S&P 500 looks like a pussy cat from up here (the 1987 crash is barely even noticeable).

2008 would have been a great year for the model (TAA in red, S&P 500 in grey)…


[growth of $1, logarithmically-scaled]

…but 2009 would have been uber painful…

…and neither is obvious just looking at the equity curve.

I show equity curves on this blog because it’s what most readers expect to see, but in my own work I instead using “rolling volatility-adjusted returns” (read more).

Consider the graph below…

Each point on the graph is the Sharpe Ratio of the model over the previous year. The Sharpe Ratio is a measure of return relative to volatility. High numbers are good and low numbers bad.

This graph makes it clear that the model has, despite that smooth equity curve, frequently gone through periods of weak performance.

I also look at rolling excess volatility-adjusted returns, or volatility-adjusted returns relative to some benchmark. The graph below shows the same rolling Sharpe Ratio minus the Sharpe Ratio for the S&P 500.

Here we see that the model has frequently underperformed a simple buy & hold investment (in some years by a ginormous margin).

. . . . .

When assessing a strategy, the 30,000 foot view definitely matters, and from a 30,000 foot view the TAA model has clearly trounced the market.

But the point of this post is to show that even the best strategy goes through long rough patches that might not be apparent from 30,000 feet.

The “rolling volatility-adjusted return” approach I’ve shown here is one tool to help the investor get a more realistic view of how consistently a strategy performed and how it might have actually felt to trade the strategy in the real world.

Happy Trading,
ms

. . . . .

To stay up to date with what’s happening at the MarketSci Blog, we recommend subscribing to our RSS Feed or Email Feed.



15 Responses to “Long Backtests and Madoff’esque Returns”

  1. 1 T

    Dear ms

    very interesting article about TAA. In times like these, one question arises: Can you do the same thing with currencies?

    t

    • 2 MarketSci

      Hello T: currently we only consider the USD index (read more about the TAA model here: http://marketsci.wordpress.com/2010/10/20/roundup-tactical-asset-allocation/).

      But I assume you’re asking if we could apply the TAA model to only trading various currencies versus each other. Interesting idea. I’ve added it to the todo list to take a look.

      michael

      • 3 T

        Dear Michael,

        many thanks for your quick reply. Yes, indeed I was thinking about currency-only portfolios. In my own models, I have very similar results to yours, when it comes to the multi-asset class approach (not much surprise here). However, when trying currencies-only (or commodities-only for that matter), it seems that all ingredients to make things work *should* be there, but the results are much less satisfactory in terms of stability over time. Must have to do with some inherent features of these asset classes (think: mean reversion) that make TAA here much more cumbersome.

        Keep up your great work!
        best,
        t

    • 4 Stefan S

      Hello T, MS,

      I had exactly the same experience. I developed a very similar TAA strategy, inspired by the same ubiquitous Mebane paper, including volatility based position sizing (aka risk parity allocation). But it was not performing as well on a currency only asset class basket. And I have the same thoughts of why this is:
      The assets used in the classic TAA represent economies and hard assets, “stores of value”, whose value grows over time, in both relative (inflation based) and absolute (due to real economic growth). So, all we are doing, in fact, is trying to smooth out the equity curve, by avoiding bear markets & corrections. But the basic trends are up, for all these asset classes, oevr the backtest timeframes, and this is what gives the strategy its performance. Whereas in a currency only portfolio, reversion to the mean dominates, and performance suffers. This are my thoughts about this phenomenon.

      Stefan

  2. 5 charley

    Michael, in some ways this post makes the long term TAA model even more attractive. Consider a long term investor in this strategy. They should be ecstatic that they didn’t lose 30-40% of their lifetime savings in 2008-2009. If they moan about subsequently getting “only” a 9% return in 2009, they seriously need to get their head examined ;-) Maybe I’m preaching to the choir a bit here… Anyway, 2nd point: I’m curious how the model did around 1980. Looks like there was a period of 3+ years of almost zero return. IMO that would be a real challenge to the typical investor. Think about it: 36 months of making allocation switches, and nothing to show for it. Would take a lot of discipline to continue following the strategy after such a stretch, don’t you think? Would *you* still continue to allocate your capital to this model if for some reason it is flat between 2010-2013?

  3. 6 Robbie Davis

    This could lead us quickly to the slippery slope of excessive curve-fitting, but have you given any thought of using the rolling excess 1-year Sharpe ratio to modify exposure? For example, you could add more juice when the the ratio dips below -1 and reduce exposure when the ratio exceeds 3.

    • 7 MarketSci

      Hello Robbie – I think you’re right – that slope is waaay too slippery. michael

  4. 8 DWS

    MS – I’m not sure how you got the S&P chart for 2009? I show Jan -8.21% and Feb -10.74%?

    • 9 MarketSci

      DWS – did you adjust for dividends? michael

    • 10 MarketSci

      RE to DWS: I get those same number dividend adjusted (and that’s what the graph reflects). I don’t understand what the question is. michael

      • 11 steve

        an excellent post michael. that these strategies don’t work ALL the time is why they work MOST of the time. disgruntled investors pile in when they’re working well vis a vis the market and pile out when underperforming. can’t change human nature!

  5. 12 Rod

    Good work Michael.

  6. 13 Lars Bech

    As always it has been a very interesting read. I too believe it will be hard for investors to witness a lower return in raging bull years. But it is quite important to stress out that it is about minimizing the draw down (pay back to the market) and thereby returning a nice return over a long period. In my former position as an investment advisor I know this to be a tough task.

    Looking forward to reading more from you.


  1. 1 Forget the Kauffman Blogger Survey, here's what they're really saying | New Rules of Investing
  2. 2 Tuesday links: a radical transformation Abnormal Returns

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s