Mutual fund ratings not all they're cracked up to be

Whether an investor is a 401(k) plan sponsor or a participant, once any investor selects the asset classes in which they intend to invest, they face the difficult issue of selecting specific managers. A variety of methods have been used over time, but one method that seems to have captured the focus of many investors is that of the performance-based rating system.

Unfortunately, these rating systems tend to be imperfect when it comes to predicting future performance. The basis of most rating systems is some combination of past performance — typically over three-, five- or 10-years — and a measure of the fund's risk. The problem with using purely quantitative rating systems is that the information used for each manager is historical. If you have ever read a mutual fund's prospectus, you have encountered the disclaimer that past performance is not an indication of future performance. The Securities and Exchange Commission requires that mutual funds use that disclaimer for good reason: It is absolutely true.

Just because a mutual fund performed well in the past does not mean it will in the future. In fact, it might mean the fund will underperform the market over the next several years.

A good example of this is the performance of growth-oriented mutual funds which focused on the information technology sector during the late 1990s. For many years, funds with an overweight exposure to technology stocks generally performed better than their peers. The ratings under various systems for many of these technology-focused mutual funds were due to the explosive growth of the companies in which they invested during the high-tech boom. But as we all know, these funds suffered significantly during the high-tech bust. This is a clear example of the problem with the rating systems.

Savvy investors understand that they cannot blindly invest in a mutual fund just because it has been highly rated.

A 2009 study by Advisor Perspectives evaluated the predictive ability of the Morningstar rating system. The study measured the probability that a randomly selected higher-rated fund will outperform a randomly selected lower-rated fund. The study's organizers believed that this metric was the most meaningful way to assess the usefulness of the rating system. The firm evaluated the incremental improvement that they would obtain by, for example, trading up from a 4-star to a 5-star fund.

The study found that a one-star improvement across the five fund categories tested (U.S. equity funds, international funds, balanced funds, taxable bond funds, municipal bond funds) was 50.6%. Essentially a 50-50 chance whether a higher-rated fund would outperform a lower-rated fund.

In response to the study, Russel Kinnel, director of mutual fund research for Morningstar Inc., stated: “In short, the star rating is a backward-looking measure of past performance. What it is not is a forward-looking measure of fundamentals.”

In 2010, C. Thomas Howard, a professor at the Reiman School of Finance, University of Denver, confirmed Advisor Perspectives' evaluation. Mr. Howard's study evaluated the predictive power of the Morningstar weighted historical returns using a sample of all active U.S. equity mutual funds spanning the period from January 1980 through June 2008. He found that the fraction of the equity funds' subsequent one-year return volatility explained by these weighted historical returns is .002 (i.e. r-squared). This essentially zero correlation is right in line with Advisor Perspective's findings that star ratings are not predictive of performance.

An additional test of the predictive value of historical performance was conducted by investment consulting firm DiMeo Schneider. Its analysis identified funds in various investment categories that had been in the top quartile over the past 10 years — funds that many investors would be drawn to. The study concluded nearly 85% of those top-quartile funds (across all categories) delivered investment results below their respective benchmarks for at least one three-year period (12 consecutive quarters) during that same 10-year period.

"The Morningstar Ratings Methodology" document states, “While the long-term overall star rating formula seems to give the most weight to the 10-year period, the most recent three-year period actually has the greatest impact because it is included in all three rating periods.” Based on this, there should be a concern that funds which have performed well in the most recent three-year period will have a higher rating and will lead investors to the conclusion that these funds will outperform going forward.

A discussion of the current environment might shed light on the risk of chasing recent outperformers. It is commonly believed that high-quality stocks act as a cushion during market downturns, thereby providing downside protection. If this thesis is correct, then one would expect that in periods of declining risk, narrowing credit spreads and flattening yield curves, the quality premium would tend to be negative and managers who invest in the stocks of high-quality companies would underperform.

This has proved true during the recent market recovery. From January 1, 2009, and March 31, 2011, those stocks with higher variability in growth and stability of earnings and dividends have dramatically outperformed the stocks of higher quality companies.

The ratings of many low-quality, high-beta focused mutual funds have increased dramatically based on their short-term outperformance during this timeframe. But as markets refocus on the quality of earnings, we would expect these managers to underperform, disappointing investors who have relied on their recently improved ratings.

We concur with the research presented above that ratings are not an effective forward-looking measure. This is not to say that rating systems have no value. Past performance can be one indication of a well-managed fund. In addition, a rating system can be used as a first filter when developing an overall investment strategy. For example, the rating system could be used as an introduction to prospective funds — but not the deciding factor or the only factor to use when researching and selecting a mutual fund.

We believe it is best practice for defined contribution plan sponsors to limit the use of wording that refers to any purely quantitative performance-based rating in their investment policy statements and recommend that those sponsors and their consultants incorporate qualitative factors into the selection of the managers for their investment menu.

Ratings and past performance must be combined with other information. The correct approach to fund selection relegates historical performance to a secondary consideration. The primary focus in manager choice should be qualitative, including a thorough analysis of the product's organization, management, investment style, philosophy and process, with attention paid to those factors which provide long-term and forward looking success.

Gordon Tewell is a principal, consultant and member of Innovest's investment committee, which makes decisions on investment related research and due diligence.