Plan sponsors, consultants and money managers have a keen interest in the results of benchmarking, the comparison of a manager's returns to the returns of an index.
Benchmarking is intended to provide information to plan sponsors and others about managers' alphas, alpha variability and correlation between manager alphas. The information is then used to create a superior team of managers.
Likewise, managers use the results of benchmarking to better understand themselves and to communicate with clients, prospects and consultants. Managers also can be compensated on their performance vs. a benchmark.
In our earlier article (Pensions & Investments, April 28), we recognized all portfolios, including indexes, have alphas when compared with the value-weighted market portfolio of all stocks. We then demonstrated the index alphas were clearly large enough in the 1991-'95 period to affect the evaluation of a manager's skill. We also suggested other influences that, when combined with benchmark alphas, render the information gained from benchmarking all but useless in properly evaluating active managers.
This report discusses the evaluation problems created when a benchmark does not have a style profile identical to that of the manager being evaluated.
Style profile is the term we use to describe the exposure of a portfolio to the persistent factors that affect domestic equity returns. We use the concepts of Eugene Fama and Kenneth French to develop a portfolio's style profile.
Three factors watched
Fama-French find the persistent factors that drive portfolio returns are related to capitalization and book-to-market, representing the style dimension commonly referred to as value/growth, and the market factor. Many common factors in equity returns have exposures that could have been included in a portfolio style profile. Except for the persistent factors identified by Fama-French, almost all of these other factors are transient in the sense that they leave no consistent imprint on average returns. The style profile we estimate consists of a time series of betas or portfolio exposures to size, value/growth and the market.
We then develop a custom benchmark for each portfolio by multiplying the factor exposures of the style profile at the beginning of the period times the factor returns during the period. To measure a manager's skill, we subtract the returns of the custom benchmark from the returns of the manager being evaluated.
In our previous report, we identified the style profile and alphas of five Russell indexes. Now, we add four active managers and a manager composite. First we calculate a custom benchmark for each manager and then compute his alpha. For contrast, we also evaluate each manager against an index with a similar style profile to demonstrate index contamination error. We do the same for the manager composite. We also show why misfit risk between the manager composite and its target must be clearly understood and dealt with by investment committees.
Benchmark with alpha of 0
We use monthly returns to compute each portfolio's average style profile over the five-year period through 1995. The style profile's factor exposures are expressed as betas relative to those of the value-weighted market portfolio of all stocks. The betas of the market portfolio with respect to size, value/growth and the market are 0, 0 and 1, respectively. We then calculate the returns to the custom benchmark and compute the manager's alpha.
Our approach provides a custom benchmark whose alpha is zero and whose style profile is identical to the portfolio being evaluated. Our use of size, value/growth and market factors ties in nicely with the industry's style classification attempts. Manager performance is almost always stated in terms of a growth rate. While a growth rate conveys important information, it isn't the best measure for decision-making purposes. The volatility of a manager's growth rate can be diversified away in a properly constructed portfolio. For a manager of managers, the preferred measure is arithmetic returns, because they can be subjected to standard statistical methods and yield exact decompositions of both percent and dollar returns.
All returns in Exhibit 1 are arithmetic returns except for the Growth Rate column, which shows geometric returns to the portfolios for the five-year period. The table also shows the portfolio style profile components, stated as betas. The average total return of each portfolio is then broken into the returns from each component of its custom benchmark, size, value/growth, the market and the risk-free rate. The alpha computed from its custom benchmark is then shown. Manager vs. benchmark represents the results from benchmarking a manager, that is, subtracting the return of some index (not the custom benchmark) from the return of the manager. Benchmark error represents the difference between the alpha computed from the custom benchmark and the results of benchmarking.
The vertical axis of the scatter diagram in Exhibit 1 represents the book-to-market betas, which range from -0.6 to 0.4. The larger the beta, the more value-oriented the portfolio is. Growth portfolios have negative book-to-market betas. The horizontal axis shows the size betas, which range from -0.4 to 1.1. Portfolios with the greatest exposure to small-capitalization stocks have the largest size betas. Large-cap portfolios have negative betas. The largest, most growth-oriented portfolios are located in the lower left-hand corner of the scatter diagram. The background dots in the diagram represent about 600 mutual funds with five years of data.
Manager C, the large-cap value manager for this multimanager team, has a size beta of -0.02, a book-to-market beta of 0.34 and a market beta of 1.02. The alpha computed from its custom benchmark is -39 basis points per year over the past five years. When compared to the Russell 1000 Value index, Manager C outperformed by 48 basis points per year. Benchmarking overstates the manager's alpha by 87 basis points and is reported under Benchmark error.
Understanding benchmark error
To understand the reasons benchmarking provides erroneous information, we must compare the style profile of the manager to that of the index used for benchmarking. If the style profiles are not identical, the factor returns will affect the return of the index to a different degree than the portfolio return. This difference, combined with the alpha of the benchmark, will contaminate manager evaluation. The differences in the size and value/growth components of the style profile are plotted in the scatter diagram. The table shows all three parts.
Exhibit 2 breaks down the components of the benchmark error. It shows the return impact of the style profile exposure differences as well as the impact of the index's own alpha.
The index chosen as a benchmark for each manager can be found in the column to the right of the manager's name. In the case of Manager C, the benchmark is Portfolio V, which represents the Russell 1000 Value index. Benchmarking overstates Manager C's alpha by 87 basis points. This benchmark error has four identifiable and measurable components. Manager C gains 29 basis points per year from having more size exposure (-0.02 vs. -0.1 for the benchmark), that is, it is smaller than the Russell 1000 Value index, during a period in which the size factor returned 360 basis points per year. Manager C also gains four basis points by having more book-to-market exposure than the benchmark (0.34 vs. 0.33) that is, it is more value-oriented than the index, while the book-to-market factor returned 388 basis points per year. The manager gains 25 basis points by having greater market exposure, a market beta of 1.02 vs. 1.0 for the benchmark, during a period in which the market returned 1240 basis points. During this five-year period, all three risk factor returns were positive and the manager's greater exposures resulted in additional returns of 58 basis points (29 + 4 + 25) compared to the Russell Value index. In addition to the 58 basis points from style profile differences, the -29 basis point alpha of the Russell Value index makes Manager C look even better. Overall, benchmarking makes Manager C seem to be 87 basis points per year better than his true alpha of -39 basis points.
Manager D, a large-cap growth manager, has an alpha of -47 basis points computed from its custom benchmark. Benchmarked vs. the Russell 1000 Growth index, Manager D outperforms by 12 basis points, an overstatement of 59 basis points per year. The style profile of Manager D has larger betas than does the Russell 1000 Growth index. These larger betas add 160 basis points of return. The alpha of the Russell 1000 Growth benchmark happens to be 101 basis points.
Manager A, a small-cap value manager, has an alpha of 66 basis points per year, the largest alpha of the manager team. Benchmarked to the Russell 2000 Value index, this manager looks poor, 130 basis points worse than the index. Manager A's alpha is understated by 196 basis points by benchmarking. Compared to its benchmark, Manager A has less exposure to size, value/growth and the market and loses 50 basis points of return in the comparison. The Russell 2000 Value benchmark has an alpha of 146 basis points. This makes Manager A's performance look that much poorer. Overall, benchmarking makes Manager A look 196 basis points per year worse than he really is.
Manager B, a small-cap growth manager, has an alpha of -66 basis points per year but looks 68 basis points better when compared to his benchmark, the Russell 2000 Growth index. Compared to his benchmark, Manager B has less size and value/growth exposure thereby losing 144 basis points, but he gains 63 basis points from having more market exposure. The net loss due to style profile differences is -81 basis points. When we look at this entire manager team, only Manager A, identified by benchmarking as a poor performer, has a positive alpha. All of the others, identified through benchmarking as adding value, have negative alphas.
This example demonstrates why benchmarking provides low quality information for evaluating managers. Both the alpha of the index and the style profile differences vs. the manager must be recognized and taken into account for the evaluation to be uncontaminated. It's quite apparent that a manager of managers using benchmarking has been relying on low quality information and needs to re-evaluate the managers' alphas and reassess the structure of the manager team. Moreover, it is obvious from Exhibit 1 that we have selected funds with style profiles very similar to those of the benchmarks. The benchmark errors we observe in practice are usually much larger than those in this example.
An investment manager relying on benchmarking to identify his skills also has been using poor quality information. The index to which his performance is compared is most likely totally inappropriate for skill identification and leaves the manager with a false impression of his capabilities. It's one thing to agree to be evaluated by a certain index. It's another to have a false impression about one's strengths and weaknesses. If the firm strives for long-term success, resources must be intelligently allocated to its investment decision process and to its client servicing and new business efforts. Effective allocation can't be accomplished when poor quality information leaves management with a false impression of its capabilities.
Another major concern is misfit risk at the manager composite level. We originally chose managers whose style profile was close to that of the index used for benchmarking. We further allocated assets within the manager composite in line with the allocation of the Russell 1000 and 2000 indexes to the Russell 3000 index. Allocations are 90% to large-cap and 10% to small-cap.
These are then split equally between value and growth. We aggregate the individual manager style profiles and alphas. Manager E is the domestic equity composite with an alpha of -39 basis points per year. Benchmarking indicates the manager team added 49 basis points per year, an overstatement of 88 basis points.
What's going on here? To the investment committee relying on benchmarking, the small-cap value manager is underperforming his benchmarks but all other managers are outperforming and the manager composite overall appears to be doing well! The Russell 3000 index represents the target for this domestic equity composite. This index, identified as Manager M, has exposures on average of -0.02 to size, 0.00 book-to-market and 1.01 to market. The domestic equity composite has larger betas for each factor. Style misfit results in 114 basis points of additional return. The Russell 3000 alpha of 26 basis points is offset against the 114 basis points to leave a net benchmark error of 88 basis points.
Using the three-factor analysis, the investment committee now recognizes the composite outperformance of the target came not from good manager selection or superior team building but rather from having greater exposure to size, value and the market than the Russell 3000. This is a dramatically different scenario for the committee than the benchmark analysis that shows their composite outperforming by 49 basis points per year. It is important for a manager of managers and the investment committee to understand the skill of its managers and the misfit risk of its manager composite.
We conclude the evaluation of active managers using benchmarks that do not have the required characteristics always provides poor quality information with which to assess the manager's alpha characteristics, masks the desirability of including that manager in the lineup and impedes the formation of a superior team of managers.
What should be used is a custom benchmark with a zero alpha and a style profile identical to that of the manager being evaluated.
The technique we use meets these requirements.
Jim MacBeth is with EMA, a Dallas consulting firm, and George Richvalsky is with Richvest, a consulting firm in Short Hills, N.J