Returns-based investment style analysis, developed by William F. Sharpe and now marketed by several firms in software programs to pension sponsors, is untrustworthy, contends a study by Frank Russell Co.
The personal-computer-based analytics, nonetheless, is growing in popularity among pension sponsors and some consultants. The Florida State Board of Administration just initiated broad-scale use of a style evaluation method that includes returns-based analysis.
The 16-page Russell study by Jon A. Christopherson, an analyst for the Tacoma, Wash., consulting company, is the first serious analysis to criticize the reliability of the methodology.
Returns-based analysis uses only the returns of a portfolio to determine the investment style of an equity or fixed-income manager.
The study calls returns-based analysis unreliable in determining the investment style characteristics of a manager and the risks a manager bears.
Returns-based analysis methods "do not contain much more information about manager style than choosing a style index based upon reading a manager's brochure," Mr. Christopherson asserts in the study.
But the use of returns-based analysis, although still new and small, is growing in popularity and is becoming a powerful competitor to traditional consultants.
The $36 billion Florida pension fund will use StylScan, developed by SEI Corp., Wayne, Pa., which in part uses returns-based analysis, said William O. Bell, chief-management policy.
StylScan incorporates other methodologies in addition to returns-based analysis, to screen all its managers to categorize investment managers into clusters based on style and capitalization, he said.
"Differentiating among all potential managers in each asset class is extremely difficult because of the increasing complexity and growth of the investment business," said Mr. Bell.
"StylScan significantly cuts down on the amount of information needed to conduct an initial screen."
SEI executives couldn't be reached for comment.
Mr. Sharpe, who's familiar with the Russell study, said Mr. Christopherson's analysis is flawed. For example, he said, the Russell study fails to account for "effective" asset allocation, which is more important than actual asset allocation.
Mr. Sharpe said returns-based style analysis has a superb reliability. At pension sponsors with which he has worked, Mr. Sharpe said returns-based style analysis has explained 97% of the source of returns, effectively the styles of the portfolios.
"It's quite remarkable how much information you can get with so little effort" by using returns-based analysis, Mr. Sharpe said.
Because it is easy and inexpensive, the returns-based analysis software programs independently developed based on Mr. Sharpe's methodology are gaining popularity among pension executives. Returns-based style analysis is making inroads on traditional consultants and their fundamental approach to analyzing issues because, with personal computers, sponsors can evaluate money managers without having to use an outside consultant.
Zephyr Associates Inc., Zephyr Cove, Nev., was the first to market a returns-based analysis software in 1992 and the biggest currently in terms of clientele. Zephyr has sold the software to more than 30 pension funds and other tax-exempt institutional investors, including AT&T Corp., Brunswick Corp., California Public Employees' Retirement System, Georgia Division of Investments, Rockefeller Foundation, Virginia Retirement System, United Technologies Corp., Digital Equipment Corp., Illinois State Board of Investments, MacArthur Foundation and Stanford Management Co., which runs Stanford University's endowment.
More recently, BARRA Inc., Berkeley, Calif.; Ibbotson Associates Inc., Chicago; and Northfield Information Systems Inc., Boston, have each begun selling their own versions of returns-based analysis software, based on the Sharpe methodology.
Mobius Group, Research Triangle Park, N.C., while it doesn't sell software, uses returns-based analysis to analyze the style of money managers in its investment manager database, providing the analysis in book form to clients.
Mr. Sharpe - who said he has no financial interests or involvement in the firms developing any of the software based on his methodology - said returns-based analysis assumes the pattern of returns of a portfolio imply a certain set of exposures, or portfolio characteristics.
In other words, the assumption is that managers pigeonholed to a particular style pick stocks from the same universe.
Mr. Sharpe acknowledges that returns-based analysis often will imply a manager has a portfolio holding certain securities the manager in fact never has bought. The implied holdings are the portfolio's "effective" investments, which he said is more important than knowing its actual holdings.
He said, for instance, a returns-based analysis might imply, say, a portfolio held some foreign bonds, when in fact the manager has never bought foreign bonds. But Mr. Sharpe said some of the domestic bonds in the portfolio may be sensitive to foreign interest rates movements, an important characteristic that return-based analysis picks up.
Returns-based analysis also would show a portfolio of utility stocks has an effective allocation to fixed income, because such stocks are sensitive to interest rates.
Returns-based analysis shows the true "economic characteristics" of a portfolio, he added, saying those are significant analytical tools. "It's not what they (managers) hold that's important," Mr. Sharpe added, "it's the true economic relationships."
Returns-based analysis helps the institutional investors reduce the dimensionality of the investment task," Mr. Sharpe added.
"This whole thing has revolutionized the consulting industry. It has made it so much easier to analyze portfolios," said R. Steven Hardy, Zephyr president.
In his study, Mr. Christopherson says returns-based analysis methods "have limited capacity to forecast manager's future returns and, consequently, their risks. Yet, knowledge of risk and return is the primary reason we are concerned with style."
In an interview, Mr. Christopherson said returns-based analysis often gives spurious correlations, showing a manager's style indicates a portfolio holds, say, international bonds, when in fact the portfolio has never held them.
To test the usefulness of returns-based analysis, Mr. Christopherson sampled 180 managers, 20 of whom were small-cap managers, and found it was wrong more than 50% of the time in classifying small-cap managers, using a five-year window.
In another test of the usefulness of returns-based analysis, Mr. Christopherson put together a random sample of 100 stocks from the Russell 2000 index of smaller capitalization stocks and used their returns to determine the style of the portfolio based on the Sharpe methodology.
Mr. Christopherson found that 16 stocks, or 16% of the portfolio, correlated more closely with the Russell 1000 index of larger-cap stocks, suggesting the analysis placed much of the portfolio in the wrong style universe.
A more traditional deconstruction of the portfolio by securities and their characteristics showed it was small-cap, the study says.
Mr. Christopherson said the returns-based analysis placed much of the portfolio in the wrong style because of the "noise in the data."
"The large-capitalization behavior of some small stocks occurs because over the time period tested, these stocks happened by chance to correlate more highly with a large-cap stock index than they did with a small-cap stock index," he argues in the study.
Returns-based analysis "cannot tell the difference between noise and true factor exposures," the study says.
In addition, the study contends returns-based analysis - or what he also calls "correlation analysis" - "tends to be blind to style dynamics," that is, shifts or drifts in style.
One reason is that in using a typical 60-month measurement period of returns-based analysis, the study says, "the most recent time point is as important as the most remote time point."
"Inevitably, there is a delay in the recognition of a style change," the study adds.
The study contends the more fundamental methods used by Russell and other traditional consulting firms of analyzing the securities in a portfolio for their fundamental characteristics and talking with portfolio managers is the best way to ensure a manager is placed in its correct investment style.
But Mr. Hardy, who read a preliminary draft of the study, said: "People don't care what stocks or bonds are in a portfolio. They hire managers to get a certain returns distribution.
"You hire a value manager to get a different distribution than a growth manager. You don't care about the particular stocks.
"The manager could use derivatives to get a certain distribution.
"People buy portfolios to get returns, not characteristics," Mr. Hardy continued, referring to fundamental characteristics, such as low price-earnings ratios, or high earnings-per-share growth rates.
Mr. Hardy, along with others, contend PC software is making pension executives less reliant on their consultants to do analytics.
"The relatively inexpensive returns-based programs are hurting consultants' business," Mr. Hardy added.