As the scorecard we use to determine if we've succeeded or failed, the evaluation of investment performance is extremely important to all investors. Every day, significant decisions are made on the basis of performance evaluation. Policies are re-examined, strategies are altered, managers are hired and fired. Yet, despite its importance, investment performance evaluation remains in the Dark Ages, especially in light of the technological revolution we're enjoying as we enter the 21st century.
In the early 1970s, I developed a large performance evaluation system for A.G. Becker Inc., the premier pension consulting firm of its day. This system encompassed all of the number crunching (rates of return, risk measures, portfolio profiles, etc.), universe creation, attribution analytics and associated presentations that many of us use today. Now my software probably will be discarded because it is not Y2K compliant. I say good riddance. It was never intended to last this long.
We've learned a lot in the past 30 years that makes many of the old ways obsolete, and the technological revolution enables us to do things we couldn't three decades ago. Back then, some of what I'll describe here would not have fit on huge mainframe computers, but today it easily works on a desktop computer.
Before I describe the lessons we've learned in this information age, it's important to think about why these lessons haven't made their way into the contemporary toolboxes of performance evaluators, namely, plan sponsors and consultants.
Here's the major reason: Fear of change. We've learned a lot, especially in the past 10 years, and some plan sponsors and consultants already have put some of these lessons to work. However, I believe the performance evaluation industry generally has clung, and will continue to cling, to the old ways mainly because of comfort and fear of the unfamiliar.
There's also the economic reason that many evaluation firms have built an infrastructure based on antiquated technology these firms are now reluctant to abandon. In the future, this infrastructure will be available on the Internet, so sponsors and consultants will have a veritable smorgasbord of great ideas from which to select. As with all things in nature, the fit will adapt, and the weak will be left behind.
So now here's what we've learned (see if you agree):
* Investment policy is the most important determinant of investment performance.
* Managing managers, or policy management, has a big effect on performance. Deviations from policy targets, albeit within policy guidelines, are the responsibility of policy managers, namely, pension or other fund sponsors.
* Style is probably the third most important factor explaining performance.
* Peer groups make awful backdrops for performance evaluation. Everyone who has studied for the Level 3 Chartered Financial Analyst examination has learned this lesson. As one of the highest designations in the industry, CFAs learn the flaws of peer groups.
* Appropriate benchmarks might be better than peer groups, but you have to wait 100 years for the statistics to sort out because it takes this long for the value added of even a skillful manager to become statistically significant.
* Performance reporting standards, such as those of the Association for Investment Management and Research, improve the quality of information, but unless other lessons listed above are brought to bear, we're no smarter than we were without the standards.
* Managing to objectives is more sensible than managing to beat the other guy, because we all have different needs and emotions.
Although tools have been developed that incorporate these lessons, they still haven't found their way into most sponsors' and consultants' toolboxes for reasons cited above. So most plan sponsors are not benefiting from them.
For starters, the two most important components of performance -- policy and policy management -- should be calculated routinely, and their contributions to overall risk also should be monitored. The policy and policy management components commonly explain most of performance.
What's left is manager effects. Tools to incorporate all of the lessons we've learned about evaluating manager effects include a modern replacement for peer groups and benchmarks, and attributions that separate style from skill in sector and stock selection. While benchmarks solve the problems with peer groups, they introduce a new problem of their own: You have to wait 100 years to accumulate sufficient manager performance history to ensure the accuracy of your results.
This waiting problem is solved by computer simulations that cyberclone a manager over and over again by creating all of the portfolios that could be held following the manager's portfolio construction rules.
The result is a peer group without the numerous biases associated with old-fashioned approaches. Statistical significance is determined in the cross-section rather than across time, thus resolving the waiting problem.
In a poetic sense the manager is hoisted with his own petard. Cyberclone peer groups have been available for about five years, but the industry continues to embrace the old methods despite their documented deficiencies. Once performance is deemed to be a success or failure using cyberclone peer groups, the reasons for either are determined through performance attribution. While old-fashioned attributions use economic sectors to make these determinations, we know today that most managers manage to style rather than economic sectors. So style-based attributions make much more sense.
As for managing to objectives, you may wonder how the other guy did, but -- in absolute terms -- it really doesn't matter much since he doesn't walk your walk. Here again, sponsors and consultants seem to feel a special attachment to peer group comparisons, despite their lack of meaning.
These lessons are important and should be applied as we enter a new century. Otherwise nothing will have been learned.
Ronald J. Surz is managing director of Roxbury Capital Management Inc., Santa Monica, Calif. He has developed performance evaluation software, found at www.PPCA-Inc.com.