Commentary: Hedge fund indexes can be incredibly valuable — if you know what's in them

When evaluating hedge fund performance, a quality index can serve as a meaningful benchmark. The index you choose may help determine your overall asset allocation or be the difference between investing in or redeeming from a manager. But quality indexes have historically been few and far between, leaving allocators with a very limited and often misleading visibility into the returns different managers are able to generate.

Let's illustrate with a quick example. When you are looking to buy a car, you probably review several high-profile websites that contain listings of available makes and models that fit your search criteria. But what if these websites each omitted a significant portion of the total car market? And what if your limited data sample was itself not accurate and complete in terms of price and availability? And lastly, imagine if each website presented its information as comprehensive and accurate.

Would you buy a car based on flawed data? Unfortunately, this is a daily experience for institutional investors, and hedge fund allocators in particular.

Historically, the only thing an asset owner could do to address these shortcomings was to build indexes or peer group benchmarks themselves. That requires substantial resources and technical knowhow. Other than a few sovereign wealth funds and large pension plans, few have even tried. Additionally, even with the best intentions, internal indexes lack independence and incorporate biases of their own.

So, how can an investor determine whether an index is high quality and serves as a meaningful benchmark? Based on our experience, it comes down to four questions:

1. What are the data that go into it?
2. What is the methodology used to construct it?
3. How is the information presented?
4. Is there more to the index than just the average?

To revisit the car analogy, let's say you were in search of the best electric vehicle and decided to use one of the leading car research apps for your search. But as you dig in, you learn that the app doesn't include Tesla Inc., Toyota Motor Corp., BMW Group or Honda Motor Co. — simply because those companies choose not to provide their information.

In this example, the analysis would be incomplete and missing many of the established industry leaders. As a result, your view of the electric vehicle market could be significantly distorted. Technically, there would also be a negative selection bias regarding companies that choose to provide their information. In other words, you're not only missing the full market, but you're also experiencing a bias toward companies that choose to provide their information compared to those that don't.

While this seems ridiculous when searching for a car, institutional allocators experience these same issues in the hedge fund industry every day. Trillions of dollars of hedge fund assets are often benchmarked to indexes that not only have material gaps in their data but may also be missing many of the highest quality funds. This not only means that the index likely creates an artificially low bar but apples to oranges comparisons as well. Just like any poll or survey, if the sample is not representative of the population, then the results have much less value.

This topic is too large and technical to comprehensively cover here, but let's focus on one major issue: the data collection exercise. Many hedge fund indexes often rely on the generosity of managers that opt into their indexes. However, even those managers that agree to provide their data in general, often disagree on when to provide their information. In fact, many hedge fund marketers decide each month whether to post their returns to an index portal at all.

An extreme example would be March 2020, the height of the initial wave of COVID-19. Many hedge funds, especially those that suffered large losses, chose not to provide their return data to index providers on their regular schedule. Some waited weeks until after they communicated with their investors, while others waited months to see if their returns would rebound. Some stopped reporting altogether.

Monthly decisions by managers to withhold their data, especially during times of poor performance, can create what's called a short-term survivorship bias. This survivorship bias artificially enhances and smooths performance in ways that is not representative of the index's normal sample of reporting funds.

And depending on the rules of the index provider, an index may be constantly revised in the future because those who rebounded begin reporting once again — some filling in their historical return gaps, while others leave gaping holes. This makes reconciling performance relative to indexes that much harder, as they are in a constant state of change. In addition, these short-term irregularities in index performance, when combined with longer-term selection biases, can add up even for investors with long-term horizons, as their asset allocation models rely on accurate and representative monthly data.

Hedge fund performance is most commonly provided monthly. Accordingly, hedge fund indexes usually follow the same monthly cadence. Based on the liquidity of the strategy, it is reasonable that hedge funds provide their previous month's returns to index providers throughout the following month. The less liquid the strategy, the longer the returns usually take to report. However, index providers often race to be the first to publish each month, regardless of whether their numbers are stable.

Most hedge fund indexes are sent out without reference to the number of funds, percentage of those likely to report, how much has already changed, and how much it is likely to change going forward. Consequently, investors are often frustrated by constant index fluctuations, at times material, from day to day and throughout the month.

Without some level of transparency into the stability of the number reported, it is just that: a number. And one that is hard to provide confidence in for investors who are trying to benchmark their managers and communicate performance to their clients or board/investment committee.

When you take the average of a large distribution, the best performers are blurred in with the rest and become indistinguishable. Further, the broader the index, the more the average may hide what's happening in the distribution. For example, the mean return for a hedge fund composite index that includes many unique substrategies would not paint the full picture.

Accordingly, given hedge funds' abilities to execute numerous unique strategies, lumping them into one summary statistic is almost always inadequate. It could generate sweeping conclusions about a fund or even the industry which, by design, would underestimate the value of its parts.

It's always important to remember that what's true in the aggregate tells you nothing about the distribution around the mean. An index that reports a return over some time horizon, is merely providing the average of all the fund returns that have been given to them at that point in time. Having the ability to break down performance not only by cohort and quartiles with transparency at the constituent level, can help ensure you evaluate your hedge fund investments in the right context. It may even help you separate the best from the average.

Evaluating hedge fund performance is difficult. While indexes can add important clarity, not all indexes are created equally.

Knowing what's in the index, how it was constructed, the process for gathering data and how the information is presented can help investors know whether an index is a good benchmark.

Jon Caplis is the founder and CEO of PivotalPath Inc., a hedge fund consultant and analytics company, in New York. This content represents the views of the author. It was submitted and edited under Pensions & Investments guidelines but is not a product of P&I's editorial team.