The Standard & Poor's 500, as a trading system (500 stocks, ranked and weighted by market capitalization, reweighted and ranked quarterly, etc.), may be fairly said to represent the returns from certain groups of stocks — e.g. large, high tech or food retailing. The Dow Jones industrial average, while also a U.S. index, follows a different set of rules (30 companies, price weighted, etc.). In Brazil, the Ibovespa uses trading volumes as a rule for setting weights. The most popular form of index construction is the market-cap-weighted index, but here, again, there are numerous different indexes: e.g. Nasdaq, MSCI World, Russell 3000, S&P 1500. Indeed, contradicting Harry Markowitz's fundamental insight of diversification underlying the concept of the “market,” the tendency in recent years has been to define ever more refined indexes comprising smaller numbers of stocks.
Each index, however, is a different trading system and there is an infinity of other trading systems applying to the same assets that do not represent accepted indexes.
In modern financial mathematics, the terms alpha and beta have gained great currency. The two mathematical symbols are used to represent the intersection of the y-axis and the slope of the regression line formed, when plotting the returns from the time series of an asset against the returns of an index representing “the market” over the same period. However, the observation that there is actually an infinite number of “trading systems” that can be formed from a set of assets makes it clear that there is no unique regression line and hence no unique definition of alpha and beta.
The quantities alpha and beta are only meaningful for an asset in relation to a particular trading system, of which an index is a special case. To infer that beta represents a skill-less return further assumes that there can be no skill in designing trading systems. This contradicts the fact that indexes are designed with some skill (for example to be broad, representative and investible) and are not simply arbitrary selections of investible stocks.
Originally stock indexes were not designed as investments, but as barometers of market sentiment and as a measuring tool. They have developed into widely used investments because over time the vast majority of investment managers has not outperformed them; disillusioned investors have responded by mimicking the indexes themselves and reducing fees to the minimum possible. This is an understandable but not wholly constructive response. The fact that an activity is difficult does not normally lead us to conclude that we should abandon it. It is more usual for it to act as a spur to human endeavor and entrepreneurial skill.
Furthermore, it is now widely accepted that, alongside their strengths, most stock indexes have one fundamental weakness as trading systems. This weakness is that the weighting of their returns is heavily skewed in favor of the largest stocks; typically 50% of the risk in index portfolios is in the top 10% of stocks and very little weight is in the smallest stocks. This is particularly damaging as there is strong evidence that smaller stocks have tended to produce higher returns and Sharpe ratios over both a wide selection of markets and a long historical period.
In “Triumph of the Optimists: 101 Years of Global Investment Returns” (2002), Elroy Dimson, Paul Marsh and Mike Staunton offer the most complete study of historical global market returns. Eugene Fama and Kenneth French's three-factor model, developed in the early 1990s reached the same conclusion while also finding support for the long-term outperformance of value stocks.
What has led to the overlooking of this great weakness has been the compensating strength that market-capitalization-weighted stock indexes require no rebalancing as price levels fluctuate; they are the minimum turnover and cost option. This probably accounts for their relatively good historical performance as they are extremely cost efficient to run.
Nevertheless, with powerful modern computer techniques it is possible to retain the advantage of low costs while rectifying the weakness of market-capitalization weighting by finding an appropriate trade-off between optimal portfolio design and costs.
The evidence of our research so far is that it is possible to produce trading systems that are significantly superior to traditional indexes, by formulating a mathematical approach in terms of forecast returns and risks, and not alpha and beta.