<!-- Swiftype Variables -->


Some new ideas in financial mathematics

David Harding is the founder, chairman and head of research at Winton Capital Management, London.

Financial mathematicians have built an increasingly elaborate structure around the idea of “the market,” the concept of passive investing and the notions of beta — the return that the market can give to investors with no investing skill, and alpha — the idiosyncratic returns that can be earned by only skillful investors.

In this article, I intend to challenge some of these foundational concepts with the intention of destabilizing the intellectual structure that has been erected on top of them and to demonstrate how a randomly generated portfolio can beat “the market,” as this term has commonly come to be used.

In actual fact, the stock market indexes that are usually described as “the market” are trading systems.

The definition of a trading system is a set of rules and definitions that can be used retrospectively to specify an unambiguous set of portfolio holdings and actions, such that a time series of historic returns for that system can be constructed. This perfectly applies to any modern stock index.

However, this viewpoint is distinctly at odds with the idea that the index represents “the market.” In fact, there are numerous different indexes, none of which is — or can ever be — the market.


The Standard & Poor's 500, as a trading system (500 stocks, ranked and weighted by market capitalization, reweighted and ranked quarterly, etc.), may be fairly said to represent the returns from certain groups of stocks — e.g. large, high tech or food retailing. The Dow Jones industrial average, while also a U.S. index, follows a different set of rules (30 companies, price weighted, etc.). In Brazil, the Ibovespa uses trading volumes as a rule for setting weights. The most popular form of index construction is the market-cap-weighted index, but here, again, there are numerous different indexes: e.g. Nasdaq, MSCI World, Russell 3000, S&P 1500. Indeed, contradicting Harry Markowitz's fundamental insight of diversification underlying the concept of the “market,” the tendency in recent years has been to define ever more refined indexes comprising smaller numbers of stocks.

Each index, however, is a different trading system and there is an infinity of other trading systems applying to the same assets that do not represent accepted indexes.

In modern financial mathematics, the terms alpha and beta have gained great currency. The two mathematical symbols are used to represent the intersection of the y-axis and the slope of the regression line formed, when plotting the returns from the time series of an asset against the returns of an index representing “the market” over the same period. However, the observation that there is actually an infinite number of “trading systems” that can be formed from a set of assets makes it clear that there is no unique regression line and hence no unique definition of alpha and beta.

The quantities alpha and beta are only meaningful for an asset in relation to a particular trading system, of which an index is a special case. To infer that beta represents a skill-less return further assumes that there can be no skill in designing trading systems. This contradicts the fact that indexes are designed with some skill (for example to be broad, representative and investible) and are not simply arbitrary selections of investible stocks.

Originally stock indexes were not designed as investments, but as barometers of market sentiment and as a measuring tool. They have developed into widely used investments because over time the vast majority of investment managers has not outperformed them; disillusioned investors have responded by mimicking the indexes themselves and reducing fees to the minimum possible. This is an understandable but not wholly constructive response. The fact that an activity is difficult does not normally lead us to conclude that we should abandon it. It is more usual for it to act as a spur to human endeavor and entrepreneurial skill.

Furthermore, it is now widely accepted that, alongside their strengths, most stock indexes have one fundamental weakness as trading systems. This weakness is that the weighting of their returns is heavily skewed in favor of the largest stocks; typically 50% of the risk in index portfolios is in the top 10% of stocks and very little weight is in the smallest stocks. This is particularly damaging as there is strong evidence that smaller stocks have tended to produce higher returns and Sharpe ratios over both a wide selection of markets and a long historical period.

In “Triumph of the Optimists: 101 Years of Global Investment Returns” (2002), Elroy Dimson, Paul Marsh and Mike Staunton offer the most complete study of historical global market returns. Eugene Fama and Kenneth French's three-factor model, developed in the early 1990s reached the same conclusion while also finding support for the long-term outperformance of value stocks.

What has led to the overlooking of this great weakness has been the compensating strength that market-capitalization-weighted stock indexes require no rebalancing as price levels fluctuate; they are the minimum turnover and cost option. This probably accounts for their relatively good historical performance as they are extremely cost efficient to run.

Nevertheless, with powerful modern computer techniques it is possible to retain the advantage of low costs while rectifying the weakness of market-capitalization weighting by finding an appropriate trade-off between optimal portfolio design and costs.

The evidence of our research so far is that it is possible to produce trading systems that are significantly superior to traditional indexes, by formulating a mathematical approach in terms of forecast returns and risks, and not alpha and beta.

Market conundrum

To illustrate this, consider the central tenet of the Efficient Market Hypothesis; namely an investor cannot consistently beat the market. Put more crudely, the EMH suggests that throwing darts at the financial pages will select a portfolio of shares that will perform equivalently in the long term to one carefully selected by experts. Of course the opposite is also true: this randomly selected portfolio can't beat the experts or the market, either.

I do not believe this to be a reasonable description of the realities of investment management. So we attempted to show how a randomly selected portfolio can beat the market (represented here by the S&P 500) over the long term.

We at Winton conducted the following experiment. Every month, we selected 100 stocks from the S&P 500 (using a random-number generator) and held them for the following month. Obviously, there are many different combinations we could choose (for any one month there are more than 10100), and so we repeated this process to produce 1,000 different portfolios, each holding a significantly different selection of stocks at any one time.

We found that, on average, the random portfolios beat the market in 68% of the 46 years between 1965 and 2011 (see Figure 1) leading to an average yearly outperformance of more than two percentage points. Over the full 46-year period, even the worst-performing portfolio was still up on the market.

So what was the secret of this success? Obviously, it was not the stocks that were chosen. Instead, it was the weights with which they were held. The advantage came from using a more optimal system to select these weights than the index. Even using a random portfolio, this system still achieved significant long-term benefits over the market.

Repeating the experiment, we found portfolios that used a market-value weighting system outperformed the market index in only 50% of years (see Figure 2), no better than chance, and on average show no sign of outperformance in the long term.

Obviously, picking stocks at random makes for a difficult sales pitch. Also, the costs of following the strategy outlined here — where we build our portfolio from scratch each month — are prohibitive. However, a more realistic approach, captures the benefit of an improved weighting scheme while keeping costs low. Significant rewards exist for investors who choose the right weighting scheme.

In summary, I believe the innovations of the 1970s and "80s such as CAPM, alpha and beta — which started off being such useful intellectual tools — are now in danger of becoming obstacles to further innovation in financial mathematics. I would argue that too much current research effort, both academic and commercial, in this field has become to paraphrase John Maynard Keynes enslaved to some defunct, or not even defunct, economist.

Random portfolios vs. the S&P 500


Market-value-weighted portfolios vs. the S&P 500