In the July 2010 paper (download here from SSRN; acknowledgement to Stingy Investor where I found the link) On the Economic Costs of Index-Linked Investing, NYU prof and NBER research associate Jeffrey Wurgler reviews some research results that are disquieting for investors who follow a passive index strategy based on popular indices such as the S&P 500.
Wurgler says: "... the increasing popularity of index-linked investing may well be reducing its ability to deliver its advertised benefits ..." The problems:
- the inclusion of stocks in the index pushes up prices, by around 9% around the time of the event, a factor that has been getting worse as indexing has gained popularity; this effect is observed for other indices besides the S&P 500, like the TSX 300 (now the delicately named TSX Composite, which hides the fact the fact that it has been shrinking steadily in number of stocks over the years to 235 today)
- active managers who are benchmarked against the index have an incentive to overweight index members, even if they think a non-index stock will appreciate the same percentage, due to lesser tracking error
- stocks that join a leading index such as the S&P 500 suddenly begin to move much in tandem and keep doing so, which Wurgler vividly likens to the movements of a school of fish; he calls this effect "detachment"
- the S&P 500 school of fish members moves on their own and less like the overall market, a net loss of diversification for the investor
- S&P 500 membership has in the past over the long period of 1980 to 2005 conferred an increasing price premium; he cites one study that found the S&P 500 stocks got an 82 basis point annual alpha return premium; while it might seem like a good thing to get a hefty excess return, he says it might be a sign of an "indexing bubble" that will sooner or later deflate
- as a consequence, bubbles and crashes are more likely; he discusses the mechanism that may explain both the 1987 crash and the May 2010 flash crash
- the risk and return relationship actually does not hold - low beta(risk) stocks have been found to generate better returns, by a lot, than high beta stocks, a phenomenon he rightly calls a spectacular anomaly; he explains how fund managers benchmarked to an index will favour high beta stocks
- he raises the possibility that the S&P cap-weighted index amounts to a strategy of large-cap growth and momentum ... "Clearly, the line between passive and active investment is blurrier than usually presented."
What does he suggest one do about it?
- instead of the S&P 500, pick a broader index like the Wilshire 5000 - "The S&P 500 Index's detachment means, however, that it is reflecting less and less the performance of the full stock market. Index funds based on the more comprehensive Wilshire 5000 (which has included as many as 7,200 stocks) are now providing more robust diversification and stock market exposure."
- exploit the observed "spectacular anomaly" through strategies that focus on low-beta stocks, e.g. employ maximum Sharpe ratio, minimum volatility and absolute returns, though I'd guess that is probably beyond most individual investors' capability