Irrational Optimism

Abstract

We address the tendency of many investors to overestimate the rewards and underestimate the risks of investing in stocks over the long term - that is, investors' irrational optimism. In particular, we examine the widely held belief that stocks are a "safe" investment for the long run. The probability of experiencing a real loss on equities depends on the expected real return and standard deviation of stocks. Judgments about the future magnitude of these two parameters typically involve extrapolating from history. We use a global database of real equity returns from 16 countries during the 103-year period from 1900 through 2002 to confront the optimism of investors with the reality of history.

Since 1900, the worldwide real return on equities averaged close to 5 percent a year (before costs, fees, and taxes). This is appreciably lower than is frequently quoted from historical averages, a difference that arises because we use a longer time frame than other studies and adopt a global focus. Prior views on the long-run safety of equities have been overly influenced by the experience of the United States. Furthermore, the US evidence that, over the long haul, stocks have beaten inflation over all 20-year periods is based on relatively few nonoverlapping observations and is hence subject to large sampling error.

To counteract this dependency on projections of the US experience, we examine the histories of other countries. We find only three non-US equity markets (with a fourth on the borderline) that never experienced a shortfall in real returns over a 20-year period. The worst 20-year real returns of 11 countries were negative. Historically, in 6 of the 16 countries, investors would need to have waited more than 50 years to be assured of a positive return.

We also analyze the future shortfall risk of an equity portfolio. The base case for the projections is a worldwide historical volatility level of 20 percent and mean real return of 5 percent, and we also examine a lower return of 4 percent. The projected shortfall risk exceeds the historical risk of shortfall - partly because of the lower assumed real returns, and partly because, even though volatility was projected to be the same as in the past, the shortfall analysis focuses on the full range of possible future returns rather than a single historical outcome. By construction, historical returns converged on long-term realized performance, but the forward-looking analysis shows that there is always risk from investing in volatile securities.

Although the probable rewards from equity investment are attractive, stocks did not and cannot offer a guaranteed superior performance over the investment horizon of most investors. Furthermore, their prospective returns are lower than many investors project, whereas their risk is higher than many investors appreciate. Investors who assume that favorable equity returns can be relied on in the long term or that stocks are safe so long as they are held for 20 years are optimists. Their optimism is irrational.

Keywords: Portfolio management, asset allocation, long-run returns, shortfall analysis

JEL Classification: F30, G12, G15, G23, J26, N20