Irrational Optimism

Investors who assume stocks will give safe, favorable returns in the long run are suffering from irrational optimism. This article addresses the tendency of many investors to overestimate the rewards and underestimate the risks of investing in stocks over the long term—that is, investors’ irrational optimism. In particular, it examines the widely held belief that stocks are a “safe” investment for the long run. The probability of experiencing a real loss on equities depends on the expected real return and standard deviation of stocks. Judgments about the future magnitude of these two parameters typically involve extrapolating from history. In the study reported here, we used a unique global database of real equity returns from 16 countries during the 103-year period from 1900 through 2002 to confront the optimism of investors with the reality of history.Since 1900, as measured for all 16 countries, the worldwide real return on equities averaged close to 5 percent a year (before costs, fees, and taxes). This percentage is appreciably lower than is frequently quoted from historical averages.The difference arises because we used a longer time frame (1900–2002) than other studies and we adopted a global focus. Prior views have been heavily influenced by the U.S. experience, but the United States has been an exceptionally successful economy; so, not surprisingly, we found that U.S. stock returns have been somewhat higher than the average for the other 15 countries.Prior views on the long-run safety of equities have also been overly influenced by the U.S. experience. Furthermore, the U.S. evidence that over the long haul, stocks have beaten inflation over all 20-year periods is based on relatively few nonoverlapping observations and is hence subject to large sampling error. To counteract this dependency on projections of the U.S. experience, we examined the histories of other countries. We found only three equity markets other than the United States (with a fourth on the borderline) that never experienced a shortfall in real returns over a 20-year period. The worst 20-year real returns of 11 countries were negative. Indeed, historically, in 6 of the 16 countries, investors would need to have waited more than 50 years to be assured of a positive return.Ultimately, investors are concerned with the future rather than the past. Thus, in addition to examining historical data, we also analyze the future shortfall risk of an equity portfolio. The base case for the projections is a worldwide historical volatility level of 20 percent and mean real return of 5 percent. We also examine a lower return of 4 percent—on the grounds that financial history has been kind to investors, especially in the second half of the 20th century. Over that period, equity cash flows almost certainly exceeded expectations, and past returns were advantaged by upward revaluations. Stock markets have thus risen for reasons that are unlikely to be repeated.The projected shortfall risk exceeds the historical risk of shortfall—partly because of the lower assumed real returns (relative to the U.S. market) of 5 percent and 4 percent and partly because, even though volatility was projected to be the same as in the past, the shortfall analysis focuses on the full range of possible future returns rather than a single historical outcome. By construction, historical returns converged on long-term realized performance, but the forward-looking analysis shows that there is always risk from investing in volatile securities. Although the probable rewards from equity investment are clearly attractive, stocks did not and cannot offer a guaranteed superior performance over the investment horizon of most investors. Furthermore, their prospective returns are lower than the performance that many investors project whereas their risk is higher than many investors appreciate. Investors who assume that favorable equity returns can be relied on in the long term or that stocks are safe so long as they are held for 20 years are optimists. Their optimism is irrational.