The Bankruptcy of Modern Finance Theory

Outcomes are easy to measure in financial markets: you either beat the index or you don’t. And the results could not be clearer about how few people possess any real skill.

A recent study found that 70 percent of actively managed funds have failed to beat their benchmark index and just 2.3 percent have delivered excess returns of more than 2.5 percent—and those are pre-fee numbers.¹ Fees make the picture even worse.

Trying to pick in advance which funds will be among the few that beat the market has proven just as difficult. Morningstar’s famed star ratings have a poor track record of picking winning funds. Brokerage accounts advised by financial advisers achieve lower net returns and inferior risk-return tradeoffs than self-directed accounts.² Even the expensive investment consultants who advise the world’s biggest money pools have shown limited ability to pick funds.³ The academic research now shows that the best metric for picking funds is the expense ratio: the less you pay experts to manage your money, the more you keep.⁴

Very few of the highly paid and well-credentialed professionals that run mutual funds and advise client investments actually add value. Yet clients still flock to actively managed funds and expensive advisers. There is, in finance, a massive calibration error: a gap between claimed expertise and actual results.

At the heart of this calibration error is bad theory. The profession of finance as practiced today relies heavily on modern finance theory: the dividend discount model and the capital asset pricing model. These models are the centerpieces of most business school finance courses, and surveys suggest that they are used by over 70 percent of CFOs for capital budgeting. They are also used by almost every fundamental investor and most financial advisers.

Yet these theories have been empirically invalidated time and again in well-publicized, peer-reviewed studies. Their continued usage represents another calibration error—between the claimed precision of the models and the actual unpredictability of markets.

John Burr Williams and the Dividend Discount Model

The intellectual foundations of modern finance lie in John Burr Williams’s 1937 Harvard dissertation-turned-book, The Theory of Investment Value (Harvard University Press, 1938), and in this ambitious man’s attempt to use scientific thinking to end the wild volatility of markets that had wreaked havoc on the country during the Great Depression.

Williams began his doctoral program at Harvard in 1932 with the goal of discovering the true causes of the crash of 1929. Just as the financial crisis of 2008 formed the first imprint on the minds of those of us who began our careers in finance in the past decade, the stock market crash formed an indelible impression on Williams and the cadre of young scholars who would attempt to invent the theory of finance. He believed that the “wide changes in stock prices during the last eight years, when prices fell as much as 80 or 90 percent from their 1929 peaks only to recover much of their decline later, are a serious indictment of past practice in Investment Analysis.”⁵

Like many in his day, Williams believed in the promise of technocratic governance, that the world’s problems could be solved by men of solid intellect meeting in wood-paneled rooms to weigh in on matters of urgency for society. The dividend discount model, and the application of The Theory of Investment Value, had a simple purpose: making the excess volatility of markets disappear by putting experts in charge of setting prices.

Experts, not traders, would set the prices of the securities in the marketplace. “The time seems to be ripe for the publication of elaborate monographs on the investment value of all the well-known stocks and bonds listed on the exchanges,” he wrote. “The last word on the true worth of any security will never be said by anyone, but men who have devoted their whole lives to a particular industry should be able to make a better appraisal of its securities than the outsider can.”⁶

In addition to making investing a far less exciting practice, Williams believed this new expert-based approach would result in “fairer, steadier prices for the investing public.” These experts needed only his formulas.

Williams’s new science of investing featured the now-familiar dividend discount model. In short, this model quantifies the idea that the investment value of a security is equal to “the present worth of the expected future dividends.”⁷ Williams laid this out in simple formula form:

Armed with this formula and a desired rate of interest, investors needed only a way to forecast future dividends. Williams believed he had a straightforward solution. The investor should “make a budget showing the company’s growth in assets, debt, earnings, and dividends during the years to come.”⁸ Rather than using an accountant’s ledger book, however, Williams proposed that investors use algebraic formulas instead. Williams believed that growth could be modeled using a logarithmic curve:

With this logarithmic model for growth (the familiar “S-curve” that Bain & Company consultants so dearly love) and a few additional formulas to provide the “terminal value” once competitive forces had stalled growth, Williams believed the true value of a company could be determined.

Combining a company’s budget with these new algebraic formulas offered an approach that Williams believed was “altogether new to the accountant’s art.” Williams grew almost giddy with excitement contemplating the beauty of his new technique. “By the manipulation of algebraic symbols, the warp of development is traced through the woof of time with an ease unknown to ordinary accounting.”⁹ One can only imagine what literary metaphors might decorate Williams’s prose had he lived to see the modern Excel model!

Yet as he waxed philosophical about the new art of algebraic accounting, Williams acknowledged that some might be skeptical of the supposed ease of his method. “It may be objected that no one can possibly look with certainty so far into the future as the new methods require and that the new methods of appraisal must therefore be inferior to the old,” he wrote. “But is good forecasting, after all, so completely impossible? Does not experience show that careful forecasting—or foresight as it is called when it turns out to be correct—is very often so nearly right as to be extremely helpful to the investor?”¹⁰

Williams acknowledged this key limitation of his model—uncertainty about the future. But this, he argued, was a problem for others, not the fault of his beautiful mathematical models. “If the practical man, whether investment analyst or engineer, fails to use the right data in his formulas, it is no one’s fault but his own,” he wrote.¹¹

Williams’s theories had grown out of the chaos of the Crash of 1929, and he retained a pessimism about the “practical men” that were his contemporaries. “Since market price depends on popular opinion, and since the public is more emotional than logical, it is foolish to expect a relentless convergence of market price towards investment value,” he wrote.¹²

But the generation of finance researchers emerging after World War II was more optimistic. The success of American industry—and American science—in defeating the Nazis inspired confidence in a new generation of researchers who believed that better math and planning could change the course of human affairs.

Markowitz, Sharpe, and the Capital Asset Pricing Model

Harry Markowitz was one of these men. Born in 1927, a teenager during the war, he studied economics at the University of Chicago in the late 1940s. After Chicago, Markowitz researched the application of statistics and mathematics to economics and business first at the Cowles Commission and then at the RAND Corporation. He was at the epicenter of the new wave of technocratic, military-scientific planning institutes.

Markowitz understood the key problem with Williams’s ideas. “The hypothesis (or maxim) that the investor does (or should) maximize discounted return must be rejected,” he wrote in his 1952 paper.¹³ “Since the future is not known with certainty, it must be ‘expected’ or ‘anticipated’ returns which we discount.”

Without accounting for risk and uncertainty, Markowitz observed, Williams’s theory “implies that the investor places all his funds in the security with the greatest discounted value.”¹⁴ This defied common sense, however, and so Markowitz set out to update Williams’s theory. “Diversification is both observed and sensible; a rule of behavior which does not imply the superiority of diversification must be rejected both as a hypothesis and as a maxim,” he argued.¹⁵

Markowitz’s explanation for why diversification was both observed and sensible was that investors care about both returns and about “variance.” By mixing together different securities with similar expected returns, investors could reduce variance while still achieving the desired return. Investors, he believed, were constantly balancing expected returns with expected variance in building their portfolios.

The logical next step in this argument was that the prices of different assets should have a linear relationship to their expected variance to maintain a market equilibrium where one could only obtain a higher return by taking on a higher amount of variance. This was the logic of the Capital Asset Pricing Model developed by Markowitz’s student William Sharpe.

Sharpe’s Capital Asset Pricing Model was meant as “a market equilibrium theory of asset prices under conditions of risk.”¹⁶ He believed that “only the responsiveness of an asset’s rate of return to the level of economic activity is relevant in assessing its risk. Prices will adjust until there is a linear relationship between the magnitude of such responsiveness and expected returns.”¹⁷ Sharpe measured responsiveness by looking at each security’s historical variance relative to the market, which he labeled its beta.

To be sure, not everyone from Williams’s era was so enthusiastically enamored with the prospect of experts planning the appropriate pricing of investments. At the same time as Williams was completing his 1937 dissertation, Friedrich Hayek was attacking this type of planning mindset. If society were turned over to experts and planners, he wrote:

change will be quite as frequent as under capitalism. It will also be quite as unpredictable. All action will have to be based on anticipation of future events and the expectations on the part of different entrepreneurs will naturally differ. The decision to whom to entrust a given amount of resources will have to be made on the basis of individual promises of future return. Or, rather, it will have to be made on the statement that a certain return is to be expected with a certain degree of probability. There will, of course, be no objective test of the magnitude of the risk. But who is then to decide whether the risk is worth taking?¹⁸

Alas, it would be another forty years before the notion of state planning by experts generally fell out of fashion and Hayek was given the Nobel Prize for his 1940s era work on price discovery in the broader field of economics. It is enough to note for our purposes that in financial economics many of the hangovers of the Williams/Markowitz/Sharpe era, and its religious faith in algebraic calculations and experts’ forecasting power, lasted well beyond the nightmare of the 1970s.

The Empirical Invalidation of Modern Finance Theory

The intellectual history of modern finance theory thus followed a simple logical flow. A stock is worth the net present value of future dividends. But investors must not only care about expected return—otherwise they would put all their money in one stock—they must also care about the variance of their investment portfolios. If investors want to maximize expected return and minimize expected variance, then expected variance should have a linear relationship with expected return.

This is the core idea: combining a forecast of future cash flows with an assessment of future variance based on the historical standard deviation of a security’s price relative to the market should produce an “equilibrium” asset price.

Williams, Markowitz, and Sharpe were all brilliant men, and their models have a certain mathematical elegance. But there is one big problem with their theoretical work: their models don’t work. The accumulated empirical evidence has invalidated nearly every conclusion they present.

Robert Shiller won the Nobel Prize for conclusively proving that the dividend discount model was a failure. In a paper for the Cowles Commission thirty years after Markowitz’s work was published, Shiller took historical earnings, interest rates, and stock prices, and calculated the true price at every moment of every stock in the market with the benefit of perfect hindsight. He found that doing this could explain less than 20 percent of the variance in stock prices. He concluded that changes in dividends and discount rates could “not remotely justify stock price movements.”

Below is a graph showing the actual future value of earnings discounted back at the actual interest rates relative to the actual stock market index over time. As you can see, the stock market index is far more volatile than could possibly be explained through the dividend discount model.

Future earnings and interest rates do not explain stock price movements as Williams thought they would. Variance also fails to explain stock prices, as Markowitz and Sharpe thought they would.

In a review of forty years of evidence, Nobel Prize winner Eugene Fama and his research partner Ken French declared in 2004 that the Capital Asset Pricing Model (CAPM) was bunk: “despite its seductive simplicity, the CAPM’s empirical problems probably invalidate its use in applications.”¹⁹ These models fail because they assume predictability: they assume that it is possible to forecast future dividends and future variance by using past data.

The core prediction of the model—that stocks with higher price volatility should produce higher returns—has failed every empirical test since 1972. Below is a graph from Fama and French showing CAPM’s predictions relative to reality.

A New Philosophy of Unpredictability

So what is the alternative? A philosophy based on unpredictability. As we have seen time and time again, experts cannot predict the future. The psychologist Philip Tetlock ran a twenty-year-long study in which he picked 284 experts on politics and economics and asked them to assess the probability that various things would or would not come to pass.²⁰ By the end of the study, in 2003, the experts had made 82,361 forecasts. They gave their forecasts in the form of three possible futures and were asked to rate the probability of each of the three. Tetlock found that assigning a 33 percent probability to each possible alternative future would have been more accurate than relying on the experts.

Think back to 2016: The experts told us that Brexit would never happen. They told us that Trump would never win the Republican primary, much less the general election. In both cases, prominent academics and businessmen warned of terrible consequences for financial markets (consequences that, of course, never came to pass). These are only two glaring examples of the larger truth Tetlock has identified: experts are absolutely terrible when it comes to predicting the future.

Stanford economist Mordecai Kurz argues that there are a variety of rational interpretations of historical data that lead to different logical predictions about the future (just think of all the different analysts and researchers currently putting out very well-researched and very well-informed predictions for the price of oil one year from now). But only one of those outcomes will come true, and when the probability of that one outcome goes from less than 100 percent to 100 percent, every other alternative history is foreclosed.²¹ Kurz has developed a model that explains 95 percent of market volatility by assuming that investors make rational forecasts that future events nevertheless prove incorrect.

The forecast error arises from investors’ inability to accurately predict the future. What looks inevitable in retrospect looks contingent in the moment, the product of what Thomas Wolfe called “that dark miracle of chance that makes new magic in a dusty world.”

In his 1962 classic The Structure of Scientific Revolutions, historian of science Thomas Kuhn posited that scientists rely on simplified models, or paradigms, to understand the observed facts. These paradigms are used to guide scientific enquiry until such enquiry turns up facts that cast these paradigms into doubt. Scientific revolutions are then required to develop new paradigms to replace the old, disproven ones.

The economists Harrison Hong, Jeremy Stein, and Jialin Yu have posited that markets function in a similar way.²² Investors come up with simplified models to understand individual securities. When events unfold that suggest that these simple models are incorrect, investors are forced to revise their interpretation of historical data and develop new paradigms. Stock prices move dramatically to adjust to the new paradigm.

Ever-changing paradigms are necessary because the world is infinitely complex, and forecasting would be impossible without simplification. Think of all of the many things that affect a stock’s price: central bank policy, fund flows, credit conditions, geopolitical events, oil prices, the cross-holdings of the stocks’ owners, future earnings, executive misbehavior, fraud, litigation, shifting consumer preferences, etc. No model of a stock’s price could ever capture these dynamics, and so most investors rely instead on dividend discount models married to investment paradigms (e.g., a growth investor who looks for companies whose growth is underpriced relative to the market combines a paradigm about predicting growth with a dividend discount model that compares the rate of that growth to what is priced into the stock). But both these dividend discount models and these investment paradigms are too simple and are thus frequently proven wrong.

Market volatility results from these forecasting errors in predictable ways that fit with our empirical observations of stock prices. Stock returns are stochastic, rather than linear, reacting sharply to paradigm shifts. High-priced glamour stocks are more apt to experience unusually large negative price movements, usually after a string of negative news. The converse is true for value stocks. As a result, stocks with negative paradigms outperform stocks with positive paradigms because, when those paradigms are proven wrong, the stock prices move dramatically. These empirical truths have been widely validated.²³

A more humanistic approach to finance suggests that the most intellectually honest approach—and the most profitable trading strategy—is to bet against the experts. This theory has the benefit of wide empirical support. Consider two natural conclusions that flow from a philosophy of unpredictability: preferring low-cost funds over high-cost funds and preferring contrarian strategies that buy out-of-favor value stocks. Evidence shows that there is a significant negative correlation between expenses paid to management companies and returns for investors.²⁴

In direct contradiction to the capital asset pricing model, value stocks significantly outperform growth stocks.²⁵ Large value stocks outperform large growth stocks by 0.26 percent per month controlling for CAPM, and small value stocks outperform small growth stocks by 0.78 percent per month controlling for CAPM. This is a significant and dramatic difference in return that stems entirely from betting on out-of-favor stocks that the experts dislike, and avoiding the glamorous, in-favor stocks that professional investors like.

These findings are consistent with an assumption of unpredictability. Those who try to predict fail, and the stocks of companies that are predicted to grow underperform the stocks of companies that are predicted not to grow. Growth is simply not predictable.²⁶

Broader Implications

The man who would go on to become the youngest recipient of the Nobel Prize in economics, Kenneth Arrow, began his career during World War II in the Weather Division of the U.S. Army Air Force. The division was responsible for turning out long-range weather forecasts.

Arrow ran an analysis of the forecasts and found that his group’s predictions failed to beat the null hypothesis of historical averages. He and his fellow officers submitted a series of memos to the commanding general suggesting that, in light of this finding, the group should be disbanded and the manpower reallocated.

After months of waiting in frustration for a response, they received a terse response from the general’s secretary. “The general is well aware that your division’s forecasts are worthless. However, they are required for planning purposes.”²⁷

Finance professionals rely on discounted cash flow models and capital asset pricing assumptions not because they are correct but because they are required for planning purposes. These tools lend social power to the excellent sheep of American society, the conventionally smart but dull and unimaginative strivers: the lawyers of the baby-boom generation and the MBAs of the millennial era.

These managerial capitalists are not trying to remake the world. They are neither craftsmen, pursuing a narrow discipline for the satisfaction of mastering a métier, nor are they entrepreneurs, pursuing bold ideas to satisfy a creative or competitive id. Rather, they are seeking to extract rents from “operating businesses” by taking on roles in private equity or “product management” where their prime value add will be the creation of multi-tab Excel models and beautifully formatted PowerPoint presentations.

Ironically, these theories were invented to avoid bubbles, to replace the volatility of markets with the predictability of an academic discipline. But corporate America’s embrace of empirically disproven theories speaks instead to a hollowness at the core—the prioritization of planning over purpose, of professional advancement over craftsmanship, and rent-seeking over entrepreneurship.

Harvard Business School and the Stanford Graduate School of Business are the Mecca and Medina of this new secular faith. The curriculum is divided between courses that flatter the vanity of the students (classes on leadership, life design, and interpersonal dynamics) and courses that promote empirically invalidated theories (everything with the word finance in the course title). Students can learn groupthink and central planning tools in one fell swoop.

Indeed, the problem of the business schools is so clear that a few clever wags on Wall Street now use the “Harvard MBA Indicator” as a tool to detect bubbles. If more than 30 percent of Harvard MBAs take jobs in finance, it is an effective sell signal for stocks.

Our capitalist economy offers every possible incentive for accuracy and the achievement of excellence. Why do people keep using bad theories that have failed in practice and have been academically invalidated? Not because these theories and methods produce better outcomes for society, but because they create better outcomes for the middle manager, creating a set of impenetrably complex models that mystify the common man and thus assure the value of the expert, of the MBA, of the two-years-at-an-investment-bank savant.

Active investment management does not work for investors because it was not designed to benefit them: it was designed to benefit the managers. The last decade of sclerotic economic growth has shown that, in many cases, corporate management teams are not working to benefit the shareholders, the employees, or anyone other than themselves.

This article originally appeared in American Affairs Volume I, Number 2 (Summer 2017): 72–83.

Notes