In 2016, Research Affiliates published a series of articles¹ challenging the “smart beta” revolution. We pointed out that, while there is merit in many factor tilt and smart beta strategies, performance chasing in these strategies—buying the popular outperforming strategies whose relative valuations are at extremely high levels—can be just as dangerous as performance chasing in other realms of asset management. We observe in factors and smart beta strategies that valuations matter just as they do in stock selection and asset allocation (i.e., lower relative valuations positively correlate with higher subsequent returns, and vice versa). 

In this article, the first in a series to be published in 2017, we attempt to measure the slippage between the factor returns realized by fund managers and the theoretical factor returns constructed from long–short paper portfolios, and potential reasons for this slippage, or performance shortfall. Theoretical concepts, such as long–short factor portfolios, although helpful in advancing our understanding of a subject, are typically idealized approximations of the real world, built on a foundation of simplifying core assumptions, which are usually implausible at best. 

We find that managers who favor high factor loadings for market beta, value, or momentum generally do not derive nearly as much incremental return, relative to low beta, growth, or contrarian funds, respectively, as factor return histories would suggest. Well over half of the factor return for market beta and for value (defined as HML) disappears, as does essentially all of the momentum factor return. By all appearances, Alice’s “Drink Me” potion, responsible for shrinking her so she can pass through the door to Wonderland, has found its way into real-world factor returns. 

Our 2017 smart beta series is called “Alice in Factorland.” Our next article in the series will challenge the idea that factor tilts—portfolios combining several theoretical factor portfolios—are smart beta. We will show that factor tilts cannot be used to replicate other smart beta strategies, using Fundamental Index™, equal weight, and low-volatility strategies as illustrative examples. The factor tilts of these strategies are easy to replicate, but the resulting portfolios look very different from the original, and the replication portfolios typically have far higher turnover, lower performance especially net of trading costs, and smaller capacity than the originals.  

In a third article, we will show that the relative valuations of factor loadings can give us the courage to buy mutual funds when their factor exposures are at their cheapest, hence, the most out of favor. Along with fees, turnover, and past performance—where low fees, low turnover, and low (yes, low!) past performance are predictive of better future returns—factor loadings can help us improve our forecasts of fund returns. We find the best predictor is prior three-year performance, but with the wrong sign: buying the losers is the winningest strategy.  

Finally, a fourth article will take a closer look at momentum, where we find the realized alpha in live portfolios is essentially zero compared to a theoretical alpha of around 6% a year. We show why momentum doesn’t work in live portfolios, and also show how momentum can be saved as a useful source of alpha.

We use data from Morningstar Direct Mutual Fund Database. The dataset reports historical monthly total returns for all mutual funds, including those liquidated or merged, which ensures our mutual fund dataset is largely survivorship-bias free. To form our fund sample, we select those funds from the US open-end long-only equity fund universe that have at least two years of return history as of December 2016. Our fund sample size begins in 1990 with 658 funds (392 not counting different share classes: A-share, no-load, and institutional),² peaks in 2008 with 3,800 funds, falling to about 3,000 funds in 2016. Our total US fund sample consists of 5,323 funds—a mixture of live funds and those that no longer exist today.

We begin our analysis in 1990 because, given the small number of unique funds prior to that date, our test to estimate multifactor premia could run into identification problems. Our main analyses use net-of-expense fund returns, which is how Morningstar Direct reports the data. One of our robustness tests (all of which are provided in the unabridged article) analyzes realized returns gross of expenses for the sample of funds that do report expense information. Information on fund expense ratios is not available for many funds, especially in the early years of our sample period, but for those with available expense data, results are essentially unchanged. 

Four factors—market, size, value, and momentum—are the most widely used in manager performance evaluation. We focus on these four and ignore the myriad more recent, sometimes exotic, factors populating the “factor zoo.”³ We also limit ourselves to these four because the Morningstar data start in January 1990, and to include factors identified after that date would create look-ahead bias. Because these four factors were well known by 1990 (or shortly thereafter), both the theoretical factor returns from paper portfolios and the investors’ realized factor returns measured from actual fund performance are largely out of sample. 

We calculate the theoretical factor performance by measuring factor returns using the method defined by Fama and French (1993): we construct a long–short portfolio, which is long the desirable trait and short the undesirable trait (e.g., the value factor is defined as a long value portfolio and a short growth portfolio). Over the last quarter-century, all four factors have positive performance.⁴ The market factor is the clear champion with an 8.2% annualized average return, followed by momentum at 5.7%. Value and size are well behind, with annualized returns of 3.6% and 2.6%, respectively. 

In practice, the returns generated by long–short paper portfolios are difficult, if not impossible, to replicate for a number of reasons. These paper portfolios ignore trading costs, which is of particular importance for factors with high turnover such as momentum. Second, more than half of the factor return typically comes from trading in small-cap companies, whose real-world trading costs are likely to be high. For most factors, nearly half the return comes from the short side of the portfolio, and shorting may be expensive or impossible for some of the intended short sales. Another real-world concern is missed trades.

Paper portfolios also ignore management fees, a direct and significant drag on investor performance. In practice, investors may not be able to transact in the market place at the same prices assumed by the long–short portfolio as a result of stale prices and bid–ask spreads, which can be magnified by large institutional-sized trades. Finally, the delisting bias documented by Shumway and Warther (1999)⁵ can further overstate performance for some factors. 

These sources of implementation shortfall are unlikely to affect the long and short portfolios equally, exacting for instance a greater penalty on small-cap versus large-cap stocks (size factor), on value versus growth stocks (value factor), and performance-chasing versus contrarian investing (momentum factor).

We measure factor return slippage, albeit with some imprecision, by comparing reverse-engineered factor returns, which are estimated from mutual fund returns, with the conventionally constructed factor returns. We follow the standard empirical method known as a two-pass regression procedure for estimating factor premia, which was introduced by Fama and MacBeth (1973). Instead of using theoretical portfolios with theoretical returns to estimate the factor premia, we use market-traded portfolios that capture the returns experienced by investors through net-of-expense mutual fund returns. A detailed description of our methodology is provided in the Appendix. 

Our approach to measuring actual returns, as realized by mutual fund investors, could face a few possible criticisms. First, funds may have time-varying factor loadings, while our method assumes static fund factor loadings. Our method may provide inaccurate return estimations (that is, the return captured by mutual fund managers), if managers frequently switch their styles. Also, if factor returns are driven by factor characteristics (e.g., price-to-book ratio for value, past return for momentum, and market capitalization for size), the factor loadings we derive from regressions may poorly capture the funds’ time-varying factor exposures.⁶ Perhaps, if the data were available, it would be better to measure the factor tilts directly, applying the same methods used to construct the paper factor portfolios. We recognize these are valid concerns and will address them in later work. It bears mention that these same criticisms equally apply to factor-based historical return attribution, which is used widely.

We find that the average factor premia captured by managers are significantly lower than suggested by the theoretical returns. This is true for all factors except size, for which managers capture a return 0.7% higher than theory indicates. The managers’ realized shortfall is −4.2% for the market factor, −1.4% for the value factor, and −5.2% for the momentum factor. 

The correlation between the two types of returns (actual versus theoretical) is in the range of 0.89 to 0.96, with the average being 0.92. The average correlation suggests that the monthly behavior of the two sets of factor returns closely match each other. We carry out several robustness checks to see if our results are unique to our initial project design. All of our robustness checks (available in the unabridged version of this article) produce results that support our core findings. 

Market Factor. Over the last 26 years, the factor return (net of transaction costs and other fees) earned by mutual fund managers by loading on market beta has fallen short of the observed equity market excess return by 4.2 percentage points a year. The shortfall has a t-statistic of −3.54. In the first decade of our sample period, the reverse-engineered returns match the theoretical equity excess return reasonably well, but the gap widens in the aftermath of the dot-com bubble, accelerating in recent years. For the market factor we have a reasonable explanation for the gap; for other factors the shortfall is harder to justify. 

The market factor we use in our analysis reflects the return difference between stocks and cash. Over long periods of time, high beta stocks have beat low beta stocks by far less than the capital asset pricing model (CAPM) would have predicted, which is nothing more than the well-documented flat (or inverted in some studies) security market line where differences in stock return performance are not explained by variation in market beta.⁷ So the gap we observe is not a surprise. Some might find it surprising that the relationship is positive at all. One possible explanation for a positive realized market premium is the cash held by fund managers (and also, potentially, their use of leverage and derivatives), which introduces the conventional market-minus-cash beta sensitivity.

Size Factor. The theoretical and manager returns for the size factor demonstrate a near-perfect fit, with a correlation of 0.96, over our study period. The good fit is not a surprise. The turnover of the size factor (i.e., stocks migrating between the large-cap and small-cap categories) is one of the lowest among all conventional factors, making the funds’ replication of the factor quite easy. The magnified effect—the realized size factor in mutual funds being stronger than the theoretical long–short size factor return—is, however, a bit of a surprise. 

Our findings suggest the cumulative return derived from mutual fund performance exceeds the theoretical long–short size factor returns by a small margin of 0.7 percentage point a year (not statistically significant, with a t-statistic of 1.13). The surplus return may come from the ability to control transaction costs because turnover is low in small-cap portfolios. Further, we cannot rule out that some active small-cap managers may have better stock selection skills than their large-cap brethren.

Value Factor. The value factor premium is perhaps the most widely studied factor across world markets because value strategies are among the most widely embraced investment solutions in finance.⁸ Our research indicates, however, that most value strategies, when executed in the real world, leave some of the value effect on the table. This gap between theoretical and realized returns is rather persistent over our study period, with the exception of the first year. Over the last quarter-century, value managers captured only about 60% of the value premium indicated by the long–short value factor. Whereas a theoretical paper portfolio generates an annualized return of 3.6% a year, mutual fund managers were able to capture only 2.2% a year, an annualized slippage of 1.4 percentage points, with a t-statistic of −1.38.  While not statistically significant, the shortfall is economically meaningful.

Momentum Factor. The average annual return of the momentum factor based on long–short paper portfolios is 5.7% compared to the annualized factor return captured by momentum investors, which is close to zero, at 0.4% a year per unit of momentum loading. This shortfall translates into an annualized slippage of 5.2 percentage points over the last quarter-century,⁹ with a t-statistic of −3.43! Most of the shortfall between the actual returns earned by fund managers and the theoretical paper portfolio happens by 2003, while essentially all of the alpha generated by the momentum factor paper portfolio occurs prior to 2002. 

For the long–short factor paper portfolios, our results show no slippage for the size factor, moderate slippage for the value factor, high slippage for the market factor, and very high slippage for the momentum factor. The sources of the observed slippage are the real-world costs of implementing in practice the theory we use to define a factor return: the costs associated with trading, shorting, relying heavily on positions in small and illiquid stocks, management fees, and so forth. 

How big are the transaction costs associated with the implementation of the different factors? Several studies (e.g., Novy-Marx and Velikov [2015] and Hsu et al. [2016]), show that low-turnover strategies, such as value and size, will incur small to moderate trading costs. The higher turnover strategies, such as momentum (and to a lesser extent low volatility or betting-against-beta), display trading costs that may be large enough to wipe out the alpha, if enough money is following the strategy. This order of magnitude in trading costs related to implementation of the different factors closely matches the respective amount of slippage we observe in our study. Although we cannot prove it, we suspect transaction costs likely play a major, even dominant, role in explaining the slippage in realized versus expected factor returns.

Another possible source of slippage could be manager skill in choosing the negative factor exposure. For example, if growth fund managers have strong stock selection or timing skill, this skill will reduce the performance gap between growth and value managers and will appear as an erosion in the realized value premium when compared to the theoretical value premium. We will explore in more detail the possible drivers of factor return slippage later in our series of articles. Meanwhile, caveat emptor!

Factor investing is gaining popularity in the investment community as practice catches up with academic research, inevitably enriching investors’ toolkits. But when investors use academic tools without a proper understanding of how real-world implementation costs—ignored in theoretical paper portfolios—can impact performance, a shortfall (sometimes large) between theoretical factor returns and realized factor returns can occur. 

We find that mutual fund managers over the period of our study, 1990–2016, experience substantial shortfalls in their ability to capture factor returns compared to the returns “earned” by theoretical paper portfolios. Our research shows the shortfall is quite large for the market and value factors, whose returns to the end-investor are halved or worse. For the momentum factor, the end-investor seems to have enjoyed no benefit whatsoever from fund momentum loadings, nor any penalty for funds that have an anti-momentum bias. We strongly suspect the majority of the return shortfall is due to trading costs, a topic we will explore in more detail in future work.