Spotlight - “On Average” … The Last Refuge of Poor Analysis?
Investors should be wary of statistical averages that promise certainty in a complex market environment.
By Kerry O'Boyle, Aston Asset Management
Most investors are likely familiar with the pattern. A market commentator or news reporter blithely throws out a statistical average (“The average large-growth fund manager underperformed in 2011” or “Active managers, on average, lagged their benchmarks during the quarter.”) meant as proof of what went wrong for investors in the recent past. They then go on to advocate a recommended solution or course of action that investors can follow to avoid a similar fate—typically involving some kind of passive investing approach or macro analysis of what sector/strategy in which to invest. Little time is spent, however, in explaining or assessing how the average that serves as the basis for the entire story was derived. It’s just assumed that all-encompassing statistic holds some powerful meaning that investors can learn from. But like the story of the statistician who drowned crossing a river that was, on average, only three feet deep—are simple averages of broad categories of data really meaningful to investors?
What Does “Average” Mean?
Most people, investors included, have only a rudimentary grasp of statistical methodologies and terminology. For most of us, an average is the middle or typical value of a data set (or in fancier language, a measure of central tendency). Often left unrecognized, or unsaid, is that there are many different measures of central tendency to choose from. When talking about an average, are we signifying the arithmetic mean (sum divided by the number of data points), the median (the middle value when ranked in order), or mode (value that occurs most frequently)?
Unless otherwise stated, when an average is given it is generally understood that we are referring to an arithmetic mean. This typically works fine when operating with a defined and relatively narrow set of data points, such as the height of a group of people, but the weakness of an arithmetic mean is that it can be greatly influenced by outliers. In other words, to be most meaningful an arithmetic mean must be taken from a normal, symmetric distribution (i.e. a bell curve) of data points. Given significant outliers or a skewed distribution and the arithmetic mean doesn’t accurately convey what most would consider the “middle” or central tendency.
In recent years, investors have become well aware of the dangers of assuming bell curve distributions. Talk of “fat tails” and “outlier risk” has become prominent in market thinking after two market crashes within a decade has put many academic theories based on normal distributions to the sword. Just as investors should be wary of the potential for outlier events in the market, they should also not become beholden to statistical measures that mask the effects of skewed distributions or outliers.
Better Than Average
One only needs to look at the example mentioned above of the performance of large-growth mutual funds in 2011 to see how a seemingly simple statistic can be anything but. Per Morningstar, the average return for its Large Growth category in 2011 was -2.46%.1 Out of 425 actively managed funds with full year returns (excluding index funds, exchange-traded funds, and Fund of Funds), however, 245 (58%) outperformed that average, and only 180 did worse. This uneven distribution about the mean results in significantly more than half of funds being “above average” which doesn’t jibe with the typical concept of average as being near the middle.
Such skewed distributions often lead people to consider the median instead as a guide, but this can fall meaningfully short as well. The Morningstar Large Growth category’s median return (213th ranked fund) of -1.57% in 2011 guaranteed that half the funds did better, and half worse, but wasn’t necessarily any better an indicator of the typical investors’ experience. The reason is that the median, like the arithmetic mean, gives no indication of the range of outcomes involved. Averages ignore outliers and smooth out the extremes. Consider that the top-performing large-growth fund that year gained 14.6% while the worst performer returned -26.3%. Such a wide dispersion absent further context tends to render simple statistical averages less meaningful to the typical investor.
Adding other factors into the mix can quickly change how one interprets the data and its meaning by providing context to the results. For example, what if the largest funds in terms of assets are among those that outperformed? More assets generally mean more investors with results that were better than average. What if many of the poor performing funds in 2011 had significantly outperformed in 2010? Investors with longer investment horizons than a year may likely still be ahead, and may expect or be more willing to accept a lackluster year of performance as part of a long-term strategy.
Additional information often highlights the shortcomings of using simple averages in multi-variable situations over arbitrary periods (make no mistake, generic time periods that don’t represent actual holding periods are arbitrary). Frequently lost in the use of averages is the one thing that truly matters—the only fund that matters is the one that you own. Simple averages rarely say anything about the qualitative choices or results of each individual investor’s experience.
The failure of statistical averages to effectively capture the experience of the typical investor is at the heart of their misuse by market commentators. Averages are used to connote typical investor results despite the lack of compelling evidence that they do so in a meaningful way. Yet simple averages continue to be used frequently in attempts to promote seemingly universal truths about a whole host of subjects where historical data is widely available and has become increasingly accessible through technology.
Promoters of these simple analyses are often persuasive because most of us have a strong desire for a precise answer. We are uncomfortable with uncertainty, and are willing to overlook a great deal for the illusion of certainty. The use of statistics and quantitative data add an aura of authority to commentators above and beyond that of mere opinion. Yet, the ability to manipulate data based on assumptions and the selective selection of certain variables makes statistical analysis as subjective as qualitative opinions in most cases.
In any large-scale, complex, and highly uncertain environment (such as financial markets) averages rarely if ever represent any kind of Truth. Statistical analysis is nothing more than an interpretation of historical data with an eye towards finding better answers to uncertain situations and improving decision-making. Simple averages are typically one person’s poor attempt to interpret data without nuance or perspective in the hopes of coming up with a single, certain data point to provide meaning. But ignoring the context of highly complex and uncertain environments, involving multiple and often independent variables, misleads as often as it instructs.
The Last Refuge
When Samuel Johnson spoke the famous line, “Patriotism is the last refuge of the scoundrel” in 1775, it was widely believed that he was not critiquing patriotism, but the pretensions of self-proclaimed patriots. Similarly, investors need to guard against falling under the sway of simple financial statistics trumpeted by market commentators promoting various investing theories or practices. Not because they are scoundrels, but because they often assert a degree of authority and certainty unwarranted in such a dynamic environment. Averages and other performance or market statistics rarely capture anything more than a highly context-dependent snapshot of events or a measure of temporary trends. Although these characteristics can provide insight into past trends and guide various interpretations of current events, one must be careful not to associate any permanence to what they may reveal.
Investing is an art, not a science. There is no financial equivalent to the Laws of Planetary Motion in physics. Hence the widespread use of statistics to find meaning in the reams of the data produced by markets. Some statistical methodologies are better than others. The art comes in choosing which data to use, which to ignore, and how to interpret it. All of which demands both discipline and flexibility. A rigid adherence to a particular approach may yield favorable results for a period of time, but is unlikely to work all the time. Conversely, the lack of a defined method subjects investors more to the whims of randomness, with results seldom being repeatable. There are no easy answers, but what is clear is that investors would do well to cast a wary eye on averages and other simplistic statistical measures that are as capable of misleading as much as they are to reveal any meaningful insight.
1 Source: Morningstar Principia (April 30, 2012). To make things even more complicated, Morningstar calculates its category averages using monthly average returns (controlling for survivorship bias, recently incepted funds, and fund mergers) and then “geometrically linking” them to come up with averages for longer periods of time. The simple arithmetic mean annual return for actively managed large-growth funds still in existence at the end of 2011 was -2.23%.
Kerry O'Boyle is an Investment Strategist with Aston Asset Management. Prior to joining Aston he wrote on a variety of investment topics as a mutual fund analyst for Morningstar, Inc. He is a graduate of the U.S. Naval Academy, and holds an M.A. in Liberal Arts from St. John's College, Annapolis, MD.
For more information about Aston Asset Management, LP and its subadvisors, please call 800-597-9704, or visit www.astonasset.com.