Why Analyst Estimates Fail

For stock research and screening, we use Fiscal.ai (use code “STABLEBREAD” to save 15%). It’s the most complete all-in-one research terminal we’ve found that’s accessible to the average retail investor.

For custom analysis and flexible valuation models, we use our Automated Stock Analysis Spreadsheet, powered by Wisesheets.

These are the two tools we use to analyze and value stocks more effectively and reach quicker, more informed investment decisions. Both have free versions if you want to try them out.

🌾 Welcome to StableBread’s Newsletter!

When trying to estimate a company’s future potential and valuation, many investors take the quick/lazy approach and rely on analyst estimates for guidance.

But as I’ve written about before, this is a flawed approach. David Dreman’s "Contrarian Investment Strategies: The Psychological Edge" (2012) provides extensive evidence that builds on why these estimates fail so consistently.

The central finding of his research was that analysts average a 40% error rate annually, even though Wall Street demands earnings estimates within 3% accuracy.

The following 12 forecasting follies from Dreman reveal how analysts consistently fail, what causes these massive errors, and why the problem persists decade after decade despite overwhelming evidence that precise earnings prediction is impossible.

Wall Street treats company earnings as the major determinant of stock prices. That's why firms spend eight figure research budgets to hire top analysts who can deliver precise earnings predictions.

Institutional Investor magazine polls hundreds of financial institutions each year to select an "All Star" team of the best analysts across every major industry. Making these teams transforms careers and determines which firms capture billions in commission revenue.

But do these highly paid experts actually deliver? Financial World's 1980 study "The Superstar Analysts" tracked twenty superstar analysts' recommendations to find out.

During a period when the market rose 14.1%, following their 132 stock picks would have generated only 9.3% returns, about 34% worse than simply buying an index fund. Only 42 of the 132 recommended stocks, just one third, managed to beat the S&P 500.

As one large institutional buyer of research observed, analysts "get brave at just the wrong time and cautious just at the wrong time. It's uncanny when they say one thing and start doing the opposite."

Dreman performed a study that examined 216,576 consensus forecasts from 1973 to 2010, requiring at least four separate analyst estimates per stock. The study included well over 800,000 individual analyst predictions, making it the most complete analysis of analyst forecasting to date.

Dreman found the average error for analyst estimates was 40% annually, even though these forecasts were made less than three months before the quarter ended.

This massive error rate wasn't a fluke or limited to certain periods. The mistakes stayed consistent across different market conditions, economic cycles, and throughout nearly four decades of significant change on Wall Street:

What makes these errors particularly notable is that they persisted through the information revolution. In the early 1970s, analysts worked with paper and calculators, with nothing online. Today's analysts have instant access to competitors' reports, estimate changes, volumes of data, and sophisticated modeling tools.

Yet despite exponentially more information available now, the error rates haven't improved!

Wall Street professionals believe that reported earnings should come within 3% of consensus estimates, and many demand even better accuracy. Yet even tiny misses can devastate stock prices.

Here are a few real-world examples:

These reactions raise an obvious question. If tiny misses cause such damage, how often do analysts actually hit their targets? Dreman analyzed his database of 216,576 consensus estimates, comparing predictions to actual reported earnings:

As the figure shows:

As Dreman concludes, “putting your money on these estimates means you are making a bet with the odds heavily against you.”

Some investors accept that individual stock predictions are difficult but believe industry forecasts should be more reliable. After all, shouldn't analyzing an entire industry smooth out company-specific problems and provide better accuracy?

Dreman tested this theory by dividing his database into 24 industry groups and measuring forecast accuracy for each:

The results showed that industry level analysis doesn't solve the problem:

These findings undermine the notion that certain industries offer clearer visibility into future earnings, suggesting the problem lies not with specific industries but with the forecasting process itself.

You might expect analysts to struggle more during recessions when earnings drop sharply, with their forecasts being too high. During expansions, the opposite could happen, with estimates being too low as business outperforms expectations.

Dreman tested this theory across seven periods of business expansion and six periods of recession from 1973 to 2010:

As you can see, forecast errors stay consistent regardless of economic conditions, averaging around 39-44% whether in expansions or recessions.

Therefore, economic conditions don't magnify analysts' errors. They're about as frequent in periods of expansion or recession as they are at other times.

What did emerge clearly is that analysts are always optimistic. Their forecasts are too optimistic in recessions, and this optimism doesn't decrease during economic recoveries or in normal times.

Dreman puts it bluntly:

If analysts can't accurately predict whether a company will grow earnings at 20% or 4%, how can anyone justify paying 50x earnings for supposed growth stocks versus 15x for average companies?

Yet investors continue relying on these forecasts to make major financial decisions. They pay premium prices for stocks with supposedly predictable outcomes, even though the precision required to justify these valuations simply doesn't exist.

When reality falls short of the rosy projections, the disappointments aren't caused by “unexpected events” but by relying on estimates that were flawed from the beginning.

If decades of evidence show that earnings forecasts fail 40% of the time, why don't professional investors change their approach?

Most analysts and money managers acknowledge the poor forecasting record, but only as it applies to other people.

When confronted with statistics about forecast errors, they dismiss them as data points that don't apply to their own work, claiming factors like misleading company guidance, temporary oversights, or that better research will prevent future errors.

Psychologists call this the "illusion of validity," where people maintain high confidence despite low accuracy rates. This means professionals keep using methods that require impossible precision, convinced they'll succeed where thousands have failed.

Why do analysts consistently miss earnings targets by such wide margins? Three studies reveal the disconnect between how analysts forecast and how earnings actually behave:

Think about what this means. If analysts project future earnings by extending past trends, but earnings actually move randomly with no connection to history, massive forecast errors aren't just likely, they're guaranteed.

Harvard economist Richard Zeckhauser identified another problem with his "big bath theory." Companies manipulate earnings to show smooth growth because that's what analysts want. When they can't deliver, managers write off everything possible in one quarter, creating better comparisons for future periods. This earnings management adds yet another unpredictable element.

Career Pressures on Analysts’ Recommendations

Analysts also face career incentives that distort their forecasts and recommendations.

John Dorfman of The Wall Street Journal surveyed major brokerage houses to determine what actually drives analyst compensation. Among seven factors determining compensation, accuracy of forecasts ranked dead last.

What matters most? How the brokerage firm's sales force rates the analyst. One firm's bonus system awards 130 points for buy recommendations but only 60 points for sells. No points are added for accuracy.

The pressure against issuing sell recommendations goes beyond compensation. Companies even retaliate against analysts and entire firms who write negative reports.

Making Institutional Investor's All Star team ranks second in importance for compensation. Analysts spend March and April visiting institutional clients, lobbying for votes that will boost their careers and their firms' commission revenues.

The point is that analysts' primary job is generating commissions, not providing accurate forecasts. They must tell compelling stories that drive trading, whether or not those stories prove correct.

In other words, the system rewards optimism and punishes honesty, creating an environment where accurate forecasting takes a back seat to marketing.

More information doesn't lead to better predictions.

As previously observed, despite having instant access to vast databases and sophisticated tools that would have seemed miraculous decades ago, today's analysts are no more accurate than their predecessors who worked with paper and pencil in the 1970s.

In fact, when people receive more information, their confidence rises dramatically, but their accuracy stays flat. This gap between how right people think they are and how right they actually are explains many forecasting failures.

Studies across many professions, including clinical psychologists, lawyers, engineers, and physicians, all show the same pattern. Professionals have excessive confidence in their predictions even when those predictions consistently miss the mark.

Despite knowing their poor track records, people remain confident, believing each new situation is different (the illusion of validity).

Daniel Kahneman, who won the Nobel Prize for his research on decision making under uncertainty, identified two completely different ways people approach forecasting. The difference helps explain why analysts keep getting it wrong.

The "inside view" is what virtually every analyst uses. They dive deep into specific details about a company, examining growth rates, market share, product development, economic conditions, and dozens of other variables.

They treat each stock as unique, believing that by understanding all these moving parts, they can predict exactly where earnings will land.

The "outside view" takes the opposite approach. Instead of getting lost in details, it asks a simple question: how accurate have earnings forecasts been for similar companies in similar situations? It looks at the statistical record rather than trying to predict specific outcomes.

Kahneman concluded that "when both methods are applied with intelligence and skill, the outside view is much more likely to yield a realistic estimate."

The reason is simple. Even if you could identify every possible scenario, the odds of predicting the exact one that will occur are minuscule. Yet analysts keep trying to accomplish exactly that with their precise quarterly estimates.

How bad are the odds of maintaining accurate forecasts over time? Dreman calculated the actual probabilities, and they're worse than you might imagine.

Wall Street demands forecasts within a 3% accuracy range. Yet as we've seen, only 30% of analyst estimates fall within even a more generous 5% range in any single quarter. But that's just the beginning of the problem:

The table demonstrates what happens when you need accurate forecasts for multiple quarters, which is exactly what most valuation methods require:

Dreman puts this in perspective:

Few people would spend money on a lottery with those odds, yet millions of investors bet their savings on stocks whose valuations depend on precisely these impossible forecasts.

Some investors might say they don't mind if earnings come in above estimates. So what are the chances of simply avoiding negative surprises?

Growth stocks face an even bigger problem. Many require accurate forecasts for 10 or 15 years to justify their valuations. Nobody can accurately predict earnings that far out. Yet many investors still rely on these long term analyst estimates despite the terrible odds.

The 2007 financial crisis showed how overconfidence destroys even the most sophisticated forecasting.

As the housing bubble burst in late 2006, major banks, the Fed, and Treasury officials issued reassuring statements about minimal damage. Outside observers like Paul Krugman and Gretchen Morgenson had warned about the bubble for two years, but were ignored.

Through 2007 and 2008, banks kept underestimating their toxic mortgage losses even as dozens of subprime firms collapsed. Major banks insisted their portfolios were sound until their own survival was threatened. By the time they took massive writedowns, it was too late.

Whether it's mortgage portfolios or quarterly earnings, experts consistently overestimate their ability to predict the future. Analysts today have infinitely more information than their predecessors, yet their accuracy hasn't improved. And the investment industry continues demanding precise forecasts that are mathematically impossible to achieve.

Our Bottom Line

Instead of relying on analyst estimates or letting them bias your decisions, trust your own understanding of the business. Make conservative projections with realistic worst-case scenarios and recognize that precise forecasting simply doesn't exist.