Economic indicators ; Economic forecasting

Doug Campbell

English

Research Papers in Economics

J
im Smith tells the story this way:
It was the summer of 1971. Smith was the director of
credit market research at Sears, Roebuck and Co. One
day the chief executive, a man named Gordon Metcalf,
strolled into Smith’s office and talked about his recent visits
with international suppliers. Overseas, Metcalf said, there
was growing sentiment that the dollar was overvalued.
Metcalf realized that if the dollar decreased in value, it could
hurt Sears’ business. Sears needed a clearer picture of the
future impact of such a change.
“Get together with your friends and see what you can
do,” Metcalf ordered. So Smith dialed up his friends at the
University of Pennsylvania, where the famed Wharton
Econometric Forecasting Associates (WEFA) was housed.
At the time, the notion that the gold-backed dollar might
ever float in value was still considered far-fetched by some.
But WEFA spent a few weeks tweaking a short-term model
and ran some simulations for Smith, who duly reported the
results to the executive suite.
It turned out to be highly valuable information, especial-
ly after Aug.15, 1971, when the Nixon administration brought
an end to the  Bretton Woods Agreement of 1944 that fixed
exchange rates worldwide. From then on, the dollar would
float, its value determined by the constantly changing bal-
ance of supply and demand. While most other firms were
caught off guard, Sears was ready.
“We saved and made a ton of money as a result of fore-
casting,” Smith says today from his office in North Carolina,
where he is chief economist with Parsec Financial in
Asheville. “That model pretty well played out with all that
happened over the next two to three years.”
This tale underlines the worth of a good forecast. In his
time, Smith has made a few. In fact, after Sears he went on to
become one of the nation’s most celebrated economic fore-
casters. Since the late 1990s, the Wall Street Journal has three
times named him the nation’s most accurate forecaster.
But is there such a thing as a “star” forecaster? Are there
a handful of prognosticators whose abilities consistently sur-
pass the crowd? If so, then you would think they are either in
possession of superior instincts or superior mathematical
models. Perhaps it’s a little of both.
Models of all stripes can never perfectly predict the
future because they are not exact replicas of the actual 
economy. To get an accurate forecast, you need information
that gets closer to the current state of affairs. Maybe a cer-
tain forecaster is friends with a banker who provides the tip
that more loans are going past due. That’s information the
forecaster would want to incorporate. Of course, informa-
tion can be wrong. The loan problems might have been
limited to that single bank.
“It takes a great deal of tender, loving care to get the
forecasts to run properly,” Smith says. “Nobody is perfect
every time.”
Stars
Forecasters are constantly being ranked. Besides the Wall
Street Journal, there are rankings and surveys in USA Today
and  BusinessWeek, as well as in the monthly Blue Chip
Fall 2007 • Region Focus 21
Professional 
Prognosticators
Is Forecasting a Science or an Art?
BY DOUG CAMPBELL
B
L
U
E
 
C
H
I
P
 
E
C
O
N
O
M
I
C
 
I
N
D
I
C
A
T
O
R
SEconomic Indicators, a must-read for
chief economists at Fortune 500 firms.
The surveyed forecasts encompass
firms ranging from Morgan Stanley to
FedEx on measures ranging from GDP
to housing starts, usually predicting
changes to a tenth of a percent. Over
time, a handful of forecasters stand
out. These are the star forecasters, and
it’s fair to say that Stuart Hoffman is
among them.
Hoffman is chief economist at
Pittsburgh-based PNC Financial
Services Group. The Wall Street Journal
named him one of the nation’s top
forecasters from 1988 through 2006, 
a remarkable run. And Business
Week  named him the most accurate
forecaster for 2004.
Hoffman develops his forecasts the
way many others do. He uses a basic
model and monitors data ranging from
consumer spending to productivity.
He lets the model run for four to six
quarters out to “see what the key eco-
nomic trend looks like.” Then he
makes adjustments “based partly on
intuition and conversations with other
economists, particularly people in the
business who are contacts I have.”
This talking-and-listening approach is
most useful for short-term estimates.
It is this network of contacts to which
Hoffman attributes his success. That
and his distance from Wall Street,
where there is a tendency, Hoffman
believes, for economists to get too
caught up in the state of financial 
services and ignore other sectors of
the economy.
Though there are more data avail-
able today, which are quicker both to
obtain and to process, and models are
more intricate, Hoffman isn’t so sure
his forecasts are much superior to
what they were 20 years ago.
“Forecasting is still as much of an art
as it ever was,” he says.
Smith agrees with that assessment.
Though he is skilled in econometrics
— a leading tool of forecasters, which
uses both theory and statistical tech-
niques to evaluate data — Smith
believes that good forecasts are the
result of good information. He attrib-
utes his predictive success to his
ability to listen. “I have never found 
a substitute in my 35 years of doing 
this for asking people what they
think is going on,” Smith says. “There’s
always somebody who knows more
than you do, and you’re well-advised 
to listen.”
Building Crystal Balls
Modern-day forecasting history
begins with Jan Tinbergen and
Lawrence Klein, who both received
Nobel Prizes primarily for their work
in building multi-equation econo-
metric models. In the 1950s Klein’s
models of the U.S. economy became
the most widely used. In 1963, he
established WEFA, which used a
model bearing the association’s name.
Smith was tapping into a more evolved
version of this model helping Sears
anticipate the impact of a floating dol-
lar. As the cost of computer power
declined, forecasting models grew
richer and more complex.
For a time, there were three major
economic forecasting models — 
one used by WEFA, another by Chase
Econometrics, and a third by Data
Resources Inc., developed by its
founder Otto Eckstein. All three 
of these entities later merged to
become Global Insight, today the
largest economic forecasting firm in
the world, with 600 employees and
about $100 million in annual revenue.
Leading rivals to Global Insight
include Macroeconomic Advisers,
founded by former Federal Reserve
Governor Laurence Meyer, and
Moody’s Economy.com.
If you had models that could 
perfectly predict the future, then that
would be one thing. But as Robert
Lucas  acknowledged with rational
expectations theory, the world is an
uncertain place. Changes in economic
conditions can be no more easy to 
predict than the next roll of the 
dice. People are forward-looking. As
government policies and economic
conditions change, so do people’s
expectations about the future and
hence their actions; moreover, 
people’s actions respond both directly
to present conditions — today’s prices,
holding future expectations constant
— and to expectations of the future. It
is difficult to build a model capable of
incorporating all these factors.
Certainly, it is impossible to make 
predictions on measures like GDP
with precision to even the tenth 
decimal place.
“As long as you take the model 
forecast for what it is, models are very
useful tools,” says Roy Webb, a senior
economist with the Richmond Fed 
who has studied forecasting accuracy.
“The danger is you assign these numbers
more significance than you should.”
There is considerable academic
debate about which sort of models are
the best — for various purposes one
might choose among econometric
models, or simpler vector autoregres-
sive (VAR) models. Among the key
differences is that structural models
use economic theory to constrain 
the possible relationships among 
variables, while VARs are often 
considered “atheoretical” because 
they tend to let the data speak for
themselves.
Some observers argue that all the
subjective fiddling that goes onto
modeling strips them of any scientific
legitimacy. “Add factors” introduce an
extra degree of human error into the
process, inevitably fouling it up.
Despite such concerns, that’s how
most forecasters operate. They use a
model to get a sort of baseline, and
then add in factors that may not yet be
either showing up in the data or for
which the model may ignore. Take the
U.S. macroeconomic model used at
Global Insight. It has about 1,900 vari-
ables, with data points coming from
national income and product accounts,
price indexes, and 25 different interest
rates. Then economists take over.
“Forecasts are a combination of
econometrics and judgment,” says 
Sara Johnson, a managing director 
and economist at Global Insight. 
“The econometrics help us to draw
statistical relationships based on 
the historical record. Economists can
then insert their judgment based on
how current conditions might differ
from the past, based on factors 
22 Region Focus • Fall 2007that the models cannot, or do not,
fully incorporate.”
Which Way?
The apparent slow of economic 
activity from the third to the fourth
quarter led some to wonder whether
the economy was approaching a turn-
ing point. This is when forecasters
really earn their keep. “Whenever
there’s volatility, demand for our serv-
ices increases,” Johnson says. “Our
clients are watching our forecasts and
analyses even more closely and having
more frequent contact with us.”
For an industry strategist trying to
figure out what to do next, this might
seem a tempting time to rely on an
aggregate of multiple forecasts of the
aggregate economy. That is because
the average consistently beats individ-
ual forecasters. An Atlanta Fed study
examined forecaster rankings in 
the  Blue Chip Economic Indicators
Survey. It found that the consensus
forecast performed better over time
than any individual forecaster —
although several forecasters did quite
well. “This result is a ‘reverse Lake
Wobegon’ effect: none of the forecast-
ers are better than the average
forecaster,” the authors wrote. “There
are superior forecasters, but no indi-
vidual has access to all of the
independent information from all of
the forecasts that is incorporated into
the consensus forecast.”
This underlines a truth that will
come as no surprise to fans of 
The Wisdom of Crowds by James
Surowiecki — who argues that collec-
tive information tends to be more
reliable than individual assessments.
And yet, some individual firms and
forecasters do consistently outshine
others. For example: Blue Chip
Economic Indicators hands out an annual
award to the best forecasting record
over the past four years based on pro-
jections of real GDP, the consumer
price index, three-month Treasury
bills, and the unemployment rate. 
A few firms, including Global Insight
and Macroeconomic Advisers, are
dependably in the upper echelons 
of the rankings. (Notably, the rankings
don’t point out forecasters who 
consistently miss; there is no “Most
Inaccurate Forecaster” award.)
Randell Moore, editor of the Blue
Chip survey, notes that DuPont has
won the annual honor three times in
the past three decades — but each
time with a different chief economist.
“I don’t detect that any individual is
particularly good over long periods of
time at forecasting,” Moore says.
“That’s why using the consensus
appears to make the most sense.”
An interesting exception may be
the Federal Reserve’s “Greenbook.”
Certain economic projections from
the Greenbook are released to the
public after a five-year lag, and studies
have shown that those projections are
quite reliable compared with private
forecasts. The Greenbook process 
is a back-and-forth between the large
Federal Reserve Board model and 
subjective add-ons by staff experts.
In the most cited study, economists
Christina Romer and David Romer
with the University of California at
Berkeley attributed the Greenbook’s
accuracy to the finding that the Fed
“appears to possess information about
the future state of the economy that 
is not known to market participants.”
Princeton University economist
Christopher Sims found that the
Greenbook even beats most of the
Fed’s own model-based forecasts. Sims
agrees that there is some evidence,
though not complete, that “the superi-
ority of the Fed forecasts arises from
the Fed having an advantage in the
timing of information — even with the
view that this might arise entirely
from the Fed having advance knowl-
edge of its own policy intentions.”
Shrinking Industry
For all the potential payoff that a good
forecast can deliver, the business of
economic forecasting has been con-
tracting. In the heyday of the 1960s
and 1970s, it was customary for big
companies to keep economics depart-
ments, with several analysts reporting
to a chief economist. But cost-cutting
began in the 1980s, as many firms saw
they could simply contract for fore-
casting services, or rely on published
consensus forecasts. Bank mergers
also led to consolidation of economic
research departments.
Smith believes that businesses
which give up in-house forecasters
with see it reflected in their bottom
line. “There’s no way to cope with all
the changes that come up and have a
feel for whether something is a major
shift, or a tempest in a teapot that 
will pass, unless you have your own
internal group,” Smith says. “You won’t
find a consensus for steel demand, or
for vehicle output, and that’s of 
huge importance to many industries.
It’s a small investment and you only
have to get a few things right to pay 
for themselves.”
Of course, even in-house forecast-
ers get things wrong, as Smith readily
admits about his own career. This is
why Smith likes to quote perhaps his
field’s oldest of axioms: “He who lives
by the crystal ball must learn to love
the taste of broken glass.”  RF
Fall 2007 • Region Focus 23
Bauer, Andy, Robert Eisenbeis, Daniel Waggoner, and Tao Zha.
“Forecast Evaluation with Cross-Sectional Data: The Blue Chip
Surveys.” Federal Reserve Bank of Atlanta Economic Review,
Second Quarter 2003, vol. 88, pp. 17-31.
Romer, Christina D., and David H. Romer. “Federal Reserve
Information and the Behavior of Interest Rates.” American
Economic Review, June 2000, vol. 90, no. 3, pp. 429-457.
Sims, Christopher. “The Role of Models and Probabilities in the
Monetary Policy Process.” Brookings Papers on Economic Activity,
September 2002, no. 2, pp. 1-62.
Tulip, Peter. “Has Output Become More Predictable? Changes in
Greenbook Forecast Accuracy.” Federal Reserve Board of
Governors Finance and Economics Discussion Series, August
2005, no. 31.
R EADINGS

Professional prognosticators : Is forecasting a science or an art?

http://www.richmondfed.org/publications/research/region_focus/2007/fall/pdf/feature2.pdf

Professional prognosticators : Is forecasting a science or an art?

Abstract

Similar works

Full text

Available Versions

Research Papers in Economics