Rule-Based Forecasting: Development and Validation of an Expert Systems Approach to Combining Time Series Extrapolations

This paper examines the feasibility of rule -based forecasting, a procedure that applies forecasting expertise and domain knowledge to produce forecasts according to features of the data. We developed a rule base to make annual extrapolation forecasts for economic and demographic time series. The development of the rule base drew upon protocol analyses of five experts on forecasting methods. This rule base, consisting of 99 rules, combined forecasts from four extrapolation methods (the random walk, regression, Brown's linear exponential smoothing, and Holt's exponential smoothing) according to rules using 18 features of time series. For one-year ahead ex ante forecasts of 90 annual series, the median absolute percentage error (MdAPE) for rule- based forecasting was 13% less than that from equally-weighted combined forecasts. For six-year ahead ex ante forecasts, rule-based forecasting had a MdAPE that was 42% less. The improvement in accuracy of the rule - based forecasts over equally-weighted combined forecasts was statistically significant. Rule-based forecasting was more accurate than equal-weights combining in situations involving significant trends, low uncertainty, stability, and good domain expertise.Rule-based forecasting, time series

The Profitability of Winning

Sports and war metaphors abound in business today. For example, one management book, Thunder in the Sky, by Thomas Cleary, opens with a Chinese saying that translates: “The marketplace is a battlefield. The Asian people view success in the business world as tantamount to victory in battle.” The book advises American executives to do the same. However, such metaphors are misleading. The objective in both sports and war is to beat the competitor. Business, on the other hand, aims to create wealth.business, profits, winning,

Causal Forces: Structuring Knowledge for Time-series Extrapolation

This paper examines a strategy for structuring one type of domain knowledge for use in extrapolation. It does so by representing information about causality and using this domain knowledge to select and combine forecasts. We use five categories to express causal impacts upon trends: growth, decay, supporting, opposing, and regressing. An identification of causal forces aided in the determination of weights for combining extrapolation forecasts. These weights improved average ex ante forecast accuracy when tested on 104 annual economic and demographic time series. Gains in accuracy were greatest when (1) the causal forces were clearly specified and (2) stronger causal effects were expected, as in longer- range forecasts. One rule suggested by this analysis was: “Do not extrapolate trends if they are contrary to causal forces.” We tested this rule by comparing forecasts from a method that implicitly assumes supporting trends (Holt’s exponential smoothing) with forecasts from the random walk. Use of the rule improved accuracy for 20 series where the trends were contrary; the MdAPE (Median Absolute Percentage Error) was 18% less for the random walk on 20 one-year ahead forecasts and 40% less for 20 six-year-ahead forecasts. We then applied the rule to four other data sets. Here, the MdAPE for the random walk forecasts was 17% less than Holt’s error for 943 short-range forecasts and 43% less for 723 long-range forecasts. Our study suggests that the causal assumptions implicit in traditional extrapolation methods are inappropriate for many applications.Causal forces Combining Contrary trends Damped trends Exponential smoothing Judgment Rule-based forecasting Selecting methods

Integration of Statistical Methods and Judgment for Time Series

We consider how judgment and statistical methods should be integrated for time-series forecasting. Our review of published empirical research identified 47 studies, all but four published since 1985. Five procedures were identified: revising judgment; combining forecasts; revising extrapolations; rule-based forecasting; and econometric forecasting. This literature suggests that integration generally improves accuracy when the experts have domain knowledge and when significant trends are involved. Integration is valuable to the extent that judgments are used as inputs to the statistical methods, that they contain additional relevant information, and that the integration scheme is well structured. The choice of an integration approach can have a substantial impact on the accuracy of the resulting forecasts. Integration harms accuracy when judgment is biased or its use is unstructured. Equal-weights combining should be regarded as the benchmark and it is especially appropriate where series have high uncertainty or high instability. When the historical data involve high uncertainty or high instability, we recommend revising judgment, revising extrapolations, or combining. When good domain knowledge is available for the future as well as for the past, we recommend rule- based forecasting or econometric methods.statistical methods, statistics, time series, forecasting, empirical research

Designing large systems: Five stories; five lessons

Systems thinking and design practice share the characteristic that to be successful each must concern itself with attending to the needs of the whole and to the interactions among its parts. And they suffer the same fate when they succeed. Inevitably they produce unintended consequences and unanticipated side effects. But there is a radical difference in their typical starting places and their logics

Does AI Research Aid Prediction? A Review and Evaluation

Despite the increasing application of Artificial Intelligence (AI) techniques to business over the past decade, there are mixed views regarding their contribution. Assessing the contribution of AI to business has been difficult, in part, due to lack of evaluation criteria. In this study, we identified general criteria for evaluating this body of fiterature. Within this framework, we examined applications of AI to business forecasting and prediction. For each of the seventy studies located through our search, we evaluated how effectively the proposed technique was compared with alternatives (effectiveness of validation) as well as how well the technique was implemented (effectiveness of implementation). We concluded that by using acceptable practice and providing validated comparisons, 31% (22) of the studies contributed to our knowledge about the applicability of the AI techniques to business. Of these twenty-two studies, twenty supported the potential of AI in forecasting. This small number of studies indicates a need for improved research in this area

Designing Tailorable Technologies

This paper provides principles for designing tailorable technologies. Tailorable technologies are technologies that are modified by end users in the context of their use and are around us as desktop operating systems, web portals, and mobile telephones. While tailorable technologies provide end users with limitless ways to modify the technology, as designers and researchers we have little understanding of how tailorable technologies are initially designed to support that end-user modification. In this paper, we argue that tailorable technologies are a unique technology type in the same light as group support systems and emergent knowledge support systems. This unique technology type is becoming common and we are forced to reevaluate existing design theory, methods of analysis, and streams of literature. In this paper we present design principles of Gordon Pask, Christopher Alexander, Greg Gargarian, and Kim Madsen to strengthen inquiry into tailorable technologies. We then apply the principles to designing tailorable technologies in order for their design to become more coherent and tractable. We conclude that designers need to build reflective and active design environments and gradients of interactive capabilities in order for technology to be readily modified in the context of its use

Designing Tailorable Technologies

Tailorable technologies are technologies that are modified by users in the context of their use and are around us as desktop operating systems, web portals, and mobile telephones. While tailorable technologies provide users with limitless ways to modify the technology, as designers and researchers we have little understanding of how this should affect design. In this paper we present principles from four designers to strengthen inquiry into tailorable technologies. We then apply the principles to the case of the design of a web portal. We conclude that designers need to more consciously build reflective and active design environments and gradients of interactive capabilities in order for technology to be readily modified in the context of its use

Expert Opinions About Extrapolation and the Mystery of the Overlooked Discontinuities

We report on the opinions of 49 forecasting experts on guidelines for extrapolation methods. They agreed that seasonality, trend, aggregation, and discontinuities were key features to use for selecting extrapolation methods. The strong agreement about the importance of discontinuities was surprising because this topic has been largely ignored in the forecasting literature

Rule-based Forecasting: Development and Validation of an Expert Systems Approach to Combining Time Series Extrapolations

This paper examines the feasibility of rule-based forecasting, a procedure that applies forecasting expertise and domain knowledge to produce forecasts according to features of the data. We developed a rule base to make annual extrapolation forecasts for economic and demographic time series. The development of the rule base drew upon protocol analyses of five experts on forecasting methods. This rule base, consisting of 99 rules, combined forecasts from four extrapolation methods (the random walk, regression, Brown\u27s linear exponential smoothing, and Holt\u27s exponential smoothing) according to rules using 18 features of time series. For one-year ahead ex ante forecasts of 90 annual series, the median absolute percentage error (MdAPE) for rule-based forecasting was 13% less than that from equally-weighted combined forecasts. For six-year ahead ex ante forecasts, rule-based forecasting had a MdAPE that was 42% less. The improvement in accuracy of the rule-based forecasts over equally-weighted combined forecasts was statistically significant. Rule-based forecasting was more accurate than equal-weights combining in situations involving significant trends, low uncertainty, stability, and good domain expertise