It’s said that there are two kinds of sins: sins of omission and sins of commission. In this short commentary, the authors try to unfold some of the sins committed by forecast users and vendors and also comment on the miscommunication of forecast uncertainty from the perspective both of users and systems

Nikolopoulos, Konstantinos

Petropoulos, Fotios

Online Research @ Cardiff

This is an Open Access document downloaded from ORCA, Cardiff University's institutionalrepository: http://orca.cf.ac.uk/89028/This is the author’s version of a work that was submitted to / accepted for publication.Citation for final published version:Nikolopoulos, Konstantinos and Petropoulos, Fotios 2016. Misbehaving, misdesigning andmiscommunicating. Foresight: The International Journal of Applied Forecasting 41 , pp. 18-19. file Publishers page: Please note: Changes made as a result of publishing processes such as copy-editing, formatting and pagenumbers may not be reflected in this version. For the definitive version of this publication, pleaserefer to the published source. You are advised to consult the publisher’s version if you wish to citethis paper.This version is being made available in accordance with publisher policies. See http://orca.cf.ac.uk/policies.html for usage policies. Copyright and moral rights for publicationsmade available in ORCA are retained by the copyright holders.Misbehaving, misdesigning and miscommunicating Kostas Nikolopoulos and Fotios Petropoulos  There are two kinds of sins: sins of omission and sins of commission. In this short commentary, we will try to fold out some of the sins that FSS users fall into. Also, we point out some misdesign issues of current forecasting support systems. Finally, we commeny on the miscommunication of forecast u ertai t  fro  oth users a d s ste s  perspe ti e. USERS’ SINS Users  is eha iour is usuall  li ked ith their eed to justify their own roles and salaries. For example, while most forecasting software provide optimisation procedures for at least some key parameters of the available forecasting methods, users tend to unnecessarily change the suggested by the software optimal values so that it seems that they are involved in the forecasting process. However, in some cases they even lack the knowledge of the impact of different parameter values on the produced (suboptimal) forecasts.  The same behaviour is observed in the case of automatic method selection approaches which are often considered as black boxes ei g too o ple . Users prefer to have the sense of ownership on the produced forecasts. Even if there is evidence that judgmental model selection can be of value if performed properly, users tend to virtually create limited pools of methods by consistently selecting amongst one or two simple methods that can easily understand and feel familiar with.  Ti keri g of para eters a d passi g the s ste s re o e datio s provide the excuses  managers use to justify  their role in the forecasting process. Users are not always trained forecasters/demand planners (see also the discussion by Fildes and Goodwin, 2007). Anecdotal evidence suggests that it is not uncommon to find that demand planners  positions are occupied by people that do not possess the necessary skillsets and have been i  su h positio s a ide tall . To that respect, the professional training that is offered from various institutions and universities is of tremendous value.  Also, users are purposely but erroneously misuse their forecasting systems in order to secure bonuses. Examples of such behaviours usually lie in the category of judgmental adjustments. Managers in different positions (operations, production, marketing, finance) frequently are judged by different key performance indicators and their cost functions are sometimes asymmetric, meaning differential costs for errors of over and underforecasting.  The danger is that the result is unwarranted adjustments of the statistical forecasts in the direction that will better ensure individual performance bonuses. (See also the article by Paul Goodwin in this issue). The problem is that the profitability of company as a whole is jeopardised. A potential solution for this eha iour is, u fortu atel , poli e e for e e t . Systems could provide mechanisms under which different sets of forecasts (statistical, budget, marketing, finance, operational, final) are stored, aligned with users/managers and benchmarked separately. While each set of forecasts can be the base for a different scenario, from a research viewpoint an interesting question is the optimal reconciliation of such forecasts.  VENDOR’S SINS Scott Armstrong (2001) has attempted to gather best principles and practices in forecasting and more recently has develop a golden rule checklist  (Armstrong and colleagues, 2015) of 28 operational guidelines for conservative forecasting. Most forecasting systems fail to support the application of such forecasting principles, despite the considerable empirical evidence for their value. One could argue that such principles/rules are not always universally applicable. For example, judgmental adjustments to statistical forecasts may reduce accuracy and introduce biases, however, they might prove to be beneficial in cases where soft information is not captured in any other way. As such, a horses for ourses  approa h ould e of alue: what is suitable for one person or situation might be unsuitable for another. In the same way that different forecasting methods are suitable for different types of data, forecasting principles and operational guidelines can significantly help in improving performance if applied properly. A forecasting system that would include such principles and also guidance towards their application would be in an advantageous position to produce accurate forecasts. “oft are o pa ies ha e ee  slo  to adopt ethods that ha e ee  sho  to i pro e a ura  (Armstrong and Fildes, 2006). While this quote is already 10 years old, we can still see that software vendors continue to deliberately omit robust forecasting methods. The same methodological issue has been further discussed in a recent Foresight article (Petropoulos, 2015). Noteworthy examples of non-adopted forecasting methods include the Theta method, the winner of M3-Competition, and Syntetos and Boylan approximation, a bias- orre tio  of the Crosto s ethod for intermittent demand data. We expect that the inclusion of such methods would significantly increase the performance of the automatic selection methodologies offered by the different forecasting software. On a brighter note, we are happy to see large software vendors expanding their pool of available methods (see for example the new features of Microsoft Excel 2016). In any case, much more progress remains to be done. MISCOMMUNICATION BY USERS AND VENDORS Here, we refer to the problem of miscommunicating the uncertainty around forecasts. Both users and software tend to rely on the point forecasts and not the prediction intervals that show the likely range of error in the point forecasts. These intervals are frequently wide (large range of potential error) and forecasters avoid communicating this result if they feel that they might appear incompetent in producing forecasts and supporting respective decisions.  Similarly, software vendors avoid presenting such information in the fear that this might decrease the value of their product. In fact, presenting the uncertainty around the provided forecasts adds considerable value to the forecasting process, while it can prove useful in effective worst/best case scenario planning.  References J.S. Armstrong  Standards and Practices for Forecasting , in J.S. Armstrong (Ed.), Principles of forecasting: A handbook for researchers and practitioners (pp. 443–472), New York: Springer. J.“. Ar stro g a d ‘. Fildes  Maki g progress i  fore asti g , International Journal of Forecasting, 22, 433-441. J.“. Ar stro g, K.C. Gree  a d A. Graefe  Golde  rule of fore asti g: e o ser ati e , Journal of Business Research, 68(8), 1717-1731. R. Fildes a d P. Good i   Against Your Better Judgment? How Organizations Can Improve Their Use of Manage e t Judg e t i  Fore asti g , Interfaces, 37(6), 570-576. F. Petropoulos  Fore asti g “upport “ ste s: a s for ard , Foresight,  Fall , -11.  Konstantinos Nikolopoulos is the Director of Research (Assistant Dean—Research) for the College of Business, Law, Education and Social Sciences at Prifysgol Bangor University, Director of the forLAb (www.forLAb.eu), the forecasting laboratory, and holds the Chair in Decision Sciences at Bangor Business School, Wales, UK. Dr Fotios Petropoulos is Lecturer (Assistant Professor) at Cardiff Business School of Cardiff University and the Forecasting Support Systems Editor of Foresight. His research expertise lies in behavioural aspects of forecasting and improving the forecasting process, applied in the context of business and supply chain. He is the co-founder of the Forecasting Society (www.forsoc.net). 

Misbehaving, misdesigning and miscommunicating

https://orca.cardiff.ac.uk/id/eprint/89028/1/Foresight%202016%20Misbehaving%20%28post-print%29.pdf

Misbehaving, misdesigning and miscommunicating

Abstract

Similar works

Full text

Available Versions

Online Research @ Cardiff