LONG-TERM GOALS: The long-term goals of this project are to improve the quantitative accuracy and interpretative utility of official tropical cyclone (TC) track forecasts by enabling forecasters to successfully recognize and skillfully compensate for periods when numerical TC track forecast models are likely to be making highly erroneous track forecasts. The conceptual methodology for accomplishing these goals is the Systematic Approach to Tropical Cyclone Track Forecasting (hereafter Systematic Approach) conceived by Carr and Elsberry (1994).Award # N0001499WR3004

Carr, Lester E. III

Elsberry, Russell L.

English

Calhoun, Institutional Archive of the Naval Postgraduate School

Calhoun: The NPS Institutional ArchiveDSpace RepositoryFaculty and Researchers Faculty and Researchers' Publications1999-09Development of an Expert System Based onthe Systematic Approach to Tropical CycloneTrack ForecastingCarr, Lester E. III; Elsberry, Russell L.http://hdl.handle.net/10945/68947This publication is a work of the U.S. Government as defined in Title 17, UnitedStates Code, Section 101. Copyright protection is not available for this work in theUnited States.Downloaded from NPS Archive: Calhoun1Development Of An Expert System Based On TheSystematic Approach ToTropical Cyclone Track ForecastingLester E. Carr IIIDepartment of Meteorology, MR/CrNaval Postgraduate School589 Dyer Rd., Room 254Monterey, CA 93943-5114phone: (831) 656-2374      fax: (831) 656-3061       email: carrle@nps.navy.milRussell L. ElsberryNaval Postgraduate School589 Dyer Rd., Room 254Monterey, CA 93943-5114phone: (831) 656-2373      fax: (831) 656-3061       email: elsberry@met.nps.navy.milAward # N0001499WR30041http://www.weather.nps.navy.milLONG-TERM GOALSThe long-term goals of this project are to improve the quantitative accuracy and interpretative utility ofofficial tropical cyclone (TC) track forecasts by enabling forecasters to successfully recognize andskillfully compensate for periods when numerical TC track forecast models are likely to be makinghighly erroneous track forecasts. The conceptual methodology for accomplishing these goals is theSystematic Approach to Tropical Cyclone Track Forecasting (hereafter Systematic Approach)conceived by Carr and Elsberry (1994).OBJECTIVESThe specific objectives of this project are to: (i) develop a prototype expert system based on theSystematic Approach; and (ii) demonstrate the feasibility of such an expert system for improving theaccuracy and meteorological utility of official tropical cyclone track forecasts.  It is emphasized that thepurpose of the expert system is not to replace the human forecaster, but to proactively lead theforecaster through a methodical process of numerical guidance evaluation and forecast formulation thatproduces consistently skillful official track forecasts.APPROACHFigure 1 shows the procedural framework of the Systematic Approach, including the principal tasksthat must be accomplished in each phase. The basic approach to create a Systematic Approach expertsystem is to develop a series of inter-linked software modules that assist the forecaster to accomplisheach task. The formulation of an accurate TC forecast represents a highly complex informationmanagement problem that poses challenges with regard to timely access, effective display, andReport Documentation Page Form ApprovedOMB No. 0704-0188Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering andmaintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information,including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, ArlingtonVA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if itdoes not display a currently valid OMB control number. 1. REPORT DATE 30 SEP 1999 2. REPORT TYPE 3. DATES COVERED   00-00-1999 to 00-00-1999  4. TITLE AND SUBTITLE Development Of An Expert System Based On The Systematic ApproachTo Tropical Cyclone Track Forecasting 5a. CONTRACT NUMBER 5b. GRANT NUMBER 5c. PROGRAM ELEMENT NUMBER 6. AUTHOR(S) 5d. PROJECT NUMBER 5e. TASK NUMBER 5f. WORK UNIT NUMBER 7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) Naval Postgraduate School,Department of Meteorology, MR/Cr,589 DyerRd., Room 254,Monterey,CA,93943 8. PERFORMING ORGANIZATIONREPORT NUMBER 9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR’S ACRONYM(S) 11. SPONSOR/MONITOR’S REPORT NUMBER(S) 12. DISTRIBUTION/AVAILABILITY STATEMENT Approved for public release; distribution unlimited 13. SUPPLEMENTARY NOTES 14. ABSTRACT  15. SUBJECT TERMS 16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF ABSTRACT Same asReport (SAR) 18. NUMBEROF PAGES 6 19a. NAME OFRESPONSIBLE PERSON a. REPORT unclassified b. ABSTRACT unclassified c. THIS PAGE unclassified Standard Form 298 (Rev. 8-98) Prescribed by ANSI Std Z39-18 2SYSTEMATIC APPROACH FRAMEWORKPhase I: Numerical Guidance EvaluationPhase IA: Classify Actual Meteorological Situation     Task 1: Classify TC Structure     Task 2: Classify Environment Structure/TransitionsPhase IB: Classify Model-depicted Meteorological Situation(s)     Task 1: Classify TC Structure     Task 2: Classify Environment Structure     Task 3: Assess Numerical Guidance AccuracyPhase II: Objective Technique Evaluation     Task 1: Identify Track Guidance Groups     Task 2: Select Guidance Group for Official ForecastPhase III: Official Track Forecast Formulation     Task 1: Construct Primary/Alternate Envelopes     Task 2: Construct Official Track Forecast     Task 3: Assign Confidence/Alternate ScenarioFigure 1.  Listing of the three phases that comprise the Systematic Approach procedural framework,and the major tasks that must be accomplished in each phase.informed interpretation of various information resources available to the forecaster. In developing theexpert system modules, careful consideration is given to such things as: (i) identification of the keyinformation (e.g., numerical fields, imagery, data, etc.);  (ii) development of effective graphical userinterfaces; and (iii) development and application of knowledge bases and algorithms to assist theforecaster to interpret correctly the displayed information, particularly with regard to successfulassessment of the accuracy of the available numerical model forecasts of TC motion.  Otherconsiderations that affect how the expert system modules are developed include: (i) varying degrees ofproactivity depending on the nature and difficulty of each task being accomplished; (ii) flexible designso that modules may be separately tested, revised, and re-tested; and (iii) a HELP function, in which auser can stop the decision process and review that aspect of the knowledge base relevant toaccomplishing a particular task.WORK COMPLETEDDevelopment and documentation of an initial Model Traits knowledge base for the two numerical TCtrack forecast models (NOGAPS and GFDN) and the four NOGAPS-dependent objective techniquesavailable to forecasters at the JTWC for the western North Pacific has been completed (Carr andElsberry 1999).  Development of an expert system prototype that enables the forecaster to accomplish Phases I and II ofthe Systematic Approach concept (Fig. 1) was completed in July. The expert system prototype wasinstalled and readied for real-time utilization and evaluation at both the Naval Postgraduate School(NPS) and the JTWC in August. For ease of reference, NPS and JTWC agreed to give the expertsystem prototype the name: Systematic Approach Forecasting Aid (SAFA).A parallel real-time test of the SAFA prototype by an experimental forecast team at NPS and theJTWC forecasters (as permitted by operational workload) was conducted from 23 August to 30September to evaluate the effectiveness of SAFA with regard to: (i) display of forecast track and fieldinformation needed to assess model accuracy; and (ii) formulation of selective consensus (SCON) track3forecasts that are more accurate than a simple numerical consensus (NCON) of all available numericalmodel forecast tracks. Because of anomalous TC activity in the western North Pacific (i.e.,disproportionate number of weak and/or short-lived systems), it was decided that the NPS componentof the test would continue at least to the end of calendar year 1999 to provide a more statisticallysignificant sample of  SCON and NCON forecasts.RESULTSA summary of the error mechanism analysis results documented in Carr and Elsberry (1999) is shownin Table 1, and includes some significant changes compared to the preliminary result shown in lastyear’s report. An important and encouraging result is that in 1997 only six error mechanisms accountfor 84% of the poor NOGAPS track forecasts (Table 1; red/bold entries in NOGAPS column) and onlythree error mechanisms account for 68% of the poor GFDN track forecasts (Table 1; red/bold entries inGFDN column). The practical benefits of these result are twofold. First, during the numerical modelevaluation process (Fig. 1) the forecaster can justifiably direct the initial effort to identifyingindications of a comparatively small number of frequently occurring error mechanisms, and onlyconsider the less frequently occurring error mechanisms if no indications of the more frequent errorCAUSES OF NOGAPS OR GFDN 72-H FORECASTTRACK ERRORS GREATER THAN 300 N MINUMBER OFFORECASTS                            Phenomenon Name Acronym NOGAPS GFDNDirect Cyclone Interaction DCI 39-0  31-0  Semi-direct Cyclone Interaction          SCI on Western TC          SCI on Eastern TCSCISCIWSCIE  0-0  3-0  0-0  1-0  Indirect Cyclone Interaction          ICI on Eastern TC          ICI on Western TCICIICIEICIW0-0  0-0  3-0  0-1  Ridge Modification by TC RMT 12-0  2-1  Reverse Trough Formation RTF 10-0  2-0  Response to Vertical wind Shear RVS 10-0  1-3  Baroclinic Cyclone Interaction BCI 8-12    11-2 Midlatitude System Evolutions          Midlatitude CycloGenesis          Midlatitude CycloLysis          Midlatitude AnticycloGenesis          Midlatitude AnticycloLysisMSEMCGMCLMAGMAL0-0  0-0  3-0  0-4    19-0    0-0    0-0    0-2Tropical Cyclone Initialization Size TCS 5-0      9-0Not  discernable or explainable 2  2  Fields not available 0  9  All causes 108  99  Table 1.  Number of forecasts for which various meteorological phenomena were responsible forlarge (>300 n mi) 72-h forecast track errors (FTEs) in NOGAPS and GFDN in 1997. When twonumbers are shown, the first (second) number indicates that phenomenon occurred in the model toan excessive (insufficient) degree.  The rows that are in bold/red identify the comparatively frequenterror mechanisms.4mechanisms can be found. Second, this research result will help model developers and researchers atFNMOC, NRL Monterey, and GFDL focus their efforts to improve the TC track forecasting accuracyof NOGAPS and GFDN. Another pivotal result shown in Table 1 is that, with the exception ofBaroclinic Cyclone Interaction (BCI) that can occur to both an excessive and insufficient degree, all theother error mechanisms involve only excessive processes in NOGAPS and GFDN. The practicalbenefit of this result is that when such one-sided error mechanisms appear to be present in one or morenumerical models, the models with tracks that appear to be least affected are likely to have the mostaccurate track forecasts.    The SAFA prototype completed during this year has all of the essential functions described under theApproach section above, including: (i) highly structured, but non-binding procedural guidance withselectable levels of detail; (ii) cluster-based objective assignment of most probable error mechanismoptions to focus the model evaluation process; (iii) an error mechanism HELP feature that includes trackgraphics and field animations that illustrate the key indications of the frequently occurring errormechanisms; (iv) a highly sophisticated model tracks and fields display and animation capability; (v) aninteractive error mechanism assignment capability that instantly lets the forecaster see the impact ofrejecting certain model tracks(s) on the SCON track; and (vi) an interactive Summary Sheet feature thatrecords objective ensemble spread and cluster analysis data, objective and subjective error mechanismassignment, and forecaster comments to facilitate shift-to-shift information flow.The objectives of the real-time parallel test of SAFA by NPS and JTWC included: (i) evaluating theeffectiveness of the various information display components under operational time constraints; (ii)validating the error mechanism frequencies suggested by the previous research (Table 1); (iii) determiningwhether error mechanisms could be correctly identified without the benefit of hindsight; (iv) determiningthe circumstances under which SCON track forecasts may be produced that are significantly more accuratethan a simple NCON track forecast. Because the parallel test occurred at the end of FY99, and the NPScomponent of the real-time test will continue to the end of calendar year 1999, conclusions concerningobjectives (i)-(vi) above based on a thorough evaluation of the results of the SAFA test are not yetavailable. However, a number of tentative conclusions can be made at this time. With regard to objective(i) above, JTWC personnel were extremely impressed with the information display capabilities of theSAFA prototype (Deputy Director, JTWC, personal communication), made many valuable suggestions forfurther refining that aspect of SAFA, and expressed eagerness to have a version of SAFA that providestrack and field display through their entire Area Of  Responsibility as soon as possible.  Despite the smallsample of SAFA forecasts thus far, the NPS forecast team found that excessive direct cyclone interaction(E-DCI) was, as expected, the most frequent error mechanism, affecting one or more models for significantperiods during four of the seven TCs since the start of the test. This result validates objective (ii) for E-DCI, and it is expected that similar results will be found for the other frequently occurring errormechanisms in Table 1 as the number of SAFA forecasts grows.  In addition, NPS forecast team found thatcases of E-DCI sufficient to cause a 72-h track forecast error exceeding 300 n mi were readily identifiable,thus providing a preliminary validation of objective (iii). Finally, as shown in Fig. 2, the NPS forecast teamfound that when only frequently occurring error mechanisms were occurring (E-DCI and E-RTF thus farduring the test), it is possible to create SCON track forecasts that are much more accurate than NCONforecasts. However, in situations where infrequent error mechanisms were the primary source of modelerror, or compensating and different error mechanisms were occurring in different models, it was notpossible to consistently identify the erroneous models and thus make SCON more accurate than NCON.5Figure 2. Example of the track display portion of the SAFA screen at 1200 UTC 03 September 1999for Tropical Storm Wendy, except that the subsequent track of the TC has been added for thepurpose of verification. Notice the difference between the inaccurate numerical consensus track(NCON), which is comprised of all available numerical models, and the selective consensus (SCON)track, which in this case is based only on the track of GFDN (green) because the tracks of the other three numerical models (NGPS, EGRR, and JGSM in red) were correctly rejected due to indicationof excessive direct cyclone interaction (E-DCI).IMPACT/APPLICATIONA thorough analysis of the results of the SAFA prototype test is still pending and subject to endorsementby JTWC. However, it is anticipated that the feasibility of using SAFA to help the forecaster successfullyidentify situations in which the official track forecast should be based on a selective consensus of theavailable numerical TC track forecast models (as in Fig. 2) as opposed to an indiscriminate numericalconsensus will be demonstrated, thus warranting further development of  the SAFA prototype into a fullyoperational system.RELATED PROJECTSThis project is a follow-on to, and utilizes the results of, the project entitled SYSTEMATICAPPROACH TO TROPICAL CYCLONE TRACK FORECASTING by Lester E. Carr III andcollaborators, which appeared in the FY97 annual report.6REFERENCESCarr, L. E., III, and R. L. Elsberry, 1994: A systematic and integrated approach to tropical cyclone trackforecasting. Part I. Approach overview and description of meteorological basis. Tech. Rep. NPS-MR-94-002, Naval Postgraduate School, Monterey, CA 93943-5114, 273 pp.Carr, L. E., III, and R. L. Elsberry, 1999: Systematic and integrated approach to tropical cyclone trackforecasting. Part III: Traits knowledge base for JTWC track forecast models in the western NorthPacific. Tech. Rep. NPS-MR-99-002, Naval Postgraduate School, Monterey, CA 93943-5114, 227 ppCarr, L. E., III,  R. L. Elsberry, and M. A. Boothe, 1997: Condensed and updated version of thesystematic approach meteorological knowledge base--Western North Pacific. Tech. Rep. NPS-MR-98-002, Naval Postgraduate School, Monterey, CA 93943-5114, 169 pp.PUBLICATIONSCarr, L. E., III, and R. L. Elsberry, 1999: Systematic and integrated approach to tropical cyclone trackforecasting. Part III: Traits knowledge base for JTWC track forecast models in the western NorthPacific. Tech. Rep. NPS-MR-99-002, Naval Postgraduate School, Monterey, CA 93943-5114, 227 ppCarr, L. E., III, R. G. Schnabel, and R. L. Elsberry, 1999: Systematic approach to tropical cyclone trackforecasting-2: Analysis of large track forecast errors by NOGAPS and GFDN in the western NorthPacific during 1997. Preprints, 23rd Conf. on Hurr. Trop. Meteor., Amer. Meteor. Soc., Boston, MA02108, 839-842.Peak, J. E., L. E. Carr, III, and R. L. Elsberry, 1999: Systematic approach to tropical cyclone trackforecasting-3: Development of a tropical cyclone track forecasting expert system. Preprints, 23rd Conf.on Hurr. Trop. Meteor., Amer. Meteor. Soc., Boston, MA 02108, 1057-1060.IN-HOUSE/OUT-OF-HOUSE RATIOS28%/72%

Development of an Expert System Based on the Systematic Approach to Tropical Cyclone Track Forecasting

https://calhoun.nps.edu/bitstream/handle/10945/68947/Carr-Elsberry_Development_Of_An_Expert_System_1999-09.pdf?sequence=1

Development of an Expert System Based on the Systematic Approach to Tropical Cyclone Track Forecasting

Abstract

Similar works

Full text

Available Versions

Calhoun, Institutional Archive of the Naval Postgraduate School