Dispatcher is a computer simulation being used in a series of experiments involving the study of real-time dynamic decision making. The design of the simulation is based on current theory of providing decision support for this type of decision making task. This paper provides an overview of the process used in designing and implementing Dispatcher, as well as a description of some of the main features of the simulation as they relate to current research efforts in the study of decision support for RTDDM

Ballou, Deborah J.

Huguenard, Brian R.

English

AIS Electronic Library (AISeL)

Association for Information SystemsAIS Electronic Library (AISeL)SAIS 2006 Proceedings Southern (SAIS)3-1-2006Dispatcher: A Theory-Based Design for Study ofReal-Time Decision-MakingBrian R. Huguenardbhuguenard@tntech.eduDeborah J. BallouFollow this and additional works at: http://aisel.aisnet.org/sais2006This material is brought to you by the Southern (SAIS) at AIS Electronic Library (AISeL). It has been accepted for inclusion in SAIS 2006 Proceedingsby an authorized administrator of AIS Electronic Library (AISeL). For more information, please contact elibrary@aisnet.org.Recommended CitationHuguenard, Brian R. and Ballou, Deborah J., "Dispatcher: A Theory-Based Design for Study of Real-Time Decision-Making" (2006).SAIS 2006 Proceedings. 20.http://aisel.aisnet.org/sais2006/20Proceedings of the 2006 Southern Association for Information Systems Conference 117 DISPATCHER: A THEORY-BASED DESIGN FOR STUDY OF REAL-TIME DYNAMIC DECISION-MAKING Brian R. Huguenard Tennessee Technological University bhuguenard@tntech.edu Deborah J. Ballou University of Notre Dame dballou@nd.edu Abstract Dispatcher is a computer simulation being used in a series of experiments involving the study of real-time dynamic decision making.  The design of the simulation is based on current theory of providing decision support for this type of decision making task.  This paper provides an overview of the process used in designing and implementing Dispatcher, as well as a description of some of the main features of the simulation as they relate to current research efforts in the study of decision support for RTDDM. Keywords:  decision making, simulation tool, interface design  Introduction A real-time, dynamic decision making (RTDDM) environment is one with the following characteristics: (1) A series of decisions is required to reach the goal.  Achieving and maintaining control is a continuous activity requiring many decisions, each of which can only be understood in the context of the other decisions; (2) The decisions are not independent…later decisions are constrained by earlier decisions, and, in turn, constrain those that come after them; (3) The state of the decision problem changes, both autonomously and as a consequence of the decision maker's actions; (4) The decisions have to be made in real time. (Brehmer, 1990; Brehmer, 1992). RTDDM tasks are frequently encountered across a wide variety of domains: governmental response to natural disasters; military actions carried out during a conflict; the scheduling of jobs to machines in an industrial environment; and air-traffic controllers managing the flow of aircraft.  Substantial research literature has described the many difficulties of decision making in RTDDM environments and has attempted to identify the determinants of effective decision making (Gonzalez, Vanyukov, & Martin, 2005; Gibson, Fichman, & Plaut, 1997; Kluger & DeNisi, 1996; Kerstholt, 1994; Kleinmuntz, 1993; Brehmer, 1992).   Given the prevalence of RTDDM in many important domains, researchers have also turned to the study of how to best provide decision support for real-time, dynamic decision making tasks (Gonzalez, 2005; Lerch & Harter, 2001; Hsiao, 2000; Brehmer, 1995; Diehl & Sterman, 1995; Sengupta & Abdel-Hamid, 1993).  Several different categories of decision support for RTDDM have been identified, including the use of outcome feedback (in which decision makers are informed of their performance results), cognitive feedback (in which decision makers are given instruction on how to perform the decision task), and feedforward (in which decision makers are given the ability to perform “what-if” type explorations of potential future decisions).  The research reported here concerns the development of a computer simulation (called Dispatcher) built for the investigation of alternative forms of decision support in RTDDM environments.   Dispatcher is being used in a series of experimental studies in which subjects are observed performing decision tasks while being provided with different types of decision support.  This paper discusses the process of developing and refining this simulation tool. Proceedings of the 2006 Southern Association for Information Systems Conference 118  Task Domain Description In searching for an RTDDM task domain to use as the basis for our simulation, we wanted a task that would not be totally foreign to our pool of subjects (undergraduate students).  Given the frequency with which undergraduates tend to consume delivered pizza, we decided to base our simulation on the task of dispatching pizzas to drivers for delivery. Before developing the simulation, we conducted interviews with managers from four different pizza delivery businesses.   From the results of these interviews we developed a generic description of the “typical” pizza dispatcher task, and satisfied ourselves that it met all the requirements to be classified as a RTDDM task.  Orders for pizzas come from customers at varying rates during a given shift.  In order to remain competitive with other pizza delivery businesses, there is always pressure to minimize the time between placement of an order and delivery of that order.  When an order is ready for delivery, the dispatcher must decide which of the available drivers should deliver that order.  If the volume of orders coming in is high, the dispatcher may assign more than one order to a driver.  As each driver is dispatched with an order, the set of available drivers decreases, until drivers return from completed deliveries.  We can see that this decision task does indeed satisfy the requirements of a RTDDM task: it involves a series of interdependent decisions (which orders to assign to which drivers), the environment changes not only because of changing demand, but also because of the previous dispatch decisions made, and all decisions are made under time pressure.  Developing the Simulation Development of the simulation tool was an interesting process, since the researchers involved were at two different universities more than 500 miles apart.  Much of the dialog concerning interface design and simulation features had to be conducted by way of telephone, email, and collaborative software.  For example, during the initial design of the basic interface, we debated the merits of representing the delivery area map as a grid of rectangular quadrants, or as a series of concentric circles, or as a combination of both.  Appendix A shows a whiteboard session used by the authors to brainstorm during this initial design process.  The whiteboard in Appendix A shows rather crude drawings used to represent an initial attempt at using radio buttons to indicate when a driver was to be sent on a delivery (upper left corner of whiteboard), initial ideas of what the delivery map would look like (upper right corner), initial ideas on how to provide performance feedback to the user (middle of whiteboard), and at the bottom of the whiteboard are some text blocks reflecting ideas on various assumptions and decision rules to be considered.  After several initial brainstorming sessions such as this, we began actual development of the simulation using Visual Basic as the development environment.  As development progressed, intermediate versions of the simulation were sent back and forth between the authors as email attachments, until finally a satisfactory working version (Dispatcher) was completed.  Overview of Dispatcher Interface There are several different configurations of the Dispatcher user interface, depending on which aspects of RTDDM are being investigated.  Figure 1 shows a screen image of the “basic” user interface.    On the left of the screen is the delivery map, represented as a series of concentric circles labeled as A, B, and C.  The circles are further divided by a simple set of x/y axes, allowing for the further labeling of specific sections of each circle, as in C1, B2, A4, etc.  These labels provide one of the ways in which the location of an order can be specified.  The very center point of the map represents the location of the pizza store itself…all delivery trips begin and end there.  The numbers that appear on the map represent different orders for pizzas, with the lower value numbers representing orders that were placed longer ago, and the higher value numbers representing more recent orders.  As a simulation session begins, order numbers begin to appear on the map, with the rate of order appearance and their location randomly determined.   The subject’s job is to assign these orders to delivery cars so as to minimize the total amount of time past delivery deadlines for all pizzas.    In the upper left of the screen are two different time displays.  The left-most of these time displays represents the current time in “simulation time”.  Every run of the simulation starts at 4:00 and ends at 8:30.  This 4.5 hour run (in simulation time) is accomplished in 15 minutes of real time.  Once an order has appeared on the map, the subject has 30 minutes (simulation time) before the delivery deadline for that order has passed.  For all orders that are delivered past their deadlines, the number of minutes past deadline is added to the running total displayed in the right-most time display.  This “Total Time Over Deadline” represents the subject’s overall performance measure…the lower the value the better.    Figure 1.  User Interface to Simulation The delivery cars are represented in the upper right of the screen.  There are 5 cars to be used, and each car can hold up to 4 different orders.  To assign an order to a car, the subject clicks on an order on the map with the mouse, and then in the pop-up menu that appears, chooses a car.  Once assigned to a car, the order number is highlighted on the map and appears in one of the available slots in the car (as in Figure 1, where order number 13 is highlighted in section C3 of the map and has been assigned to car 5).  To send a car out to deliver its order(s), the subject clicks on the “Send” checkbox to the right of the car.  Once a car has been sent on a trip, any orders it is carrying are removed from the map.  Progress of the car on its trip can be monitored through the progress-bar shown to the right of each checkbox. If a car contains orders but has not yet been sent on a trip, the subject can “undo” the assignment of orders by dragging order numbers from the car back to the map, or by dragging order numbers to another available car slot.  Once a car has been sent, however, the car contents cannot be modified. At the lower right of Figure 1 is a scrollable list showing the history of all orders that have appeared on the map, detailing each order’s location, deadline, and (once the order has been delivered) a performance value.  Negative performance values represent early deliveries (before the deadline), and positive values represent late deliveries.  Proceedings of the 2006 Southern Association for Information Systems Conference 119 Proceedings of the 2006 Southern Association for Information Systems Conference 120 Using Dispatcher To Study RTDDM While the previous section gave an overview of Dispatcher’s basic user interface, there are many additional features that can be enabled in order to address the study of different types of support for RTDDM.   We will now briefly describe some of the basic and additional interface features as they relate to the main categories of decision support for RTDDM that were mentioned in the Introduction: outcome feedback, cognitive feedback, and feedforward.   Outcome Feedback Outcome feedback provides the decision maker with information concerning the performance-related results of actions already taken.  Dispatcher provides this type of support in two main ways: through the “Total Time Over Deadline” display at the top of the screen, and through the order history list at the bottom right of the screen.  In some studies, subjects are taught a particular decision heuristic to be implemented while assigning orders…for these subjects, an additional form of outcome feedback can be provided in the form of a “Rule Match” percentage.  This percentage gives the subject feedback on how well their most recent decision matches to a correct implementation of the rule they were taught…a value of 100% indicating that the rule was implemented correctly, and values below 100% indicating increasing deviation from correct rule implementation.  When the “Rule Match” percentage is used, it is displayed to the right of “Total Time Over Deadline”.  Cognitive Feedback Cognitive feedback provides the decision maker with advice on how to perform the decision task.  Dispatcher can provide this type of support though the delivery map.  After any given order has been assigned to a car, the subject then has to determine which order (out of all the unassigned ones currently on the map) to assign next…this can involve a lot of searching when there are many orders on the map.  During the training sessions for using Dispatcher, some subjects are taught a decision heuristic to be used in choosing the next order for assignment.  This heuristic is not optimal, and the subjects are told that they might be able to perform better than the heuristic after they become more experienced.  One version of Dispatcher will highlight the order on the map that should be assigned next based on this decision heuristic, thus providing cognitive feedback for this portion of the task.  In addition, when the simulation becomes very busy with incoming orders subjects need to assign more than one order per car. Under these conditions the choice of which orders to assign to which cars becomes quite complex.  One version of Dispatcher will not only highlight the order that should first be assigned, but also the other orders that should be assigned with it (according to the heuristic).  For this type of cognitive feedback all the orders that should be assigned together are shown as a loop, being connected by a line starting and ending at the store location (the center of the map).  Feedforward Feed forward allows the user to try out possible future decisions in a “what-if” mode, without committing to them.  We have not yet implemented feed forward support in Dispatcher, but plan on providing it in at least two different ways.  The first would involve telling the subject when each car would be available again for another trip.  For example, if car1 is currently on a trip, the user would see a display of the time (in simulation time) when car1 would be returning to the store location.  The second form of feedforward would involve orders that have been placed into a “loop”.  In addition to assigning orders directly to cars, some subjects are allowed to construct loops of multiple orders that begin and end at the store location…once a loop has been constructed, it can be assigned as a whole to a car.  For these subjects feedforward can be provided by telling them (before a loop of orders has been assigned to a car) what the total amount of time over deadline will be if the loop of orders is actually sent on a delivery trip.  Proceedings of the 2006 Southern Association for Information Systems Conference 121 Conclusion Real time dynamic decision making is an important component of many real-world decision tasks, increasing the difficulty of decision-makers reaching high levels of performance.  The provision of effective decision support for RTDDM tasks is therefore an important goal.  Dispatcher is a flexible simulation tool that allows us to explore and evaluate many different methods of providing decision support for RTDDM.  More importantly, the design of Dispatcher is driven by current theory concerning the types of decision support that could be useful in the execution of RTDDM tasks.  Dispatcher provides an example of the use of theory to drive software design, followed by the use of that same software as a workbench for testing the theory itself.   References Brehmer, B. (1990). Strategies in real-time, dynamic decision making. In R. M. Hogarth (Ed.), Insights in Decision Making (pp. 262–279). Chicago, IL: University of Chicago Press. Brehmer, B. (1992). Dynamic decision making: Human control of complex systems. Acta Psychologica, 81(3), 211–241. Brehmer, B. (1995). Feedback delays in complex dynamic decision tasks. In P. A. Frensch & J. Funke (Eds.), Complex Problem Solving: The European Perspective. Hillsdale, NJ: Lawrence Erlbaum Associates. Diehl, E., & Sterman, J. D. (1995). Effects of feedback complexity on dynamic decision making. Organizational Behavior and Human Decision Processes, 62(2), 198–215. Gibson, F. P., Fichman, M.,  & Plaut, D. C. (1997).  Learning in dynamic decision tasks: Computational model and empirical evidence.  Organizational Behavior and Human Decision Processes, 71, 1-35. Gonzalez, C. (2005).  Decision support in real-time, dynamic decision-making tasks.  Organizational Behavior and Human Decision Processes, 96(2), 142-154. Gonzalez, C., Vanyukov, P., & Martin, M.K. (2005). The use of  micro-worlds to study dynamic decision making.  Computers in Human Behavior, 21, 273-286. Hsiao, N. (2000). Exploration of outcome feedback for dynamic decision making. University at Albany, State University of New York, Albany, NY. Kerstholt, J.H. (1994). The effect of time pressure on decision-making behavior in a dynamic task environment. Acta Psychologica, 86, 89-104. Kleinmuntz, D. N. (1993).  Information processing and misperceptions of the implications of feedback in dynamic decision making.  System Dynamics Review, 9(3), 223-237. Kluger, A. N., & DeNisi, A. (1996).  Effects of feedback intervention on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory.  Psychological Bulletin, 119(2), 254-284. Lerch, F. J., & Harter, D. E. (2001). Cognitive support for real-time dynamic decision making. Information Systems Research, 12(1), 63–82. Sengupta, K., & Abdel-Hamid (1993). Alternative conceptions of feedback in dynamic decision environments: An experimental investigation. Management Science, 39(4), 411–428.    Appendix A: Whiteboard Session from Initial Interface Design    Proceedings of the 2006 Southern Association for Information Systems Conference 122 

Dispatcher: A Theory-Based Design for Study of Real-Time Decision-Making

https://aisel.aisnet.org/cgi/viewcontent.cgi?article=1019&amp;context=sais2006

Dispatcher: A Theory-Based Design for Study of Real-Time Decision-Making

Abstract

Similar works

Full text

Available Versions

AIS Electronic Library (AISeL)