Selection of assignments for a constrained inventory of assets and associated services requires comparable measures of their value to the potential recipients, and any associated costs. The dissemination of information or intelligence products over bandwidth limited and security constrained channels similarly requires consideration of the associated values and costs. Similar reasoning is applicable to the selection of variant effects and methods. We present an approach to value definition and prediction in the mission performance characteristics resulting from variant deployments

James, JR

Thornley, DJ

English

Spiral - Imperial College Digital Repository

    Abstract— Selection of assignments for a constrained inventory of assets and associated services requires comparable measures of their value to the potential recipients, and any associated costs. The dissemination of information or intelligence products over bandwidth limited and security constrained channels similarly requires consideration of the associated values and costs. Similar reasoning is applicable to the selection of variant effects and methods. We present an approach to value definition and prediction in the mission performance characteristics resulting from variant deployments.  Index Terms—value, utility, VoI, UoI, performance analysis I. INTRODUCTION lanning, design, deployment, support and maintenance of missions under tight, complex constraints all require a concrete, quantitative view of value in support of choice and prioritization. We are developing an achievement-centric perspective on value that builds measures using performance prediction and evaluation [1]. These analyses generate concrete utility characteristics of both benefit and cost as distinct calculations that must then be traded off according to customer-defined priorities. Definition, selection and quantitative assessment of computable expressions of command intent, operations orders, plans and goals present a research area that requires a combination of a holistic theoretical treatment [2] and detailed analysis of practice [3]. A tactical system’s performance is defined by satisfaction of command requirements posed at a range of abstractions, predicated on aspects of a number of entities within or outside our control. For example, a counter-insurgency mission requires that ingress of entities conforming to a given class be interdicted. A prediction of the probability and timing of successful interdiction requires a model covering intelligence services, combination of assets, prioritization of communications channels, situational awareness building, and physical activity. We describe the beginning of a collaboration between the Mission Abstraction Requirements and Structure (MARS) project [1,2] in the Network and Information Science International Technology Alliance (ITA), and the Flowing  This research was sponsored by the U.S. Army Research Laboratory and the U.K. Ministry of Defence and was accomplished under Agreement Number W911NF-06-3-0001. The views and conclusions contained in this document are those of the author(s) and should not be interpreted as representing the official policies, either expressed or implied, of the U.S. Army Research Laboratory, the U.S. Government, the U.K. Ministry of Defence or the U.K.  Government. The U.S. and U.K. Governments are authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation hereon. Valued Information Project (FVI) of USMA [3]. II. BACKGROUND Starr [4] emphasizes the need for structured assessment of goal-oriented measures, rather than intuitively supportable trend-based “goodness” measures. The Flowing Valued Information project [3] brings together domain experts that combine to address open research issues from formulating computable command intent, decomposition and integration of priorities in the command hierarchy, to predicting what can be communicated under bandwidth and security constraints [5]. The NATO Code of Best Practice on Assessment of Command and Control (COBP) [6] explains how this hierarchical breakdown of measures of merit (MoM) must reflect deterministic preferences and stochastic realities of the mission. The FVI project includes an emphasis on the realities of information dissemination at the lowest tactical level in coalition OOTW. The MARS project includes a component-based timed analytic stochastic modeling (TASM) approach to quality and utility prediction [7]. By developing assessments and structure for predictive models of information use, and a unified approach to command requirement specification and instrumentation in TASM, we will also create opportunities to incorporate concrete value estimation into planning policy and mechanisms research (e.g. [8]).  III. QUALITY, UTILITY AND VALUE A camera that includes a mission’s area of interest in its field of regard is clearly potentially of use when imagery is required. A NIIRS rating asserting that this will allow an operator to detect guerilla activity in dense woodland indicates that its information is of use to a guerilla activity detection mission. It is of sufficient quality, and thus will present some utility, but how much utility is not defined within the equipment or the application. Instead, the utility characteristic of a mission element with respect to an achievement, which in this case could be a state labeled “guerilla activity correctly detected”, emerges from the interaction of the relevant mission elements, the majority of which are non-deterministic in some sense, over time. Thus, predicting utility characteristics requires timed stochastic modeling. Value is assessed in practice by combining utility characteristics at one or more levels of responsibility in the mission that may contradict each other, as explained by Dodd, Moffat and Smith [9]. Value is defined as utility measured at a suitable level in the model, but as detailed in the COBP [6], this is commonly Defining and estimating value to a mission David J Thornley, Imperial College London and John R. James, USMA P   impractical, or poorly defined, and so requires a hierarchical breakdown of measures of merit (MoM) that support reasoned construction of an estimate of the value that should be used for comparison of candidate methods, assets or information. IV. MARS AND FVI The FVI project seeks to develop methods for transitioning measures of the degree to which an IP fulfils command requirements to methods for supporting decisions to share information. Implicit in this is the need for a clear definition of how the fulfillment of a command requirement may be practically expressed as a degree, rather than an absolute, so that cost/benefit analyses may be based on a concrete, quantitative end-to-end treatment of missions from command down to physics and back.    Figure 1. Utility comparable in performance predictions as probability density of achievement/failure over time. In the predicted outcomes of two options for the supporting service chosen from A or B. In the first graph, service A clearly leads to generally earlier completion than B. In the second, while B’s mean completion time is earlier, late completion is more likely than with service A. In the third, we show development of negative outcomes. The mean time to achievement of positive outcomes is the same in each case, and while A’s success rate is lower, failure is separated, for better abort decisions.  The major objectives of the Flowing Valued Information are: (1) to advance the science for estimation of the relative value of information for goal-oriented behaviors and (2) to use these value estimates to dynamically adjust movement of data to maximize value of information moved over time. FVI investigations will extend previous results achieved by team members [5,10] to provide an estimate of the current and future states of an operation at the lowest tactical levels across the diverse networks of interest, In the MARS approach, utility characteristics are constructed as time-dependent probability of satisfaction of a predicate defined as a logical construct on mission states. For example, in our validation experiment using fratricide risk in decision making, we measure the probabilities that an entity is correctly or incorrectly identified. We also measure the quality of the decision as described in the COBP ([6] section 5.4.1). To find a definition for value that supports quantitative estimation, we note that value is perceived by a recipient or commissioner of an entity or process. The COBP explains that value is given by utility estimated at a suitable level in the mission. We have performed a basic demonstration that mission utility and cost characteristics can be predicted from a mission description [7], and we can identify some basic conditions for perception of value. Higher value is perceived when achievement is earlier, success more likely and personnel safer. This is illustrated in Figure 1 with hand drawn examples, and initial results of stochastic modeling of a decision making sequence from [7] is shown in Figure 2.   Figure 2. Earlier is better. Our example model of a decision making scenario [7] predicts instantaneous decision rates of correct pass and fratricide risk in a Red/Blue screen. Gound truth is Blue, but briefing places high prior confidence in Red. (C) Fratricide risk arises from insufficient information arriving and being assimilated before the decision point from EO, HQ and automated target ID. (B) Provision of a scout supports earlier decision making, and reduces fratricide rates. (A) Siting the scout at the observation position further improves value, but by a significantly smaller amount. V. CONCLUSIONS Value of an information product is perceived by a recipient, estimated by a handler, and required by a commander. Value is defined as utility measured in the state of a whole mission, but computing or even modeling this is generally impossible. Value is judged in a tradeoff between utility characteristics and costs, often addressing the needs of a range of customers. The FVI project seeks to codify command requirements and operational orders that the MARS project breaks down into components and synthesizes to give performance metrics in support of value estimation. REFERENCES [1] DJ Thornley, RI Young, JP Richardson, “Introducing MARS”, ITA tech report 2008 and Imperial College London Dept of Comp DTR09-10 [2] D. J. Thornley, R. Young, J. Richardson “From mission specification to quality of information measures – closing the loop in military sensor”, ACITA 2008 (see http://usukita.org/papers/3876/details.html) [3] "Flowing Valued Information", US DoD contract MIPR9FDATXR048 [4] SH Starr, “Lessons Recorded from Applying the NATO Code of Best Practice for C2 Assessment to OOTW”, ICCRTS 2001 [5] J James and F Mabry, "Building Trustworthy Systems: Guided State Estimation as a Feasible Approach for Interpretation, Decision and Action Based on Sensor Data", HICSS, 2004 [6] NATO Code of Best Practice (COBP) on the Assessment of C2, www.rta.nato.int/Pubs/RDP.asp?RDP=RTO-TR-009 [7] DJ Thornley, RI Young, JP Richardson, “Toward mission-specific service utility estimation using analytic stochastic process models”, SPIE Defense, Security and Sensing 7352A (2009) [8] D Mott, J Hendler, Progress on the Collaborative Planning Model, First Annual Conference of ITA, September, 2007 [9] L Dodd, J Moffat and J Smith, “Discontinuity in decision-making when objectives conflict: a military command decision case study”, Journal of the Operational Research Society (2006) 57, 643–654  [10] J James and F Maymi, "An Environment for Comparing Command and Control Architectures", Proceedings of the Command and Control Research and Technology Symposium, San Diego, CA, June, 2004. 

Defining and estimating value to a mission

http://spiral.imperial.ac.uk/bitstream/10044/1/95284/2/DTR09-12.pdf

Defining and estimating value to a mission

Abstract

Similar works

Full text

Available Versions

Spiral - Imperial College Digital Repository