VERIFICATION AND VALIDATION OF NAVAL AIR WARFARE CENTER AIR DIVISION (NAWCAD) F/A-18 SUSTAINMENT MODEL

This thesis addresses Verification and Validation (V&V) efforts for the Naval Air Warfare Center Air Division (NAWCAD) FA-18 sustainment model to ensure the model can make predictions regarding Tier One mission needs of the Air Division. For the development of the model, the Institute for Defense Analytics (IDA) used SIMLOX as the software tool and developed a data pre-processing pipeline using historical supply data as the inputs for the pipeline. Once the data is pre-processed, the data is used as inputs to the model. This thesis verified the model for correctness using a structured walkthrough. Model validation was performed to ensure the model can predict the number of expenditures using a t-test and percent error between historical and projected expenditures. Using a t-test, the model failed to produce a confidence level of 0.95 to use the model for sustainment decisions in the future. Additionally, the model under-predicted the total number of expenditures required by 66 percent and 70 percent for all depot-level repairable items. These findings will be used to improve the model for the purpose of receiving accreditation as a performance and pricing model.Lieutenant, United States NavyApproved for public release. Distribution is unlimited

EVAQ: Person-Specific Evacuation Simulation for Large Crowd Egress Analysis

Timely crowd evacuation in life-threatening situations such as fire emergency or terrorist attack is a significant concern for authorities and first responders. An individual’s fate in this kind of situation is highly dependent on a host of factors, especially (i) agent dynamics: how the individual selects and executes an egress strategy, (ii) hazard dynamics: how hazards propagate (e.g., fire and smoke spread, lone wolf attacker moves) and impair the surrounding environment with time, (iii) intervention dynamics: how first responders intervene (e.g., firefighters spread repellents) to recover environment. This thesis presents EVAQ, a simulation modeling framework for evaluating the impact of these factors on the likelihood of survival in an emergency evacuation. The framework captures the effect of personal traits and physical habitat parameters on occupants’ decision-making. In particular, personal (i.e., age, gender, disability) and interpersonal (i.e., agent-agent interactions) attributes, as well as an individual’s situational awareness are parameterized in a deteriorating environment considering different exit layouts and physical constraints. Further, the framework supports a variety of hazard propagation schemes (e.g., fire spreading in a given direction, lone wolf attacker targeting individuals), and intervene schemes (e.g., firefighters spreading repellents, police catch the attacker) to support a wide range of emergency evacuation scenarios. The application of EVAQ to crowd egress planning in an airport terminal and a shopping mall in the fire emergency is presented in this thesis, and results are discussed. Result shows that the likelihood of survival decreases with a decrease in availability of the nearest exits and a resulting increase in congestions in the environment. Also, it is observed that the incorporation of group behavior increases the likelihood of survival for children, as well as elderly and disabled people. In addition, several verifications and validation tests are performed to assess the reliability and integrity of EVAQ in comparison with existing evacuation modeling tools. As personalized sensing and information delivery platforms are becoming more ubiquitous, findings of this work are ultimately sought to assist in developing and executing more robust and adaptive emergency mapping and evacuation plans, ultimately aimed at promoting people’s lives and wellbeing

Urban segregation as a complex system : an agent-based simulation approach

Urban segregation represents a significant barrier for achieving social inclusion in cities. To overcome this, it is necessary to implement policies founded upon a better understanding of segregation dynamics. However, a crucial challenge for achieving such understanding lies in the fact that segregation is a complex system. It emerges from local interactions able to produce unexpected and counterintuitive outcomes that cannot be defined a priori. This study adopts an agent-based simulation approach that addresses the complex nature of segregation. It proposes a model named MASUS, Multi-Agent Simulator for Urban Segregation, which provides a virtual laboratory for exploring theoretical issues and policy approaches concerning segregation. The MASUS model was first implemented for São José dos Campos, a medium-sized Brazilian city. Based on the data of this city, the model was parameterized and calibrated. The potential of MASUS is demonstrated through three different sets of simulation experiments. The first compares simulated data with real data, the second tests theories about segregation, and the third explores the impact of anti-segregation policies. The first set of experiments provides a retrospective validation of the model by simulating the segregation dynamics of São José dos Campos during the period 1991-2000. In general, simulated and real data reveal the same trends, a result that demonstrates that the model is able to accurately represent the segregation dynamics of the study area. The second set of experiments aims at demonstrating the potential of the model to explore and test theoretical issues about urban segregation. These experiments explore the impact of two mechanisms on segregation: income inequality and personal preferences. To test the impact of income inequality, scenarios considering different income distributions were simulated and compared. The results show how decreasing levels of income inequality promote the spatial integration of different social groups in the city. Additional tests were conducted to explore how the preferences of high-income families regarding the presence of other income groups could affect segregation patterns. The results reveal that the high levels of segregation were maintained even in a scenario where affluent households did not take into account the income composition of neighborhoods when selecting their residential location. Finally, the third set of experiments provides new insights about the impact of different urban policies on segregation. One experiment tests whether the regularization of clandestine settlements and equitable distribution of infrastructure would affect the segregation trends in the city. The simulated outputs indicate that they had no significant impact on the segregation patterns. Besides this test focusing on a general urban policy, two specific social-mix policy approaches were explored: poverty dispersion and wealth dispersion. The results suggest that policies based on poverty dispersion, which have been adopted in cities in Europe and the United States, are less effective in developing countries, where poor families represent a large share of the population. On the other hand, the policy based on wealth dispersion was able to produce substantial and long-term improvements in the segregation patterns of the city.Städtische Segregation als komplexes System : Ein agentenbasierter Simulationsansatz Die städtische Segregation stellt eine bedeutende Barriere für die Erreichung der sozialen Inclusion in den Städten dar. Um diese zu überwinden, ist es notwendig, eine Politik zu betreiben, die die Dynamiken der Segregation besser versteht und berücksichtigt. Eine besondere Herausforderung für ein besseres Verständnis dieser Dynamik ist die Tatsache, dass Segregation ein komplexes System ist. Dieses System entsteht aus lokalen Interaktionen, die zu unerwarteten und nicht eingängigen Ergebnissen führt, die nicht von vornherein bestimmt werden können. Diese Studie wendet einen multi-agenten Simulationsmodel an, das die komplexe Natur der Segregation berücksichtigt. Es schlägt ein Modell mit dem Namen MASUS (Multi-Agent Simulator for Urban Segregation) vor. Dieses bietet ein virtuelles Labor für die Untersuchung der theoretischen Aspekte und Politikansätze der Segregation. Das Modell wurde für São José dos Campos, eine mittelgroße brasilianische Stadt, eingesetzt. Das Modell wurde auf der Grundlage der Daten dieser Stadt parametisiert und kallibriert. Das Potenzial von MASUS wird durch drei verschiedene Arten von Simulationsexperimente dargestellt. Die erste vergleicht simulierte Daten mit realen Daten, die zweite prüft Segregationstheorien, und die dritte untersucht die Auswirkungen von Antisegregationspolitik. Die erste Gruppe von Experimenten liefert eine rückblickende Validierung des Modells durch die Simulation der Segregationsdynamiken von São José dos Campos im Zeitraum 1991-2000. Die simulierten und realen Daten zeigen im Allgemeinen die gleichen Trends. Dies zeigt, dass das Modell in der Lage ist, die Segregationsdynamik im Untersuchungsgebiet korrekt darzustellen. Die zweite Gruppe von Experimenten hat zum Ziel, das Potenzial des Modells hinsichtlich der Untersuchung und Prüfung der theoretischen Aspekte städtischer Segregation darzustellen. Diese Experimente untersuchen die Auswirkung von zwei Mechanismen auf Segregation: Einkommensungleichheit und persönliche Präferenzen. Um die Auswirkungen von Einkommensungleichheit zu prüfen, wurden Szenarien mit unterschiedlichen Einkommensverteilungen simuliert und verglichen. Die Ergebnisse zeigen wie abnehmende Einkommenshöhen die räumliche Integration von verschiedenen sozialen Gruppen in der Stadt fördern. Zusätzliche Tests wurden durchgeführt, um zu untersuchen wie die Präferenzen von Haushalten mit hohen Einkommen im Bezug auf das Vorhandensein anderer Einkommensgruppen die Segregationsmuster beeinflussen könnten. Die Ergebnisse zeigen, dass die Segregation auf hohem Niveau blieb sogar in einem Szenario wo wohlhabende Haushalte das Einkommensgefüge der Nachbarschaft bei der Wahl ihrer Wohngegend nicht berücksichtigten. Die dritte Gruppe von Experimenten führt zu neuen Einsichten über die Auswirkungen von verschiedenen städtischen politischen Maßnahmen auf die Segregation. Ein Experiment prüft ob die Regulierung von illegalen Siedlungen und die gleichmäßige Verteilung der Infrastruktur die Segregationstrends in der Stadt beeinflussen. Die Ergebnisse der Simulation zeigen, dass diese keine signifikante Auswirkung auf die Segregationsmuster haben. Neben diesem Test, der die allgemeine städtische Politik zum Inhalt hat, wurden zwei Ansätze der spezifischen Sozialen-Mix-Politik untersucht: Armutsverteilung und Wohlstandsverteilung. Die Ergebnisse deuten daraufhin, dass eine Politik der Armutsverteilung, die aus europäischen und nordamerikanischen Städten bekannt ist, weniger wirkungsvoll in Entwicklungsländern ist, wo arme Familien einen Großteil der Bevölkerung darstellen. Auf der anderen Seite führte eine Politik der Wohlstandsverteilung zu erheblichen und langfristigen Verbesserungen der Segregationsmuster der Stadt

R&D for computational cognitive and social models : foundations for model evaluation through verification and validation (final LDRD report).

Sandia National Laboratories is investing in projects that aim to develop computational modeling and simulation applications that explore human cognitive and social phenomena. While some of these modeling and simulation projects are explicitly research oriented, others are intended to support or provide insight for people involved in high consequence decision-making. This raises the issue of how to evaluate computational modeling and simulation applications in both research and applied settings where human behavior is the focus of the model: when is a simulation 'good enough' for the goals its designers want to achieve? In this report, we discuss two years' worth of review and assessment of the ASC program's approach to computational model verification and validation, uncertainty quantification, and decision making. We present a framework that extends the principles of the ASC approach into the area of computational social and cognitive modeling and simulation. In doing so, we argue that the potential for evaluation is a function of how the modeling and simulation software will be used in a particular setting. In making this argument, we move from strict, engineering and physics oriented approaches to V&V to a broader project of model evaluation, which asserts that the systematic, rigorous, and transparent accumulation of evidence about a model's performance under conditions of uncertainty is a reasonable and necessary goal for model evaluation, regardless of discipline. How to achieve the accumulation of evidence in areas outside physics and engineering is a significant research challenge, but one that requires addressing as modeling and simulation tools move out of research laboratories and into the hands of decision makers. This report provides an assessment of our thinking on ASC Verification and Validation, and argues for further extending V&V research in the physical and engineering sciences toward a broader program of model evaluation in situations of high consequence decision-making

The Power of Forgotten Opinions: Why an Organization Chooses Inaction over the Public's Safety

This dissertation aims to explain why organizations and their members cannot prevent large-scale accidents even when there is prior information concerning a problem. Large-scale accidents, which are also called organizational accidents, are low-probability events, but once they occur, the outcomes are disastrous both in and out of an organization. However, due to their rarity and specific characteristics, few organization theories explain in a simple, straightforward manner how members' choices lead to organizational-level decisions that cause such accidents. This shortcoming is conspicuous with respect to members' decisions in a gray zone, in which no clear crises or threats to organizational goals and performance are present. In this dissertation, I explain the relationship between members' choices and organizational-level decisions in a gray zone rather than organizational preparedness for, and responses to, accidents or crises. For this purpose, I draw on theories on social psychology, organizational cognition, and group behaviors and utilize multi-level models and agent-based simulation. The particular focus of this dissertation is how differences in organizational conditions surrounding speaking up on and taking action against potential problems change members' and organizational decisions. This dissertation clarifies that organizational choices of action or inaction depend on the opinions of members who are detached from the discussions of potential problems. The detached members include non-experts, members with higher power and status, and those with more or fewer peers, depending on organizational conditions. In addition, as organizational conditions become less favorable to speaking up and taking action, the opinions of the detached are more likely to prevent members from reaching a consensus on what to do. In a gray zone, in which a clear threat to organizational performance has not emerged yet, the opinions of the detached members tend to be left unheard or forgotten. When these forgotten opinions favor inaction against potential problems, it is more likely that organizations do nothing and the inaction eventually leads to rare but salient events.Doctor of Philosoph