Assimilating Non-Probabilistic Assessments of the Estimation of Uncertainty Bias in Expert Judgment Elicitation Using an Evidence Based Approach in High Consequence Conceptual Designs

One of the major challenges in conceptual designs of complex systems is the identification of uncertainty embedded in the information due to lack of historic data. This becomes of increased concern especially in high-risk industries. This document reports a developed methodology that allows for the cognitive bias, estimation of uncertainty, to be elucidated to improve the quality of elicited data. It consists of a comprehensive literature review that begins by defining a \u27High Consequence Conceptual Engineering Environment\u27 and identifies the high-risk industries in which these environments are found. It proceeds with a discussion that differentiates risk and uncertainty in decision-making in these environments. An argument was built around the identified epistemic category of uncertainty, the impact on hard data for decision-making, and from whom we obtain this data. The review shifts to defining and selecting the experts, the elicitation process in terms of the components, the process phases and steps involved, and an examination of a probabilistic and a fuzzy example. This sets the stage for this methodology that uses evidence theory for the mathematical analysis after the data is elicited using a tailored elicitation process. Yager\u27s combination rule is used to combine evidence and fully recognize the ignorance without ignoring available information. Engineering and management teams from NASA Langley Research Center were the population from which the experts for this study were identified. NASA officials were interested in obtaining uncertainty estimates, and a comparison of these estimates, associated with their Crew Launch Vehicle (CLV) designs; the existing Exploration Systems Architecture Study Crew Launch Vehicle (ESAS CLV) and the Parallel-Staged Crew Launch Vehicle (P-S CLV) which is currently being worked. This evidence-based approach identified that the estimation of cost parameters uncertainty is not specifically over or underestimated in High Consequence Conceptual Engineering Environments; rather, there is more uncertainty present than what is being anticipated. From the perspective of maturing designs, it was concluded that the range of cost parameters\u27 uncertainty at different error-state-values were interchangeably larger or smaller when compared to each other even as the design matures

Basic research planning in mathematical pattern recognition and image analysis

Fundamental problems encountered while attempting to develop automated techniques for applications of remote sensing are discussed under the following categories: (1) geometric and radiometric preprocessing; (2) spatial, spectral, temporal, syntactic, and ancillary digital image representation; (3) image partitioning, proportion estimation, and error models in object scene interference; (4) parallel processing and image data structures; and (5) continuing studies in polarization; computer architectures and parallel processing; and the applicability of "expert systems" to interactive analysis

Are Delayed Issues Harder to Resolve? Revisiting Cost-to-Fix of Defects throughout the Lifecycle

Many practitioners and academics believe in a delayed issue effect (DIE); i.e. the longer an issue lingers in the system, the more effort it requires to resolve. This belief is often used to justify major investments in new development processes that promise to retire more issues sooner. This paper tests for the delayed issue effect in 171 software projects conducted around the world in the period from 2006--2014. To the best of our knowledge, this is the largest study yet published on this effect. We found no evidence for the delayed issue effect; i.e. the effort to resolve issues in a later phase was not consistently or substantially greater than when issues were resolved soon after their introduction. This paper documents the above study and explores reasons for this mismatch between this common rule of thumb and empirical data. In summary, DIE is not some constant across all projects. Rather, DIE might be an historical relic that occurs intermittently only in certain kinds of projects. This is a significant result since it predicts that new development processes that promise to faster retire more issues will not have a guaranteed return on investment (depending on the context where applied), and that a long-held truth in software engineering should not be considered a global truism.Comment: 31 pages. Accepted with minor revisions to Journal of Empirical Software Engineering. Keywords: software economics, phase delay, cost to fi

Aeronautical Engineering: A special bibliography with indexes, supplement 51

This bibliography lists 206 reports, articles, and other documents introduced into the NASA Scientific and Technical Information System in November 1974

Reliable ABC model choice via random forests

Approximate Bayesian computation (ABC) methods provide an elaborate approach to Bayesian inference on complex models, including model choice. Both theoretical arguments and simulation experiments indicate, however, that model posterior probabilities may be poorly evaluated by standard ABC techniques. We propose a novel approach based on a machine learning tool named random forests to conduct selection among the highly complex models covered by ABC algorithms. We thus modify the way Bayesian model selection is both understood and operated, in that we rephrase the inferential goal as a classification problem, first predicting the model that best fits the data with random forests and postponing the approximation of the posterior probability of the predicted MAP for a second stage also relying on random forests. Compared with earlier implementations of ABC model choice, the ABC random forest approach offers several potential improvements: (i) it often has a larger discriminative power among the competing models, (ii) it is more robust against the number and choice of statistics summarizing the data, (iii) the computing effort is drastically reduced (with a gain in computation efficiency of at least fifty), and (iv) it includes an approximation of the posterior probability of the selected model. The call to random forests will undoubtedly extend the range of size of datasets and complexity of models that ABC can handle. We illustrate the power of this novel methodology by analyzing controlled experiments as well as genuine population genetics datasets. The proposed methodologies are implemented in the R package abcrf available on the CRAN.Comment: 39 pages, 15 figures, 6 table

A Flexible Zero-Inflated Poisson Regression Model

A practical problem often encountered with observed count data is the presence of excess zeros. Zero-inflation in count data can easily be handled by zero-inflated models, which is a two-component mixture of a point mass at zero and a discrete distribution for the count data. In the presence of predictors, zero-inflated Poisson (ZIP) regression models are, perhaps, the most commonly used. However, the fully parametric ZIP regression model could sometimes be restrictive, especially with respect to the mixing proportions. Taking inspiration from some of the recent literature on semiparametric mixtures of regressions models for flexible mixture modeling, we propose a semiparametric ZIP regression model. We present an EM-like algorithm for estimation and a summary of asymptotic properties of the estimators. The proposed semiparametric models are then applied to a data set involving clandestine methamphetamine laboratories and Alzheimer\u27s disease

Methodologies and advancements in the calibration of building energy models

Buildings do not usually perform during operation as well as predicted during the design stage. Disagreement between simulated and metered energy consumption represents a common issue in building simulation. For this reason, the calibration of building simulation models is of growing interest. Sensitivity and uncertainty analyses play an important role in building model accuracy. They can be used to identify the building model parameters most influent on the energy consumption. Given this, these analyses should be integrated within calibration methodologies and applications for tuning the parameters. This paper aims at providing a picture of the state of the art of calibration methodologies in the domain of building energy performance assessment. First, the most common methodologies for calibration are presented, emphasizing criticalities and gaps that can be faced. In particular the main issues to be addressed, when carrying out calibrated simulation, are discussed. The standard statistical criteria for considering the building models calibrated and for evaluating their goodness-of-fit are also presented. Second, the commonly used techniques for investigating uncertainties in building models are reviewed. Third, a review of the latest main studies in the calibrated simulation domain is presented. Criticalities and recommendations for new studies are finally provided

Data-Driven Framework for Understanding & Modeling Ride-Sourcing Transportation Systems

Ride-sourcing transportation services offered by transportation network companies (TNCs) like Uber and Lyft are disrupting the transportation landscape. The growing demand on these services, along with their potential short and long-term impacts on the environment, society, and infrastructure emphasize the need to further understand the ride-sourcing system. There were no sufficient data to fully understand the system and integrate it within regional multimodal transportation frameworks. This can be attributed to commercial and competition reasons, given the technology-enabled and innovative nature of the system. Recently, in 2019, the City of Chicago the released an extensive and complete ride-sourcing trip-level data for all trips made within the city since November 1, 2018. The data comprises the trip ends (pick-up and drop-off locations), trip timestamps, trip length and duration, fare including tipping amounts, and whether the trip was authorized to be shared (pooled) with another passenger or not. Therefore, the main goal of this dissertation is to develop a comprehensive data-driven framework to understand and model the system using this data from Chicago, in a reproducible and transferable fashion. Using data fusion approach, sociodemographic, economic, parking supply, transit availability and accessibility, built environment and crime data are collected from open sources to develop this framework. The framework is predicated on three pillars of analytics: (1) explorative and descriptive analytics, (2) diagnostic analytics, and (3) predictive analytics. The dissertation research framework also provides a guide on the key spatial and behavioral explanatory variables shaping the utility of the mode, driving the demand, and governing the interdependencies between the demand’s willingness to share and surge price. Thus, the key findings can be readily challenged, verified, and utilized in different geographies. In the explorative and descriptive analytics, the ride-sourcing system’s spatial and temporal dimensions of the system are analyzed to achieve two objectives: (1) explore, reveal, and assess the significance of spatial effects, i.e., spatial dependence and heterogeneity, in the system behavior, and (2) develop a behavioral market segmentation and trend mining of the willingness to share. This is linked to the diagnostic analytics layer, as the revealed spatial effects motivates the adoption of spatial econometric models to analytically identify the ride-sourcing system determinants. Multiple linear regression (MLR) is used as a benchmark model against spatial error model (SEM), spatially lagged X (SLX) model, and geographically weighted regression (GWR) model. Two innovative modeling constructs are introduced deal with the ride-sourcing system’s spatial effects and multicollinearity: (1) Calibrated Spatially Lagged X Ridge Model (CSLXR) and Calibrated Geographically Weighted Ridge Regression (CGWRR) in the diagnostic analytics layer. The identified determinants in the diagnostic analytics layer are then fed into the predictive analytics one to develop an interpretable machine learning (ML) modeling framework. The system’s annual average weekday origin-destination (AAWD OD) flow is modeled using the following state-of-the-art ML models: (1) Multilayer Perceptron (MLP) Regression, (2) Support Vector Machines Regression (SVR), and (3) Tree-based ensemble learning methods, i.e., Random Forest Regression (RFR) and Extreme Gradient Boosting (XGBoost). The innovative modeling construct of CGWRR developed in the diagnostic analytics is then validated in a predictive context and is found to outperform the state-of-the-art ML models in terms of testing score of 0.914, in comparison to 0.906 for XGBoost, 0.84 for RFR, 0.89 for SVR, and 0.86 for MLP. The CGWRR exhibits outperformance as well in terms of the root mean squared error (RMSE) and mean average error (MAE). The findings of this dissertation partially bridge the gap between the practice and the research on ride-sourcing transportation systems understanding and integration. The empirical findings made in the descriptive and explorative analytics can be further utilized by regional agencies to fill practice and policymaking gaps on regulating ride-sourcing services using corridor or cordon toll, optimally allocating standing areas to minimize deadheading, especially during off-peak periods, and promoting the ride-share willingness in disadvantage communities. The CGWRR provides a reliable modeling and simulation tool to researchers and practitioners to integrate the ride-sourcing system in multimodal transportation modeling frameworks, simulation testbed for testing long-range impacts of policies on ride-sourcing, like improved transit supply, congestions pricing, or increased parking rates, and to plan ahead for similar futuristic transportation modes, like the shared autonomous vehicles