Uncertainty explicit assessment of off-the-shelf software: A Bayesian approach

Assessment of software COTS components is an essential part of component-based software development. Poorly chosen components may lead to solutions of low quality and that are difficult to maintain. The assessment may be based on incomplete knowledge about the COTS component itself and other aspects (e.g. vendor’s credentials, etc.), which may affect the decision of selecting COTS component(s). We argue in favor of assessment methods in which uncertainty is explicitly represented (‘uncertainty explicit’ methods) using probability distributions. We provide details of a Bayesian model, which can be used to capture the uncertainties in the simultaneous assessment of two attributes, thus, also capturing the dependencies that might exist between them. We also provide empirical data from the use of this method for the assessment of off-the-shelf database servers which illustrate the advantages of ‘uncertainty explicit’ methods over conventional methods of COTS component assessment which assume that at the end of the assessment the values of the attributes become known with certainty

Uncertainty explicit assessment of off-the-shelf software: Selection of an optimal diverse pair

Assessment of software COTS components is an essential part of component-based software development. Sub-optimal selection of components may lead to solutions with low quality. The assessment is based on incomplete knowledge about the COTS components themselves and other aspects, which may affect the choice such as the vendor's credentials, etc. We argue in favor of assessment methods in which uncertainty is explicitly represented (`uncertainty explicit' methods) using probability distributions. We have adapted a model (developed elsewhere by Littlewood, B. et al. (2000)) for assessment of a pair of COTS components to take account of the fault (bug) logs that might be available for the COTS components being assessed. We also provide empirical data from a study we have conducted with off-the-shelf database servers, which illustrate the use of the method

Solving Macroeconomic Models with "Off-the-Shelf" Software: An Example of Potential Pitfalls

When working with large-scale models or numerous small models, there can be a temptation to rely on default settings in proprietary software to derive solutions to the model. In this paper we show that, for the solution of non-linear dynamic models, this approach can be inappropriate. Alternative linear and non-linear specifications of a particular model are examined. One version of the model, expressed in levels, is highly non-linear. A second version of the model, expressed in logarithms, is linear. The dynamic solution of each model version has a combination of stable and unstable eigenvalues so that any dynamic solution requires the calculation of appropriate “jumps” in endogenous variables. We can derive a closed-form solution of the model, which we use as our "true" benchmark, for comparison with computational solutions of both linear and non-linear models. Our approach is to compare the "goodness of fit" of reverse-shooting solutions for both the linear and non-linear model, by comparing the computational solutions with the benchmark solution. Under the basic solution method with default settings, we show that there is significant difference between the computational solution for the non-linear model and the benchmark closed-form solution. We show that this result can be substantially improved using modifications to the solver and to parameter settings.Solving non-linear models, reverse-shooting, computational economics, computer software

Reliability Assessment of Legacy Safety-Critical Systems Upgraded with Fault-Tolerant Off-the-Shelf Software

This paper presents a new way of applying Bayesian assessment to systems, which consist of many components. Full Bayesian inference with such systems is problematic, because it is computationally hard and, far more seriously, one needs to specify a multivariate prior distribution with many counterintuitive dependencies between the probabilities of component failures. The approach taken here is one of decomposition. The system is decomposed into partial views of the systems or part thereof with different degrees of detail and then a mechanism of propagating the knowledge obtained with the more refined views back to the coarser views is applied (recalibration of coarse models). The paper describes the recalibration technique and then evaluates the accuracy of recalibrated models numerically on contrived examples using two techniques: u-plot and prequential likelihood, developed by others for software reliability growth models. The results indicate that the recalibrated predictions are often more accurate than the predictions obtained with the less detailed models, although this is not guaranteed. The techniques used to assess the accuracy of the predictions are accurate enough for one to be able to choose the model giving the most accurate prediction

Target-adaptive CNN-based pansharpening

We recently proposed a convolutional neural network (CNN) for remote sensing image pansharpening obtaining a significant performance gain over the state of the art. In this paper, we explore a number of architectural and training variations to this baseline, achieving further performance gains with a lightweight network which trains very fast. Leveraging on this latter property, we propose a target-adaptive usage modality which ensures a very good performance also in the presence of a mismatch w.r.t. the training set, and even across different sensors. The proposed method, published online as an off-the-shelf software tool, allows users to perform fast and high-quality CNN-based pansharpening of their own target images on general-purpose hardware

PREDICTIVE MAINTENANCE USING MACHINE LEARNING AND EXISTING DATA SOURCES

Includes supplementary materialThe United States Marine Corps must address material-readiness challenges with emerging technologies at minimum cost. Predictive maintenance using machine learning is a growing field that can be applied using free or commercial-off-the-shelf software. Naval aviation organizations already maintain a network of data repositories that collect and store current and historical data on repairable flight-critical components. Many components fail before their expected structural life as published their manufacturers, which results in costly unscheduled maintenance. The ability to predict component failures and plan for their replacement or repair can significantly increase operational readiness. This thesis develops and analyzes machine-learning models to predict the conditional probability of failure of various MV-22B flight-critical components using data from existing Naval aviation repositories. Data preprocessing, model training, and predictions use commercial-off-the-shelf software. This work can help improve material readiness and acclimatize military-aviation personnel to emerging technologies in decision making.Captain, United States Marine CorpsApproved for public release. Distribution is unlimited

Requirements Management in Off-The-Shelf Software Implementation Projects: A Case Study in Playtech

Nõuete haldust peetakse kvaliteetsete tarkvaralahenduste pakkumisel üheks põhipädevuseks. Samas on see ka üks peamisi tarkvaraprojektide ebaõnnestumise põhjuseid. Mõlemad väited kehtivad nii rätsepalahenduste kui valmistarkvara-põhiste lahenduste korral. Tarkvara pakkujate jaoks on väljakutse tagada edukas valmistarkvara rakendusprojektide elluviimine nii, et kõik kliendi vajadused ja nõuded saaksid rahuldatud ja et aja- ja ressursikulu oleks sealjuures võimalikult väike. Tänapäeval ei leidu nõuete haldusele ühtset lähenemist, mis oleks kohandatud valmistarkvara-põhistele rakendusprojektidele tarkvara pakkuja vaatest ning ühtlasi arvestaks interneti hasartmängutööstuse eripäradega. Magistritöö eesmärgiks on täita see tühimik, tuginedes juhtumiuuringule internetipõhist hasartmängutarkvara tootvas ettevõttes Playtech, ning leida vastus küsimusele, kuidas saaks interneti hasartmänguvaldkonnas valmistarkvara-põhistes rakendusprojektides nõudeid hallata. Tuginedes analüüsile ja teaduspõhistele parimatele praktikatele nõuete halduse valdkonnas, pakutakse magistritöös välja protsess nõuete efektiivseks haldamiseks hasartmängu-valmistarkvara rakendusprojektides. Nimetatud protsess hõlmab tegevusi, mis praeguses praktikas puudu või olemas vaid osaliselt, nagu näiteks vajaduste hindamine, nõuete halduse planeerimine, nõuete seire ja kontrollimine, süsteemne vigadest õppimine ja projekti tugivastutuse üleandmine. Nõuete halduse protsessi eduka juurutamise eelduseks on joondumine kõikide sidusgruppide ning juhtkonnaga.Requirements management is considered a core competency for delivering quality software solutions. It is also counted among the main causes for project failure. This is true in the context of greenfield development as well as off-the-shelf (OTS) based software solutions. The challenge in OTS software implementation projects today from the software provider’s perspective is ensuring successful completion of the solution setup that meets customer needs and satisfies requirements without compromising on delivery time and cost. Today, there is no requirements management approach that would consider the specifics of OTS based software implementation projects from the supplier’s perspective and the particularities of the online gambling industry. This thesis addresses the lack of systematic approach to requirements management in OTS based online gambling software context in case company Playtech and attempts to answer the research question of how requirements can be managed when implementing OTS based online gambling solutions. Based on analysis and best practices from background research, a process is suggested for efficient requirements management in OTS gambling software implementation projects. The proposed process incorporates activities that are not present or are present only partially in the current practices, such as needs assessment, requirements management planning, requirements monitoring and controlling, reporting lessons learned and support transition. Alignment between all stakeholders as well as management is required to enable successful establishment of the requirements management process

Automated Real-Time Testing (ARTT) for Embedded Control Systems (ECS)

Developing real-time automated test systems for embedded control systems has been a real problem. Some engineers and scientists have used customized software and hardware as a solution, which can be very expensive and time consuming to develop. We have discovered how to integrate a suite of commercially available off-the-shelf software tools and hardware to develop a scalable test platform that is capable of performing complete black-box testing for a dual-channel real-time Embedded-PLC-based control system (www.aps.anl.gov). We will discuss how the Vali/Test Pro testing methodology was implemented to structure testing for a personnel safety system with large quantities of requirements and test cases. This work was supported by the U.S. Department of Energy, Basic Energy Sciences, under Contract No. W-31-109-Eng-38.Comment: 6 pages, 8 figures, ICALEPCS 2001, Poster Sessio