Leveraging Explainable Artificial Intelligence (XAI) to Understand Performance Deviations in Load Tests of Large Software Systems

Performance testing generates vast amounts of data, making it challenging for human analysts to process within a reasonable timeframe. Therefore, black-box machine learning models are often used to determine pass/fail status, but these models lack transparency and cannot explain why a test has failed, leading to a time-consuming manual analysis process. To address this issue, this thesis proposes using Explainable Artificial Intelligence (XAI) to improve the trustworthiness of black-box and interpretable models in performance testing. The proposed approach leverages the Shapley Additive Explanation (SHAP) algorithm as a surrogate model to help performance analysts understand the decision-making process of black-box machine learning models. By wrapping SHAP around black-box models, analysts can gain explainability on why a model predicted a test\u27s pass or fail status and identify the relative importance of performance data to machine learning models. The proposed approach was evaluated through several load text experiments on a real testbed, using industry-standard performance benchmarks, manually injecting performance bugs into the system to synthesize ground truth and building machine learning models using a black box learner (Artificial Neural Network) and an interpretable learner (Random Forest) to predict the test\u27s pass or fail status. The results demonstrate that classical performance measures such as precision, recall, F-measure, and accuracy are not sufficient to gauge the reliability and trustworthiness of machine learning models. Instead, the proposed approach stands out by providing the explanations behind the decisions made by learning algorithms and enhancing their trustworthiness. The proposed approach, though evaluated through load testing can be generalized to other domains and require little or no effort to operate

Understanding Persistent-Memory Related Issues in the Linux Kernel

Persistent memory (PM) technologies have inspired a wide range of PM-based system optimizations. However, building correct PM-based systems is difficult due to the unique characteristics of PM hardware. To better understand the challenges as well as the opportunities to address them, this paper presents a comprehensive study of PM-related issues in the Linux kernel. By analyzing 1,553 PM-related kernel patches in-depth and conducting experiments on reproducibility and tool extension, we derive multiple insights in terms of PM patch categories, PM bug patterns, consequences, fix strategies, triggering conditions, and remedy solutions. We hope our results could contribute to the development of robust PM-based storage systemsComment: ACM TRANSACTIONS ON STORAGE(TOS'23

Intelligent tutoring system for real estate management

The review on the worldwide intelligent tutoring systems and their application possibilities is presented in the paper. The intelligent tutoring system for real estate management developed by the authors is described. This system is applied in Vilnius Gediminas Technical University, Department of Construction Economics and Property Management. Besides the common components ‐ student model, domain model, pedagogical model and graphical interface, the new developed system has testing model, decision support subsystem and database of computer learning systems. Domain model includes knowledge with the supplemental audio and video material for 63 modules being taught in Vilnius Gediminas Technical University. Student model enables to adapt to a learner needs and knowledge level. Decision support subsystem is used for all components of intelligent tutoring system giving them different level of intelligence. Database of computer learning systems enables using the following web‐based learning systems: construction, real estate, facilities management, international trade, ethics, innovation, sustainable development, building refurbishment, etc. Tutor and testing model provide a model of the teaching process and support transition to a new knowledge state. Graphic interface is used to create an effective system‐user dialogue. Santruka Straipsnyje pateikiama išsami intelektinių mokymo sistemų bei jų taikymo galimybių analizė. Aprašoma nekilnojamojo turto vadybos intelektinė mokymo sistema, sukurta autorių. Ji taikoma Vilniaus Gedimino technikos universiteto Statybos ekonomikos ir nekilnojamojo turto vadybos katedroje. Be bendrų intelektinėms mokymo sistemoms komponentų ‐ studento modelio, pedagoginio modelio, disciplinų duomenų bazės ir grafinės sąsajos, į naują sistemą įtrauktas sprendimų paramos posistemis, kompiuterinių mokymo sistemų duomenų bazės ir žinių vertinimo posistemis. Disciplinų duomenų bazėje pateikiamos 63 modulių, dėstomų Vilniaus Gedimino technikos universitete, žinios spausdinta, vaizdo bei garso forma. Studento modelis sudaro galimybę pritaikyti mokymą prie studijuojančiojo poreikių ir žinių lygio. Sprendimų paramos posistemis taikomas visuose intelektinės mokymo sistemos komponentuose, suteikia jiems skirtingo lygmens intelektualumo savybių. Kompiuterinių mokymo sistemų duomenų bazė leidžia naudotis šiomis internetinėmis mokymo sistemomis: statybos, nekilnojamojo turto, pastatų ūkio valdymo, tarptautinės prekybos, etikos, inovacijų, subalansuotos plėtros, renovacijos ir kt. Pedagoginis ir žinių vertinimo modelis pateikia kitą ‐ mokymo proceso modelį, padeda pereiti į kitą žinių lygmenį. Grafinė sąsaja sukuria efektyvų dialogą tarp sistemos ir vartotojo

Texting and tapping : a dynamical approach to multitasking.

Jobs in various work fields (e.g., flying airplanes; Helmreich, 2000) require a high ability to successfully handle more than one task at a time, or to multitask. Researchers usually explain multitasking by having priorities in which individuals either attend to one task at a time, or one task receives more time processing than the other task. The current study approaches multitasking from a dynamical systems perspective. Fourteen general psychology students participated in the study by pressing a pedal attempting to maintain a steady beat and text messaging. Researchers recorded behavior over time (2 min. for each task and multitasking). The inputs to the data analysis were the X-Y coordinates of thumb movement (in pixels) over time and the recorded beat's deviation (in sec) from the metronome's beat over time. The patterns of behavior were recorded. Nonlinear analyses (Iterated Function Systems and a MANOVA on Hurst exponents for monofractality, and Wavelet Modulus Transform Maxima for multifractality) tested for fractal patterns which characterized both tasks in both conditions (single task or multitasking). Thumb movement's patterns during texting were not significantly different for single task and multitasking conditions, both displaying short-term correlations (brown noise). Patterns in tapping deviations were significantly different between the two conditions. Structure of deviations while only tapping was characterized by strong long-term correlations (pink noise); the structure while multitasking was also positively long-term correlated, but less strong. Results showed that texting and tapping behavior, as single tasks or during multitasking, are fractal

Report to the administrator by the NASA Aerospace Safety Advisory Panel on the Skylab program. Volume 2: Program implementation and maturity

Results of the design and manufacturing reviews on the maturity of the Skylab modules are presented along with results of investigations on the scope of the cluster risk assessment efforts. The technical management system and its capability to assess and resolve problems are studied

Advanced extravehicular activity systems requirements definition study. Phase 2: Extravehicular activity at a lunar base

The focus is on Extravehicular Activity (EVA) systems requirements definition for an advanced space mission: remote-from-main base EVA on the Moon. The lunar environment, biomedical considerations, appropriate hardware design criteria, hardware and interface requirements, and key technical issues for advanced lunar EVA were examined. Six remote EVA scenarios (three nominal operations and three contingency situations) were developed in considerable detail

Human factors aspects of air traffic control

An overview of human factors problems associated with the operation of present and future air traffic control systems is presented. A description is included of those activities and tasks performed by air traffic controllers at each operational position within the present system. Judgemental data obtained from controllers concerning psychological dimensions related to these tasks and activities are also presented. The analysis includes consideration of psychophysiological dimensions of human performance. The role of the human controller in present air traffic control systems and his predicted role in future systems is described, particularly as that role changes as the result of the system's evolution towards a more automated configuration. Special attention is directed towards problems of staffing, training, and system operation. A series of ten specific research and development projects are recommended and suggested work plans for their implementation are included

Implementation of a production Ada project: The GRODY study

The use of the Ada language and design methodologies that encourage full use of its capabilities have a strong impact on all phases of the software development project life cycle. At the National Aeronautics and Space Administration/Goddard Space Flight Center (NASA/GSFC), the Software Engineering Laboratory (SEL) conducted an experiment in parallel development of two flight dynamics systems in FORTRAN and Ada. The differences observed during the implementation, unit testing, and integration phases of the two projects are described and the lessons learned during the implementation phase of the Ada development are outlined. Included are recommendations for future Ada development projects