Improved Software Testing for Open Architecture

Proceedings Paper (for Acquisition Research Program)Applying traditional manual US Navy testing practices to OA systems will limit many benefits of OA, such as system scalability, rapid configuration changes, and effective component reuse. Pairing profile-driven automated software testing with test reduction techniques should enable these benefits and keep resource requirements at feasible levels. Test cases generated by operational profiles have been shown to be more effective than those developed by other methods, such as random or selective testing, and more resource-efficient than exhaustive approaches. This research effort increases the fidelity of the operational profile, creating an environment model referred to as a High-Fidelity Profile Model (HFPM) that can statistically describe individual software inputs. Samples from the HFPM''s probability distributions can generate operationally realistic or overly-stressful test cases for software modules under test. This process can be automated and paired with output checking functions, enabling automated effective software testing, and potentially improving reliability. Such models would be ideal for US Navy Open Architecture (OA) software because of the defined interface standards. HFPMs can enable effective testing in software reuse applications and are ideal for testing multiple releases of maturing software. This research defines the HFPM, presents a methodology to develop, validate, and apply it.Naval Postgraduate School Acquisition Research ProgramApproved for public release; distribution is unlimited

SmartUnit: Empirical Evaluations for Automated Unit Testing of Embedded Software in Industry

In this paper, we aim at the automated unit coverage-based testing for embedded software. To achieve the goal, by analyzing the industrial requirements and our previous work on automated unit testing tool CAUT, we rebuild a new tool, SmartUnit, to solve the engineering requirements that take place in our partner companies. SmartUnit is a dynamic symbolic execution implementation, which supports statement, branch, boundary value and MC/DC coverage. SmartUnit has been used to test more than one million lines of code in real projects. For confidentiality motives, we select three in-house real projects for the empirical evaluations. We also carry out our evaluations on two open source database projects, SQLite and PostgreSQL, to test the scalability of our tool since the scale of the embedded software project is mostly not large, 5K-50K lines of code on average. From our experimental results, in general, more than 90% of functions in commercial embedded software achieve 100% statement, branch, MC/DC coverage, more than 80% of functions in SQLite achieve 100% MC/DC coverage, and more than 60% of functions in PostgreSQL achieve 100% MC/DC coverage. Moreover, SmartUnit is able to find the runtime exceptions at the unit testing level. We also have reported exceptions like array index out of bounds and divided-by-zero in SQLite. Furthermore, we analyze the reasons of low coverage in automated unit testing in our setting and give a survey on the situation of manual unit testing with respect to automated unit testing in industry.Comment: In Proceedings of 40th International Conference on Software Engineering: Software Engineering in Practice Track, Gothenburg, Sweden, May 27-June 3, 2018 (ICSE-SEIP '18), 10 page

Paving the Roadway for Safety of Automated Vehicles: An Empirical Study on Testing Challenges

The technology in the area of automated vehicles is gaining speed and promises many advantages. However, with the recent introduction of conditionally automated driving, we have also seen accidents. Test protocols for both, conditionally automated (e.g., on highways) and automated vehicles do not exist yet and leave researchers and practitioners with different challenges. For instance, current test procedures do not suffice for fully automated vehicles, which are supposed to be completely in charge for the driving task and have no driver as a back up. This paper presents current challenges of testing the functionality and safety of automated vehicles derived from conducting focus groups and interviews with 26 participants from five countries having a background related to testing automotive safety-related topics.We provide an overview of the state-of-practice of testing active safety features as well as challenges that needs to be addressed in the future to ensure safety for automated vehicles. The major challenges identified through the interviews and focus groups, enriched by literature on this topic are related to 1) virtual testing and simulation, 2) safety, reliability, and quality, 3) sensors and sensor models, 4) required scenario complexity and amount of test cases, and 5) handover of responsibility between the driver and the vehicle.Comment: 8 page

Ethical and Social Aspects of Self-Driving Cars

As an envisaged future of transportation, self-driving cars are being discussed from various perspectives, including social, economical, engineering, computer science, design, and ethics. On the one hand, self-driving cars present new engineering problems that are being gradually successfully solved. On the other hand, social and ethical problems are typically being presented in the form of an idealized unsolvable decision-making problem, the so-called trolley problem, which is grossly misleading. We argue that an applied engineering ethical approach for the development of new technology is what is needed; the approach should be applied, meaning that it should focus on the analysis of complex real-world engineering problems. Software plays a crucial role for the control of self-driving cars; therefore, software engineering solutions should seriously handle ethical and social considerations. In this paper we take a closer look at the regulative instruments, standards, design, and implementations of components, systems, and services and we present practical social and ethical challenges that have to be met, as well as novel expectations for software engineering.Comment: 11 pages, 3 figures, 2 table

How Well Do Ontario Library Web Sites Meet New Accessibility Requirements?

New changes to Ontario law will require library web sites to comply with the Web Content Accessibility Guidelines, version 2.0 (WCAG 2.0). This study evaluates 64 Ontario university, college, and public library web sites to see how well they comply with WCAG 2.0 guidelines at present. An average of 14.75 accessibility problems were found per web page. The most common problems included invalid html, poor color contrast, incorrect form controls and labels, missing alt text, bad link text, improper use of headings, using html to format pages, using absolute units of measure, and issues with tables and embedded objects