Thesis for the Degree of Bachelor of Science in Computer Science by Peter Charbachi and Linus Eklund : PAIRWISE TESTING FOR PLC EMBEDDED SOFTWARE

In this thesis we investigate the use of pairwise testing for PLC embedded software. We compare these automatically generated tests with tests created manually by industrial engineers. The tests were evaluated in terms of fault detection, code coverage and cost. In addition, we compared pairwise testing with randomly generated tests of the same size as pairwise tests. In order to automatically create test suites for PLC software a previously created tool called Combinatorial Test Tool (CTT) was extended to support pairwise testing using the IPOG algorithm. Once test suites were created using CTT they were executed on real industrial programs. The fault detection was measured using mutation analysis. The results of this thesis showed that manual tests achieved better fault detection (8% better mutation score in average) than tests generated using pairwise testing. Even if pairwise testing performed worse in terms of fault detection than manual testing, it achieved better fault detection in average than random tests of the same size. In addition, manual tests achieved in average 97.29% code coverage compared to 93.95% for pairwise testing, and 84.79% for random testing. By looking closely on all tests, manual testing performed equally good as pairwise in terms of achieved code coverage. Finally, the number of tests for manual testing was lower (12.98 tests in average) compared to pairwise and random testing (21.20 test in average). Interestingly enough, for the majority of the programs pairwise testing resulted in fewer tests than manual testing

Methods for Automatic Hydraulics Calibration in Construction Equipment

In this thesis we investigate the problem of automatic calibration and control of hydraulic components in the domain of construction equipment. Methods that are able to remove a costly manual approach in favour of an automatic one are investigated and evaluated. The thesis aims to investigate what methods are available in achieving this goal as well as evaluate the performance and applicability of such methods in the domain of construction equipment. The literature indicates that a great focus is put on learning a model of the plant at run time in order to provide accurate control. Common approaches to the problem are the Recursive Least Square method and PID controllers for non-linear systems, but other methods are also present, such as the Nodal Link Perceptron Network (NLPN). The methods chosen to be compared are the existing method of manually calibrating two set points for start and end current and interpolating between them; the use of a PI controller with a static line inverse model; a PI controller with a static curve inverse model; a PI controller with an NLPN adaptive inverse model and lastly, a completely NLPN based control strategy. The methods were implemented in Matlab Simulink and evaluated in simulations based on data collected from real wheel loaders in the construction equipment domain, produced by Volvo CE. The simulations are performed on data from three machines and were evaluated twice for the adaptive methods in order to evaluate how well the methods improved. The results were then evaluated in terms of average absolute error, as well as a discussion of the behaviour shown in the plots. The evaluations indicate that the most effective method for control is the PI controller using a static line inverse model. The method produces the smallest average error of both actions evaluated, lifting and lowering of the boom, while the complete NLPN solution provide the worst results

Can Pairwise Testing Perform Comparably to Manually Handcrafted Testing Carried Out by Industrial Engineers?

Testing is an important activity in engineering of industrial software. For such software, testing is usually performed manually by handcrafting test suites based on specific design techniques and domain-specific experience. To support developers in testing, different approaches for producing good test suites have been proposed. In the last couple of years combinatorial testing has been explored with the goal of automatically combining the input values of the software based on a certain strategy. Pairwise testing is a combinatorial technique used to generate test suites by varying the values of each pair of input parameters to a system until all possible combinations of those parameters are created. There is some evidence suggesting that these kinds of techniques are efficient and relatively good at detecting software faults. Unfortunately, there is little experimental evidence on the comparison of these combinatorial testing techniques with, what is perceived as, rigorous manually handcrafted testing. In this study we compare pairwise test suites with test suites created manually by engineers for 45 industrial programs. The test suites were evaluated in terms of fault detection, code coverage and number of tests. The results of this study show that pairwise testing, while useful for achieving high code coverage and fault detection for the majority of the programs, is almost as effective in terms of fault detection as manual testing. The results also suggest that pairwise testing is just as good as manual testing at fault detection for 64% of the programs