A logic coverage measure related to MC/DC testing is used to estimate residual faults. The residual fault prediction method is evaluated on an industrial PLC logic example. A randomized form of MC/DC testing is used to maximize coverage growth and fault detection efficiency

Bishop, P. G.

English

City Research Online

Bishop, P. G. (2003). MC/DC based estimation and detection of residual faults in PLC logic networks. Paper presented at the 14th IEEE International Symposium on Software Reliability Engineering ISSRE 2003, 17 - 20 Nov 2003, Denver, Colorado.City Research OnlineOriginal citation: Bishop, P. G. (2003). MC/DC based estimation and detection of residual faults in PLC logic networks. Paper presented at the 14th IEEE International Symposium on Software Reliability Engineering ISSRE 2003, 17 - 20 Nov 2003, Denver, Colorado.Permanent City Research Online URL: http://openaccess.city.ac.uk/543/ Copyright & reuseCity  University  London has developed City  Research Online  so that  its  users  may access the research outputs of City University London's staff. Copyright © and Moral Rights for this paper are retained by the individual author(s) and/ or other copyright holders. Users may download and/ or print one  copy  of  any  article(s)  in  City  Research  Online  to  facilitate  their  private  study  or  for  non-commercial research. Users may not engage in further distribution of the material or use it for any profit-making activities or any commercial gain. All material in City Research Online is checked for eligibility for copyright before being made available in the live archive. URLs from City Research Online may be freely distributed and linked to from other web pages. Versions of researchThe version in City Research Online may differ from the final published version. Users are advised to check the Permanent City Research Online URL above for the status of the paper.EnquiriesIf you have any enquiries about any aspect of City Research Online, or if you wish to make contact  with the author(s) of this paper, please email the team at publications@city.ac.uk.Copyright 2003 Chillarege Press Fast Abstract ISSRE 2003MC/DC based estimation and detection of residual faults in PLClogic networksPeter G BishopCentre for Software Reliability, London, UKpgb@csr.city.ac.ukAbstract—A logic coverage measure related to MC/DC testingis used to estimate residual faults. The residual fault predictionmethod is evaluated on an industrial PLC logic example. Arandomized form of MC/DC testing is used to maximize coveragegrowth and fault detection efficiency.I.  INTRODUCTIONCoverage measurement has been used to estimate residualfaults in program code [1,2]. The basic idea is that therelationship between code covered and faults found is nearlylinear, so it is possible to estimate the number of residual faultsfrom the proportion of uncovered code. In this paper we applythe same concept to PLC logic networks rather thanconventional program code—combined with a random teststrategy designed to maximize coverage growth. We based ourstudy on a PLC logic network taken from an industrialexample that had 36 binary inputs and 10 binary outputs.There were also 6 known faults in the initial logicimplementation.II.  LOGIC COVERAGE MEASURESClearly to apply the approach to a logic network we neededan equivalent measure of logic coverage. We considered inputvalue coverage, where all possible combinations of inputvalues are covered, and output value coverage where allpossible combinations of output values are covered. Howeverthe main measure chosen for this study was input-output paircoverage where input values are selected such that change of agiven binary input i can “toggle” the state of a binary output j.There is a strong relationship between input-output paircoverage and the Modified Condition/Decision Coverage(MC/DC) test method used in conventional programs [3].MC/DC tests are designed so that a change in every Booleancondition “toggles” the Boolean value of the overall IFcondition.With 36 inputs and 10 outputs and a maximum of 4 input-output combinations, the maximum number of combinations is1440. In practice the number of actual combinations couldmuch less due to constraints between inputs and outputsimposed by the logic network. In this particular example, theconstraints reduced the number of I-O pairs to 236.III.  MC/DC RANDOM TEST STRATEGYConventional random testing uses a uniform random inputdistribution with a probability pi = 0.5 that an input is setTRUE for all inputs i. We propose an alternative strategywhere we seek to achieve a uniform output probability—wherethe input probabilities pi are chosen so that pj = 0.5. for alloutputs j.The rationale for uniform output probability testing is that itwill maximize the output coverage growth measure, and alsoincrease I-O pair coverage growth as there is a 50% that eachoutput can be “toggled” by a change in an input.In order to achieve uniform output coverage, it wasnecessary to devise a procedure for back-propagating assignedoutput probabilities to the inputs. The rules for backpropagation through logic are quite simple:AND: pin = pout1/nOR: pin = 1 – (1 – pout )1/nNOT: pin = (1 – pout )where:n is the number of inputs to the logic gate,px is the probability of TRUE value on link x,This is illustrated in the figure below for a simple single-output network, where a pj value of 0.5 is back propagated tothe inputs to derive input probabilities pi.ANDORANDANDABCDE NOTOR 0.50.50.710.290.710.420.240.490.490.490.490.290.24PiPjFigure 1. Back propagation of probability valuesIn practice, back propagation is constrained by networkjunctions and feedback loops. For example in the figure above,the feedback loop forces one input to the OR gate to be 0.5.When one input probability is constrained to p*in, it can beshown that he probability for the remaining inputs pin is:pin = 1 – ((1 – pout )/(1 – p*in)) 1/n-1Hence for p*in=0.5,  pout=0.71 and n=2, we obtain 0.42 for theother input.When back-propagation was applied to the actual networkof 36 inputs and 10 outputs, the interconnection constraintsmeant that the “ideal value” of pj = 0.5 could not be achievedCopyright 2003 Chillarege Press Fast Abstract ISSRE 2003(as negative values for pin are derived during backpropagation). Compromise values for the output probabilities(typically of the order of 0.3) were chosen instead to obtainvalid input probabilities. The distribution of input probabilitiesto achieve near uniform output probabilities is shown in thefigure below. It is clear that the input probabilities can be quiteextreme (close to zero or 1).00.10.20.30.40.50.60.70.80.911 36Input iP(i)Figure 2. Distribution of input probabilitiesWe can view this input distribution as the probabilisticequivalent of an MC/DC test pattern as it maximizes thechance of specifying inputs patterns where an output willchange if a single input bit is changed.IV.  Test EFFECTIVENESSI-O pair coverage was compared against faults found usingthree different test strategies: MC/DC random testing, uniformrandom input testing (pin = 0.5) and a set of 486 systematictests developed for the original industrial logic implementation(see the figure below).01234560 0.2 0.4 0.6 0.8 1I-O Pair coverageFaultsdetectedMCDC RandomUniform RandomSystematic testsFigure 3. Faults detected vs. I-O pair coverageHowever the actual number of tests needed to achieve a givencoverage varied dramatically (around 1000 random input testswere equivalent to one MC/DC test in terms of I-O paircoverage achieved). As MC/DC testing results in fast coveragegrowth and coverage is strongly correlated with faults found,we would expect MC/DC testing to find the faults with lesstests than other methods. Actual fault detection performance isshown in Table 1.TABLE 1FAULT DETECTION PERFORMANCEFaults detectedNumberof tests MCDCtestRandominputSystematictests10 4 0 ?100 5 0 ?486 6 0 41000 6 03000 6 19514 6 410000 6 4100000 6 61000000 6 6As expected, the MC/DC test appears to out-perform theother test strategies, finding the first 4 faults in 10 random testsand all 6 in 486 tests, while the systematic tests had onlydetected 4 faults at this stage, and random input testing hadfound none. 100% I-O pair coverage is achieved with 3000tests, giving high confidence that there are no residual faults.V.  CONCLUSIONS1. I-O pair coverage was strongly correlated with the faultsfound in the logic network.2. MC/DC random testing was more effective than randominput testing and an existing systematic test set—probablybecause coverage growth was faster.ACKNOWLEDGMENTThis work was funded by the UK (Nuclear) IndustrialManagement Committee (IMC) Nuclear Safety ResearchProgramme under British Energy Generation UK contract40029264 with contributions from British Nuclear Fuels plc,British Energy Ltd and British Energy Group UK Ltd. Thepaper reporting the research was produced under the EPSRCresearch interdisciplinary programme on dependability(DIRC).REFERENCES[1] P.G. Bishop, “Estimating Residual Faults from CodeCoverage”, SAFECOMP 02, Catania, Italy, 10-13 Sep2002, pp. 163-174[2] Y.K. Malaiya, J. Denton and M.N. Li., “Estimating thenumber of defects: a simple and intuitive approach”,Proceedings ISSRE 98, Paderborn, Germany, November4-7, 1998, pp. 307-315[3] RTCA DO-178B (and EUROCAE ED-12), “Softwareconsiderations in airborne systems and equipmentcertification”, 1993

MC/DC based estimation and detection of residual faults in PLC logic networks

Peter G. Bishop

CiteSeerX

http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.9.6242

MC/DC based estimation and detection of residual faults in PLC logic networks

Abstract

Similar works

Full text

Available Versions

City Research Online

CiteSeerX