Poster describing use of Artificial Neural  Networks (ANN) in a Receiver-Operating Characteristic (ROC) scenario to predict bathing water quality exceedances at beaches in the SW UK.Environment Agency (SW

Djordjevic, Slobodan

Duncan, Andrew

Keedwell, Edward

Savic, Dragan

Open Research Exeter

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 100.10.20.30.40.50.60.70.80.91False Positive RateTrue Positive RateIlfracombe ROC with Fmeasure Optima for range of "a"  ROC Curvea=0.200a=0.250a=0.333a=0.500a=1.000a=2.000a=3.000a=4.000a=5.000Single Threshold Area under Curve is: 0.769(a=0.200) Fopt Euc Dist is: 0.153(a=0.250) Fopt Euc Dist is: 0.175(a=0.333) Fopt Euc Dist is: 0.215(a=0.500) Fopt Euc Dist is: 0.297(a=1.000) Fopt Euc Dist is: 0.394(a=2.000) Fopt Euc Dist is: 0.649(a=3.000) Fopt Euc Dist is: 0.685(a=4.000) Fopt Euc Dist is: 0.785(a=5.000) Fopt Euc Dist is: 0.785Early Warning System for  Bathing Water Quality Risks from bathing water quality model errors An ideal classifier model would have both zero false positive rate (FPR)[P/F] and zero false negative rate (FNR)[F/P] (top left corner of plot). In practice a trade-off exists between these. False positives [P/F] mean that the model predicts it’s safe to swim, when a sample would exceed Bacti threshold (2006/7/EC) i.e. a potential public health issue. Conversely false negatives [F/P] mean that the model predicts it is unsafe to swim, when a sample would pass, i.e. economic issues (tourism). Assigning a relative importance factor between these two (‘a’ in formula to right) – sets an optimum operating point on the trade-off curve for the model (centre). Stidson et al. (SEPA report) and others have used a=4 (dashed-red-line). This corresponds with a threshold for the classifier trade-off module (top).  Andrew Duncan, Ed Keedwell, Slobodan Djordjević, Dragan Savić Further Information For further information please contact Prof. Dragan Savić (d.savic@exeter.ac.uk) False negatives [F/P] vs False positives [P/F] Case study beaches Modelling Approach   Inputs (examples) 1. Time of sample w.r.t. HW 2. Tidal Height at HW 3. Est Tidal Height at Sampling Time 4. Tidal Range at Standard Port 5. Tide Level Class (Spring/Mean/Neap) 6. 24hr Antecedent Rainfall Total 7. 48hr Antecedent Rainfall Total 8. 72hr Antecedent Rainfall Total 9. 96hr Antecedent Rainfall Total 10.120hr Antecedent Rainfall Total 11.Salinity        F = measure of skill of model a = Relative importance of negatives [failures] to positives [passes] TN = count of true negatives [F/F] FP = count of false positives [P/F] FN = count of false negatives [F/P]        Benefits of ANN approach 1. Use of Artificial Neural Network (ANN) allows non-linear relationships between inputs and water quality to be modelled 2. Model can be trained for new data / beaches, automatically 3. Most suitable operating point for any trade-off  ‘a’ (FP:FN) found 4. Model can be optimised using area under the trade-off curve to maximise accuracy 5. Flexible options for model inputs used 6. Use of forecast of catchment rainfall would allow Bacti forecasting 7. Principal Component Analysis (PCA) can be applied to select the input signals giving the most skilful model       (Stidson et al., 2012, SEPA report) Trade-off curve:   Terminology [Predicted/Actual]  is used throughout:  [P/F] = predicted pass / actual fail = false positive  [F/P] = predicted fail /   actual pass = false negative  [P/P] = predicted pass / actual pass = true positive  [F/F] = predicted fail /   actual fail = true negative         [P/P] [P/F] Area under Curve: 0.769 Distance from Ideal F 0.153 0.583 0.175 0.576 0.215 0.571 0.297 0.569 0.394 0.597 0.649 0.628 0.685 0.672 0.785 0.716 0.785 0.752 Key Results 

Early Warning System for Bathing Water Quality

Abstract

Similar works

Full text

Available Versions

Open Research Exeter