Castro-Santos, T.

Nebiolo, K.

English

ScholarWorks@UMass Amherst

University of Massachusetts AmherstScholarWorks@UMass AmherstInternational Conference on Engineering andEcohydrology for Fish PassageInternational Conference on Engineering andEcohydrology for Fish Passage 2017Jun 21st, 4:30 PM - 4:50 PMFalse Positive Reduction: An AlgorithmicApproachK. NebioloKleinschmidtT. Castro-SantosUSGSFollow this and additional works at: https://scholarworks.umass.edu/fishpassage_conferenceThis Event is brought to you for free and open access by the Fish Passage Community at UMass Amherst at ScholarWorks@UMass Amherst. It hasbeen accepted for inclusion in International Conference on Engineering and Ecohydrology for Fish Passage by an authorized administrator ofScholarWorks@UMass Amherst. For more information, please contact scholarworks@library.umass.edu.Nebiolo, K. and Castro-Santos, T., "False Positive Reduction: An Algorithmic Approach" (2017). International Conference onEngineering and Ecohydrology for Fish Passage. 7.https://scholarworks.umass.edu/fishpassage_conference/2017/June21/7False Positive Reduction: An Algorithmic ApproachK. NebioloT. Castro-SantosWhat are false positives? Why do we care?• Problems with telemetry data of all types (Beeman & Perry 2012)– Received signal may not correctly assigned– Not all records generated by receivers are from tagged fish– Not all tagged fish are recorded when present• This introduces false positives into our data– Bias results in favor of presence and overestimate frequency of occurrence– Or overestimate residence time within critical infrastructure, which increases measures of delayCurrent False Positive Reduction Methods• Current false positive reduction methods rely simple metrics and subjective opinion – Power floors – Consecutive detections (2, 4, 6, etc…) (Beeman & Perry, 2012)– Logical errors in site progression• Manual classifications are labor intensive– For very large studies with many releases and sites (whole river studies) classification can become cost prohibitive Naïve Bayes Classifier (Minsky 1961)• Bayes rule (conditional probability) can estimate the probability that a record is either true or false positive given observed data  • Prior: marginal probability for a detection class ( ) • Likelihood: The conditional probability of an observable event (    ) given each state of nature ( ) • Posterior: the probability of the class occurring given the observable event • Training data set contains observations of feature variables (   …    ) and known detection classifications ( )• Naïve assumptionPosterior PriorLikelihoodPredictors – Creating the Detection HistoryP X P X P X P P P1 0 1 0 1 0 1 1 10 2 4 6 858565452Current detection in record secondsforwards in timebackwards in timeWas it heard here?and here?Predictors – continued• Derived from detection history– Hit Ratio – Consecutive Hit Length• Power• Noise RatioDetection History (+/- 4) Hit RatioConsecutive Hit Length1 0 1 0 1 0 1 1 1 6/9 31 0 1 0 1 0 1 0 1 5/9 11 1 1 1 1 0 0 0 0 5/9 51 1 1 1 1 1 1 1 1 9/9 9Implementation • Algorithm split into two parts, Training and Classification• Training:– Beacon tags placed at strategic locations throughout the study area  - provide information on what known true positive detections look like– Miss coded and noise transmissions provide information on what false positive detections look like  – Loop over all beacon tag and miss coded detections and derive metrics• (detection history, hit ratio, etc.)– Store to RDBMS in this case SQLite• Classification: – Loop through records for known study tags, – Derive metrics – Calculate posterior for true and false positive – use MAP to classify• Algorithm Accuracy assessed with k-fold cross validation (k = 10)– Sensitivity, Specificity, Positive Predictive Value (PPV) and Negative Predictive Value (NPV)Beacon Tag LocationsResults – Orion Training• In total – 2,644,990 beacon tag hits and 517,881 known false positives ClassifiedFalse TrueTrue Negative 479,349 38,532NPV: 0.93True Positive 39,745 2,605,245PPV: 0.98Specificity: 0.92Sensitivity: 0.99Results – LotekTraining• In total – 2,215,818 beacon tag hits and 331,612 known false positives ClassifiedFalse TrueTrue Negative 298,927 32,685NPV: 0.90True Positive 5,073 2,210,745PPV: 0.99Specificity: 0.98Sensitivity: 0.99Results – Orion Classification575,548 study tag detections• 503,347 true• 72,201 false positive14% False Positive removal rateResults – LotekClassification131,088 study tag detections• 106,693 true• 24,395 false positive23% False Positive removal rateConclusion• Once the algorithm is up and running it takes minimal supervision– ~ 15 minutes every hour – Computationally, detection history creation most time consuming part• Does a great job of removing garbage detections with minimal effort• Does not do a good job at discriminating position when receiver detection zones overlap  • From raw data through statistical analysis with 300 tagged at 14 stations – 2 weeksFuture Research• Investigate lag time between detections – Good detections = stable lag– Bad detections = varying lag• Utilize network/graph topology and investigate ways to pinpoint fish in space & time – 2nd round data classification• Make training data betterWorks CitedBeeman, John W. and Russel W. Perry. "Telemetry Techniques: A User Guide for Fisheries Research; Chapter 9: Bias from False-Positive Detections and Strategies for their Removal in Studies Using Telemetry.“  American Fisheries Society, 2012. 505-517.Minsky, Marvin. "Steps toward artificial intelligence." Proceedings of the IRE 49.1 (1961): 8-30.Beeman, John W. and Russel W. Perry. "Telemetry Techniques: A User Guide for Fisheries Research." Ed. N.S. Adams, J.W. Beeman and J.H. Eiler. American Fisheries Society, 2012. 505-517.Thank youKevin Nebiolo, PhD.Kevin.Nebiolo@kleinschmidtgroup.comTed Castro-Santos, PhD.

False Positive Reduction: An Algorithmic Approach

https://scholarworks.umass.edu/cgi/viewcontent.cgi?article=2147&amp;context=fishpassage_conference

False Positive Reduction: An Algorithmic Approach

Abstract

Similar works

Full text

Available Versions

ScholarWorks@UMass Amherst