Algorithms and the Foundations of Software technolog

Bruin, G.J. de

Herik, H.J. van den

Pereira Barata, A.P.

Takes, F.W.

Veenman, C.J.

Leiden University Scholary Publications

Data-driven risk assessment in infrastructure networksPereira Barata, A.P.; Bruin, G.J. de; Takes, F.W.; Veenman, C.J.; Herik, H.J. van denCitationPereira Barata, A. P., Bruin, G. J. de, Takes, F. W., Veenman, C. J., & Herik, H. J. van den.(2018). Data-driven risk assessment in infrastructure networks. Retrieved fromhttps://hdl.handle.net/1887/3283777 Version: Not Applicable (or Unknown)License: Leiden University Non-exclusive licenseDownloaded from: https://hdl.handle.net/1887/3283777 Note: To cite this publication please use the final published version (if applicable).Data-driven Risk Assessment in Infrastructure NetworksAntónio Pereira Barata1,2, Gerrit-Jan de Bruin1,2, Frank Takes1, and Cor Veenman1,3{a.p.pereira.barata,g.j.de.bruin,f.w.takes,c.j.veenman}@liacs.leidenuniv.nl1Leiden Center of Data Science, Leiden University2Human Environment and Transport Inspectorate, Ministry of Infrastructure and Water Management3Data Science Department, TNO1 IntroductionGovernmental bodies commit to ensuring citizensafety, security, and trust. In order to make thehighest impact possible, these agencies need towisely choose how to allocate their limited assets.To accomplish this, the Human Environment andTransport Inspectorate (ILT) recognized the needfor data science and network science as tools fordata-driven approaches for their inspections. Inthis paper we describe a use case in which ma-chine learning is able to identify possible fraudu-lent waste notices. The Waste Shipments Regu-lation (WSR) comprises the legislation that com-panies must follow in order to transport wastematerials through a European Union (EU) mem-ber state. The legislation requires that a com-pany wanting to transfer waste reports the typeof waste, as the cost for each type is different.As a result, some companies might intentionallymislabel their waste. Hence, finding these mis-labeled waste transports using machine learningtechniques is the topic of this paper.2 DataILT publishes all notices of waste transport ontheir website1. We obtained a dump of this databelonging to the years 2008 through 2016. Thedata contains 300 thousand entries of 19 vari-ables. Because some of them were of low qual-ity, only 9 nominal and 1 numerical variableswere selected. Type of waste (target) is catego-rized in 20 classes defined in the European Listof Waste (LoW)2.1https://english.ilent.nl/themes/international-shipment-of-waste2http://ec.europa.eu/environment/waste/framework/list.htm3 Problem statementThe goal of this study is to use anomaly detec-tion to find transports that strongly deviate fromother transports within the reported waste cate-gory, which could imply the fraudulent behaviordescribed in Section 1.4 ApproachWe perform anomaly detection by means of su-pervised learning. A standard linear support vec-tor machine classifier (LSVMC) was chosen tomodel our data [1]; a five-fold train-test splitwas performed for cross-validation. It is neces-sary for categorical variables to have their val-ues represented numerically. Therefore, we per-formed variable binarization on all categoricalvariables. We model each category class (one-vs-rest) and determine, for every point labeledas that category, whether it is an outlier. Anoutlier is defined as a data point with a score be-low Q1 − 1.5 × IQR or above Q3 + 1.5 × IQR,where IQR (inter-quartile ratio) depicts the valuedifference between the first and third quartiles(Q1 and Q3, respectively), within all false neg-ative (FN) scores associated to a distinct classi-fier. If a data point is determined as an outlierby a model with good performance, this couldbe indicative of non-compliance. As the size ofthe class categories varies significantly, evaluationmetrics that take into account the number of truenegatives (e.g., accuracy fall-out, ROC and AU-ROC) are not applicable. Instead, we will useaverage precision (AP): a high value indicatesthat the model is able to efficiently classify datapoints regarding a particular class. [2]. Givenwell-performing classifiers, the most extreme lowscores are most probable outliers.1Figure 1: Visual representation (boxplot) of the distribution of decision function scores (log scale) belonging toFN predictions per unique classifier and associated average precision (AP) rounded to 3 decimal places.5 ResultsAs shown in Figure 1, out of the total 20 clas-sifiers, 13 exhibited an average precision score≥ 0.9 while only 3 classifiers were evaluated ata score < 0.8. Considering classifiers with highAP value, each lowest-scoring FN prediction percategory classifier is most likely an outlier: thedata-point is furthest away from all other datapoints pertaining to the same class, within featurespace. For example, the most reliable classifier(AP ≈ 1) is the support vector machine trainedto determine whether a data point belongs to thethird LoW category. One can expect, hence, thatits predictions are reliable. By analysing the scoredistribution of classifier 3, it is simple to observea data point with a score close to −10 (blue pointimmediately below the lower whisker of the cor-responding boxplot). This means that the fea-ture values associated to this data point are sig-nificantly different from all other points with thesame label. Therefore, it is a data point thatshould be inspected further.6 ConclusionThis work aims to determine what transportationentries are the most likely to be fraudulently doc-umented, making use of data-driven approaches.In order to validate our results, we will cooper-ate with domain experts in future work to checkwhether the outliers predicted by our model areindeed associated to non-compliant transporta-tion events and, consequently, company misbe-haviour that may result in applicable sanctions.References[1] T. Hastie, R. Tibshirani, and J. Fried-man, The Elements of Statistical Learning.Springer, 2 ed., 2009.[2] T. Saito and M. Rehmsmeier, “The Precision-Recall Plot Is More Informative than theROC Plot When Evaluating Binary Classi-fiers on Imbalanced Datasets,” PLoS ONE,vol. 10, no. 3, 2015.2

Data-driven risk assessment in infrastructure networks

https://scholarlypublications.universiteitleiden.nl/access/item%3A3283778/download

Data-driven risk assessment in infrastructure networks

Abstract

Similar works

Full text

Available Versions

Leiden University Scholary Publications