The Effect of Security Education and Expertise on Security Assessments: the Case of Software Vulnerabilities

In spite of the growing importance of software security and the industry demand for more cyber security expertise in the workforce, the effect of security education and experience on the ability to assess complex software security problems has only been recently investigated. As proxy for the full range of software security skills, we considered the problem of assessing the severity of software vulnerabilities by means of a structured analysis methodology widely used in industry (i.e. the Common Vulnerability Scoring System (\CVSS) v3), and designed a study to compare how accurately individuals with background in information technology but different professional experience and education in cyber security are able to assess the severity of software vulnerabilities. Our results provide some structural insights into the complex relationship between education or experience of assessors and the quality of their assessments. In particular we find that individual characteristics matter more than professional experience or formal education; apparently it is the \emph{combination} of skills that one owns (including the actual knowledge of the system under study), rather than the specialization or the years of experience, to influence more the assessment quality. Similarly, we find that the overall advantage given by professional expertise significantly depends on the composition of the individual security skills as well as on the available information.Comment: Presented at the Workshop on the Economics of Information Security (WEIS 2018), Innsbruck, Austria, June 201

An Effective Meaningful Way to Evaluate Survival Models

One straightforward metric to evaluate a survival prediction model is based on the Mean Absolute Error (MAE) -- the average of the absolute difference between the time predicted by the model and the true event time, over all subjects. Unfortunately, this is challenging because, in practice, the test set includes (right) censored individuals, meaning we do not know when a censored individual actually experienced the event. In this paper, we explore various metrics to estimate MAE for survival datasets that include (many) censored individuals. Moreover, we introduce a novel and effective approach for generating realistic semi-synthetic survival datasets to facilitate the evaluation of metrics. Our findings, based on the analysis of the semi-synthetic datasets, reveal that our proposed metric (MAE using pseudo-observations) is able to rank models accurately based on their performance, and often closely matches the true MAE -- in particular, is better than several alternative methods.Comment: Accepted to ICML 202

Scalable Population Synthesis with Deep Generative Modeling

Population synthesis is concerned with the generation of synthetic yet realistic representations of populations. It is a fundamental problem in the modeling of transport where the synthetic populations of micro-agents represent a key input to most agent-based models. In this paper, a new methodological framework for how to 'grow' pools of micro-agents is presented. The model framework adopts a deep generative modeling approach from machine learning based on a Variational Autoencoder (VAE). Compared to the previous population synthesis approaches, including Iterative Proportional Fitting (IPF), Gibbs sampling and traditional generative models such as Bayesian Networks or Hidden Markov Models, the proposed method allows fitting the full joint distribution for high dimensions. The proposed methodology is compared with a conventional Gibbs sampler and a Bayesian Network by using a large-scale Danish trip diary. It is shown that, while these two methods outperform the VAE in the low-dimensional case, they both suffer from scalability issues when the number of modeled attributes increases. It is also shown that the Gibbs sampler essentially replicates the agents from the original sample when the required conditional distributions are estimated as frequency tables. In contrast, the VAE allows addressing the problem of sampling zeros by generating agents that are virtually different from those in the original data but have similar statistical properties. The presented approach can support agent-based modeling at all levels by enabling richer synthetic populations with smaller zones and more detailed individual characteristics.Comment: 27 pages, 15 figures, 4 table

Climate change may have minor impact on zooplankton functional diversity in the Mediterranean Sea

Aim To assess the impact of climate change on the functional diversity of marine zooplankton communities. Location The Mediterranean Sea. Methods We used the functional traits and geographic distributions of 106 copepod species to estimate the zooplankton functional diversity of Mediterranean surface assemblages for the 1965–1994 and 2069–2098 periods. Multiple environmental niche models were trained at the global scale to project the species habitat suitability in the Mediterranean Sea and assess their sensitivity to climate change predicted by several scenarios. Simultaneously, the species traits were used to compute a functional dendrogram from which we identified seven functional groups and estimated functional diversity through Faith's index. We compared the measured functional diversity to the one originated from null models to test if changes in functional diversity were solely driven by changes in species richness. Results All but three of the 106 species presented range contractions of varying intensity. A relatively low decrease of species richness (−7.42 on average) is predicted for 97% of the basin, with higher losses in the eastern regions. Relative sensitivity to climate change is not clustered in functional space and does not significantly vary across the seven copepod functional groups defined. Changes in functional diversity follow the same pattern and are not different from those that can be expected from changes in richness alone. Main conclusions Climate change is not expected to alter copepod functional traits distribution in the Mediterranean Sea, as the most and the least sensitive species are functionally redundant. Such redundancy should buffer the loss of ecosystem functions in Mediterranean zooplankton assemblages induced by climate change. Because the most negatively impacted species are affiliated to temperate regimes and share Atlantic biogeographic origins, our results are in line with the hypothesis of increasingly more tropical Mediterranean communities