SDCOR: Scalable Density-based Clustering for Local Outlier Detection in Massive-Scale Datasets

This paper presents a batch-wise density-based clustering approach for local outlier detection in massive-scale datasets. Unlike the well-known traditional algorithms, which assume that all the data is memory-resident, our proposed method is scalable and processes the input data chunk-by-chunk within the confines of a limited memory buffer. A temporary clustering model is built at the first phase; then, it is gradually updated by analyzing consecutive memory loads of points. Subsequently, at the end of scalable clustering, the approximate structure of the original clusters is obtained. Finally, by another scan of the entire dataset and using a suitable criterion, an outlying score is assigned to each object called SDCOR (Scalable Density-based Clustering Outlierness Ratio). Evaluations on real-life and synthetic datasets demonstrate that the proposed method has a low linear time complexity and is more effective and efficient compared to best-known conventional density-based methods, which need to load all data into the memory; and also, to some fast distance-based methods, which can perform on data resident in the disk.Comment: Highlights are shortened each to about 85 character

Legal perspective on possible fairness measures – A legal discussion using the example of hiring decisions

With the increasing use of AI in algorithmic decision making (e.g. based on neural networks), the question arises how bias can be excluded or mitigated. There are some promising approaches, but many of them are based on a ”fair” ground truth, others are based on a subjective goal to be reached, which leads to the usual problem of how to define and compute ”fairness”. The different functioning of algorithmic decision making in contrast to human decision making leads to a shift from a process-oriented to a result-oriented discrimination assessment. We argue that with such a shift society needs to determine which kind of fairness is the right one to choose for which certain scenario. To understand the implications of such a determination we explain the different kinds of fairness concepts that might be applicable for the specific application of hiring decisions, analyze their pros and cons with regard to the respective fairness interpretation and evaluate them from a legal perspective (based on EU law)