Log-based Anomaly Detection of CPS Using a Statistical Method

Detecting anomalies of a cyber physical system (CPS), which is a complex system consisting of both physical and software parts, is important because a CPS often operates autonomously in an unpredictable environment. However, because of the ever-changing nature and lack of a precise model for a CPS, detecting anomalies is still a challenging task. To address this problem, we propose applying an outlier detection method to a CPS log. By using a log obtained from an actual aquarium management system, we evaluated the effectiveness of our proposed method by analyzing outliers that it detected. By investigating the outliers with the developer of the system, we confirmed that some outliers indicate actual faults in the system. For example, our method detected failures of mutual exclusion in the control system that were unknown to the developer. Our method also detected transient losses of functionalities and unexpected reboots. On the other hand, our method did not detect anomalies that were too many and similar. In addition, our method reported rare but unproblematic concurrent combinations of operations as anomalies. Thus, our approach is effective at finding anomalies, but there is still room for improvement

redMaPPer III: A Detailed Comparison of the Planck 2013 and SDSS DR8 RedMaPPer Cluster Catalogs

We compare the Planck Sunyaev-Zeldovich (SZ) cluster sample (PSZ1) to the Sloan Digital Sky Survey (SDSS) redMaPPer catalog, finding that all Planck clusters within the redMaPPer mask and within the redshift range probed by redMaPPer are contained in the redMaPPer cluster catalog. These common clusters define a tight scaling relation in the richness-SZ mass ($\lambda$--$M_{SZ}$) plane, with an intrinsic scatter in richness of $\sigma_{\lambda|M_{SZ}} = 0.266 \pm 0.017$. The corresponding intrinsic scatter in true cluster halo mass at fixed richness is $\approx 21\%$. The regularity of this scaling relation is used to identify failures in both the redMaPPer and Planck cluster catalogs. Of the 245 galaxy clusters in common, we identify three failures in redMaPPer and 36 failures in the PSZ1. Of these, at least 12 are due to clusters whose optical counterpart was correctly identified in the PSZ1, but where the quoted redshift for the optical counterpart in the external data base used in the PSZ1 was incorrect. The failure rates for redMaPPer and the PSZ1 are $1.2\%$ and $14.7\%$ respectively, or 9.8% in the PSZ1 after subtracting the external data base errors. We have further identified 5 PSZ1 sources that suffer from projection effects (multiple rich systems along the line-of-sight of the SZ detection) and 17 new high redshift ($z\gtrsim 0.6$) cluster candidates of varying degrees of confidence. Should all of the high-redshift cluster candidates identified here be confirmed, we will have tripled the number of high redshift Planck clusters in the SDSS region. Our results highlight the power of multi-wavelength observations to identify and characterize systematic errors in galaxy cluster data sets, and clearly establish photometric data both as a robust cluster finding method, and as an important part of defining clean galaxy cluster samples.Comment: comments welcom

A binary neural k-nearest neighbour technique

K-Nearest Neighbour (k-NN) is a widely used technique for classifying and clustering data. K-NN is effective but is often criticised for its polynomial run-time growth as k-NN calculates the distance to every other record in the data set for each record in turn. This paper evaluates a novel k-NN classifier with linear growth and faster run-time built from binary neural networks. The binary neural approach uses robust encoding to map standard ordinal, categorical and real-valued data sets onto a binary neural network. The binary neural network uses high speed pattern matching to recall the k-best matches. We compare various configurations of the binary approach to a conventional approach for memory overheads, training speed, retrieval speed and retrieval accuracy. We demonstrate the superior performance with respect to speed and memory requirements of the binary approach compared to the standard approach and we pinpoint the optimal configurations

The Infrared Database of Extragalactic Observables from Spitzer I: the redshift catalog

This is the first of a series of papers on the Infrared Database of Extragalactic Observables from Spitzer (IDEOS). In this work we describe the identification of optical counterparts of the infrared sources detected in Spitzer Infrared Spectrograph (IRS) observations, and the acquisition and validation of redshifts. The IDEOS sample includes all the spectra from the Cornell Atlas of Spitzer/IRS Sources (CASSIS) of galaxies beyond the Local Group. Optical counterparts were identified from correlation of the extraction coordinates with the NASA Extragalactic Database (NED). To confirm the optical association and validate NED redshifts, we measure redshifts with unprecedented accuracy on the IRS spectra ({\sigma}(dz/(1+z))=0.0011) by using an improved version of the maximum combined pseudo-likelihood method (MCPL). We perform a multi-stage verification of redshifts that considers alternate NED redshifts, the MCPL redshift, and visual inspection of the IRS spectrum. The statistics is as follows: the IDEOS sample contains 3361 galaxies at redshift 0<z<6.42 (mean: 0.48, median: 0.14). We confirm the default NED redshift for 2429 sources and identify 124 with incorrect NED redshifts. We obtain IRS-based redshifts for 568 IDEOS sources without optical spectroscopic redshifts, including 228 with no previous redshift measurements. We provide the entire IDEOS redshift catalog in machine-readable formats. The catalog condenses our compilation and verification effort, and includes our final evaluation on the most likely redshift for each source, its origin, and reliability estimates.Comment: 11 pages, 6 figures, 1 table. Accepted for publication in MNRAS. Full redshift table in machine-readable format available at http://ideos.astro.cornell.edu/redshifts.htm