Feature Selection and Weighting by Nearest Neighbor Ensembles

In the field of statistical discrimination nearest neighbor methods are a well known, quite simple but successful nonparametric classification tool. In higher dimensions, however, predictive power normally deteriorates. In general, if some covariates are assumed to be noise variables, variable selection is a promising approach. The paper’s main focus is on the development and evaluation of a nearest neighbor ensemble with implicit variable selection. In contrast to other nearest neighbor approaches we are not primarily interested in classification, but in estimating the (posterior) class probabilities. In simulation studies and for real world data the proposed nearest neighbor ensemble is compared to an extended forward/backward variable selection procedure for nearest neighbor classifiers, and some alternative well established classification tools (that offer probability estimates as well). Despite its simple structure, the proposed method’s performance is quite good - especially if relevant covariates can be separated from noise variables. Another advantage of the presented ensemble is the easy identification of interactions that are usually hard to detect. So not simply variable selection but rather some kind of feature selection is performed. The paper is a preprint of an article published in Chemometrics and Intelligent Laboratory Systems. Please use the journal version for citation

Random matrix analysis of complex networks

We study complex networks under random matrix theory (RMT) framework. Using nearest-neighbor and next-nearest-neighbor spacing distributions we analyze the eigenvalues of adjacency matrix of various model networks, namely, random, scale-free and small-world networks. These distributions follow Gaussian orthogonal ensemble statistic of RMT. To probe long-range correlations in the eigenvalues we study spectral rigidity via $\Delta_3$ statistic of RMT as well. It follows RMT prediction of linear behavior in semi-logarithmic scale with slope being $\sim 1/\pi^2$. Random and scale-free networks follow RMT prediction for very large scale. Small-world network follows it for sufficiently large scale, but much less than the random and scale-free networks.Comment: accepted in Phys. Rev. E (replaced with the final version

Relationship Between Neighbor Number and Vibrational Spectra in disordered colloidal clusters with attractive interactions

We study connections between vibrational spectra and average nearest neighbor number in disordered clusters of colloidal particles with attractive interactions. Measurements of displacement covariances between particles in each cluster permit calculation of the stiffness matrix, which contains effective spring constants linking pairs of particles. From the cluster stiffness matrix, we derive vibrational properties of corresponding “shadow” glassy clusters, with the same geometric configuration and interactions as the “source” cluster but without damping. Here, we investigate the stiffness matrix to elucidate the origin of the correlations between the median frequency of cluster vibrational modes and average number of nearest neighbors in the cluster. We find that the mean confining stiffness of particles in a cluster, i.e., the ensemble-averaged sum of nearest neighbor spring constants, correlates strongly with average nearest neighbor number, and even more strongly with median frequency. Further, we find that the average oscillation frequency of an individual particle is set by the total stiffness of its nearest neighbor bonds; this average frequency increases as the square root of the nearest neighbor bond stiffness, in a manner similar to the simple harmonic oscillator

U(1) staggered Dirac operator and random matrix

We investigate the spectrum of the staggered Dirac operator in 4d quenched U(1) lattice gauge theory and its relationship to random matrix theory. In the confined as well as in the Coulomb phase the nearest-neighbor spacing distribution of the unfolded eigenvalues is well described by the chiral unitary ensemble. The same is true for the distribution of the smallest eigenvalue and the microscopic spectral density in the confined phase. The physical origin of the chiral condensate in this phase deserves further study.Comment: LATTICE99 (theoretical developments), 3 pages, 7 figure