Fundamental Limits of Spectral Clustering in Stochastic Block Models

We give a precise characterization of the performance of spectral clustering for community detection under Stochastic Block Models by carrying out sharp statistical analysis. We show spectral clustering has an exponentially small error with matching upper and lower bounds that have the same exponent, including the sharp leading constant. The fundamental limits established for the spectral clustering hold for networks with multiple and imbalanced communities and sparse networks with degrees far smaller than $\log n$. The key to our results is a novel truncated $\ell_2$ perturbation analysis for eigenvectors and a new analysis idea of eigenvectors truncation

Clustering-based Redshift Estimation: Comparison to Spectroscopic Redshifts

We investigate the potential and accuracy of clustering-based redshift estimation using the method proposed by M\'enard et al. (2013). This technique enables the inference of redshift distributions from measurements of the spatial clustering of arbitrary sources, using a set of reference objects for which redshifts are known. We apply it to a sample of spectroscopic galaxies from the Sloan Digital Sky Survey and show that, after carefully controlling the sampling efficiency over the sky, we can estimate redshift distributions with high accuracy. Probing the full colour space of the SDSS galaxies, we show that we can recover the corresponding mean redshifts with an accuracy ranging from $\delta$z=0.001 to 0.01. We indicate that this mapping can be used to infer the redshift probability distribution of a single galaxy. We show how the lack of information on the galaxy bias limits the accuracy of the inference and show comparisons between clustering redshifts and photometric redshifts for this dataset. This analysis demonstrates, using real data, that clustering-based redshift inference provides a powerful data-driven technique to explore the redshift distribution of arbitrary datasets, without any prior knowledge on the spectral energy distribution of the sources.Comment: 13 pages. Submitted to MNRAS. Comments welcom

Community detection and stochastic block models: recent developments

The stochastic block model (SBM) is a random graph model with planted clusters. It is widely employed as a canonical model to study clustering and community detection, and provides generally a fertile ground to study the statistical and computational tradeoffs that arise in network and data sciences. This note surveys the recent developments that establish the fundamental limits for community detection in the SBM, both with respect to information-theoretic and computational thresholds, and for various recovery requirements such as exact, partial and weak recovery (a.k.a., detection). The main results discussed are the phase transitions for exact recovery at the Chernoff-Hellinger threshold, the phase transition for weak recovery at the Kesten-Stigum threshold, the optimal distortion-SNR tradeoff for partial recovery, the learning of the SBM parameters and the gap between information-theoretic and computational thresholds. The note also covers some of the algorithms developed in the quest of achieving the limits, in particular two-round algorithms via graph-splitting, semi-definite programming, linearized belief propagation, classical and nonbacktracking spectral methods. A few open problems are also discussed

Detection of the Cosmic Far-Infrared Background in the AKARI Deep Field South

We report the detection and measurement of the absolute brightness and spatial fluctuations of the cosmic infrared background (CIB) with the AKARI satellite. We have carried out observations at 65, 90, 140 and 160 um as a cosmological survey in AKARI Deep Field South (ADF-S), which is one of the lowest cirrus regions with contiguous area on the sky. After removing bright galaxies and subtracting zodiacal and Galactic foregrounds from the measured sky brightness, we have successfully measured the CIB brightness and its fluctuations across a wide range of angular scales from arcminutes to degrees. The measured CIB brightness is consistent with previous results reported from COBE data but significantly higher than the lower limits at 70 and 160 um obtained with the Spitzer satellite from the stacking analysis of 24-um selected sources. The discrepancy with the Spitzer result is possibly due to a new galaxy population at high redshift obscured by hot dust. From power spectrum analysis at 90 um, three components are identified: shot noise due to individual galaxies; Galactic cirrus emission dominating at the largest angular scales of a few degrees; and an additional component at an intermediate angular scale of 10-30 arcminutes, possibly due to galaxy clustering. The spectral shape of the clustering component at 90 um is very similar to that at longer wavelengths as observed by Spitzer and BLAST. Moreover, the color of the fluctuations indicates that the clustering component is as red as Ultra-luminous infrared galaxies (ULIRGs) at high redshift, These galaxies are not likely to be the majority of the CIB emission at 90 um, but responsible for the clustering component. Our results provide new constraints on the evolution and clustering properties of distant infrared galaxies.Comment: 50 pages, 15 figures, submitted to Ap

Fluctuation Operators and Spontaneous Symmetry Breaking

We develop an alternative approach to this field, which was to a large extent developed by Verbeure et al. It is meant to complement their approach, which is largely based on a non-commutative central limit theorem and coordinate space estimates. In contrast to that we deal directly with the limits of $l$-point truncated correlation functions and show that they typically vanish for $l\geq 3$ provided that the respective scaling exponents of the fluctuation observables are appropriately chosen. This direct approach is greatly simplified by the introduction of a smooth version of spatial averaging, which has a much nicer scaling behavior and the systematic developement of Fourier space and energy-momentum spectral methods. We both analyze the regime of normal fluctuations, the various regimes of poor clustering and the case of spontaneous symmetry breaking or Goldstone phenomenon.Comment: 30 pages, Latex, a more detailed discussion in section 7 as to possible scaling behavior of l-point function

Spectral Efficient and Energy Aware Clustering in Cellular Networks

The current and envisaged increase of cellular traffic poses new challenges to Mobile Network Operators (MNO), who must densify their Radio Access Networks (RAN) while maintaining low Capital Expenditure and Operational Expenditure to ensure long-term sustainability. In this context, this paper analyses optimal clustering solutions based on Device-to-Device (D2D) communications to mitigate partially or completely the need for MNOs to carry out extremely dense RAN deployments. Specifically, a low complexity algorithm that enables the creation of spectral efficient clusters among users from different cells, denoted as enhanced Clustering Optimization for Resources' Efficiency (eCORE) is presented. Due to the imbalance between uplink and downlink traffic, a complementary algorithm, known as Clustering algorithm for Load Balancing (CaLB), is also proposed to create non-spectral efficient clusters when they result in a capacity increase. Finally, in order to alleviate the energy overconsumption suffered by cluster heads, the Clustering Energy Efficient algorithm (CEEa) is also designed to manage the trade-off between the capacity enhancement and the early battery drain of some users. Results show that the proposed algorithms increase the network capacity and outperform existing solutions, while, at the same time, CEEa is able to handle the cluster heads energy overconsumption