Numerical Simulations of the Dark Universe: State of the Art and the Next Decade

We present a review of the current state of the art of cosmological dark matter simulations, with particular emphasis on the implications for dark matter detection efforts and studies of dark energy. This review is intended both for particle physicists, who may find the cosmological simulation literature opaque or confusing, and for astro-physicists, who may not be familiar with the role of simulations for observational and experimental probes of dark matter and dark energy. Our work is complementary to the contribution by M. Baldi in this issue, which focuses on the treatment of dark energy and cosmic acceleration in dedicated N-body simulations. Truly massive dark matter-only simulations are being conducted on national supercomputing centers, employing from several billion to over half a trillion particles to simulate the formation and evolution of cosmologically representative volumes (cosmic scale) or to zoom in on individual halos (cluster and galactic scale). These simulations cost millions of core-hours, require tens to hundreds of terabytes of memory, and use up to petabytes of disk storage. The field is quite internationally diverse, with top simulations having been run in China, France, Germany, Korea, Spain, and the USA. Predictions from such simulations touch on almost every aspect of dark matter and dark energy studies, and we give a comprehensive overview of this connection. We also discuss the limitations of the cold and collisionless DM-only approach, and describe in some detail efforts to include different particle physics as well as baryonic physics in cosmological galaxy formation simulations, including a discussion of recent results highlighting how the distribution of dark matter in halos may be altered. We end with an outlook for the next decade, presenting our view of how the field can be expected to progress. (abridged)Comment: 54 pages, 4 figures, 3 tables; invited contribution to the special issue "The next decade in Dark Matter and Dark Energy" of the new Open Access journal "Physics of the Dark Universe". Replaced with accepted versio

Dark Matter Halos from the Inside Out

The balance of evidence indicates that individual galaxies and groups or clusters of galaxies are embedded in enormous distributions of cold, weakly interacting dark matter. These dark matter 'halos' provide the scaffolding for all luminous structure in the universe, and their properties comprise an essential part of the current cosmological model. I review the internal properties of dark matter halos, focussing on the simple, universal trends predicted by numerical simulations of structure formation. Simulations indicate that halos should all have roughly the same spherically-averaged density profile and kinematic structure, and predict simple distributions of shape, formation history and substructure in density and kinematics, over an enormous range of halo mass and for all common variants of the concordance cosmology. I describe observational progress towards testing these predictions by measuring masses, shapes, profiles and substructure in real halos, using baryonic tracers or gravitational lensing. An important property of simulated halos (possibly the most important property) is their dynamical 'age', or degree of internal relaxation. The age of a halo may have almost as much effect as its mass in determining the state of its baryonic contents, so halo ages are also worth trying to measure observationally. I review recent gravitational lensing studies of galaxy clusters which should measure substructure and relaxation in a large sample of individual cluster halos, producing quantitative measures of age that are well-matched to theoretical predictions. The age distributions inferred from these studies will lead to second-generation tests of the cosmological model, as well as an improved understanding of cluster assembly and the evolution of galaxies within clusters.Comment: v2: additional references and minor corrections to match the published versio

The New Horizon Run Cosmological N-Body Simulations

We present two large cosmological N-body simulations, called Horizon Run 2 (HR2) and Horizon Run 3 (HR3), made using 6000^3 = 216 billions and 7210^3 = 374 billion particles, spanning a volume of (7.200 Gpc/h)^3 and (10.815 Gpc/h)^3, respectively. These simulations improve on our previous Horizon Run 1 (HR1) up to a factor of 4.4 in volume, and range from 2600 to over 8800 times the volume of the Millennium Run. In addition, they achieve a considerably finer mass resolution, down to 1.25x10^11 M_sun/h, allowing to resolve galaxy-size halos with mean particle separations of 1.2 Mpc/h and 1.5 Mpc/h, respectively. We have measured the power spectrum, correlation function, mass function and basic halo properties with percent level accuracy, and verified that they correctly reproduce the LCDM theoretical expectations, in excellent agreement with linear perturbation theory. Our unprecedentedly large-volume N-body simulations can be used for a variety of studies in cosmology and astrophysics, ranging from large-scale structure topology, baryon acoustic oscillations, dark energy and the characterization of the expansion history of the Universe, till galaxy formation science - in connection with the new SDSS-III. To this end, we made a total of 35 all-sky mock surveys along the past light cone out to z=0.7 (8 from the HR2 and 27 from the HR3), to simulate the BOSS geometry. The simulations and mock surveys are already publicly available at http://astro.kias.re.kr/Horizon-Run23/.Comment: 18 pages, 10 figures. Added clarification on Fig 6. Published in the Journal of the Korean Astronomical Society (JKAS). The paper with high-resolution figures is available at http://jkas.kas.org/journals/2011v44n6/v44n6.ht

The Rockstar Phase-Space Temporal Halo Finder and the Velocity Offsets of Cluster Cores

We present a new algorithm for identifying dark matter halos, substructure, and tidal features. The approach is based on adaptive hierarchical refinement of friends-of-friends groups in six phase-space dimensions and one time dimension, which allows for robust (grid-independent, shape-independent, and noise-resilient) tracking of substructure; as such, it is named Rockstar (Robust Overdensity Calculation using K-Space Topologically Adaptive Refinement). Our method is massively parallel (up to 10^5 CPUs) and runs on the largest current simulations (>10^10 particles) with high efficiency (10 CPU hours and 60 gigabytes of memory required per billion particles analyzed). A previous paper (Knebe et al 2011) has shown Rockstar to have class-leading recovery of halo properties; we expand on these comparisons with more tests and higher-resolution simulations. We show a significant improvement in substructure recovery as compared to several other halo finders and discuss the theoretical and practical limits of simulations in this regard. Finally, we present results which demonstrate conclusively that dark matter halo cores are not at rest relative to the halo bulk or satellite average velocities and have coherent velocity offsets across a wide range of halo masses and redshifts. For massive clusters, these offsets can be up to 350 km/s at z=0 and even higher at high redshifts. Our implementation is publicly available at http://code.google.com/p/rockstar .Comment: 20 pages, 14 figures. Minor revisions to match accepted versio

SOTXTSTREAM: Density-based self-organizing clustering of text streams

A streaming data clustering algorithm is presented building upon the density-based selforganizing stream clustering algorithm SOSTREAM. Many density-based clustering algorithms are limited by their inability to identify clusters with heterogeneous density. SOSTREAM addresses this limitation through the use of local (nearest neighbor-based) density determinations. Additionally, many stream clustering algorithms use a two-phase clustering approach. In the first phase, a micro-clustering solution is maintained online, while in the second phase, the micro-clustering solution is clustered offline to produce a macro solution. By performing self-organization techniques on micro-clusters in the online phase, SOSTREAM is able to maintain a macro clustering solution in a single phase. Leveraging concepts from SOSTREAM, a new density-based self-organizing text stream clustering algorithm, SOTXTSTREAM, is presented that addresses several shortcomings of SOSTREAM. Gains in clustering performance of this new algorithm are demonstrated on several real-world text stream datasets

Outlier Detection Techniques For Wireless Sensor Networks: A Survey

In the field of wireless sensor networks, measurements that significantly deviate from the normal pattern of sensed data are considered as outliers. The potential sources of outliers include noise and errors, events, and malicious attacks on the network. Traditional outlier detection techniques are not directly applicable to wireless sensor networks due to the multivariate nature of sensor data and specific requirements and limitations of the wireless sensor networks. This survey provides a comprehensive overview of existing outlier detection techniques specifically developed for the wireless sensor networks. Additionally, it presents a technique-based taxonomy and a decision tree to be used as a guideline to select a technique suitable for the application at hand based on characteristics such as data type, outlier type, outlier degree

Outlier detection techniques for wireless sensor networks: A survey

In the field of wireless sensor networks, those measurements that significantly deviate from the normal pattern of sensed data are considered as outliers. The potential sources of outliers include noise and errors, events, and malicious attacks on the network. Traditional outlier detection techniques are not directly applicable to wireless sensor networks due to the nature of sensor data and specific requirements and limitations of the wireless sensor networks. This survey provides a comprehensive overview of existing outlier detection techniques specifically developed for the wireless sensor networks. Additionally, it presents a technique-based taxonomy and a comparative table to be used as a guideline to select a technique suitable for the application at hand based on characteristics such as data type, outlier type, outlier identity, and outlier degree