Electrostatic Field Classifier for Deficient Data

This paper investigates the suitability of recently developed models based on the physical field phenomena for classification problems with incomplete datasets. An original approach to exploiting incomplete training data with missing features and labels, involving extensive use of electrostatic charge analogy, has been proposed. Classification of incomplete patterns has been investigated using a local dimensionality reduction technique, which aims at exploiting all available information rather than trying to estimate the missing values. The performance of all proposed methods has been tested on a number of benchmark datasets for a wide range of missing data scenarios and compared to the performance of some standard techniques. Several modifications of the original electrostatic field classifier aiming at improving speed and robustness in higher dimensional spaces are also discussed

X-ray Lighthouses of the High-Redshift Universe. II. Further Snapshot Observations of the Most Luminous z>4 Quasars with Chandra

We report on Chandra observations of a sample of 11 optically luminous (Mb<-28.5) quasars at z=3.96-4.55 selected from the Palomar Digital Sky Survey and the Automatic Plate Measuring Facility Survey. These are among the most luminous z>4 quasars known and hence represent ideal witnesses of the end of the "dark age ''. Nine quasars are detected by Chandra, with ~2-57 counts in the observed 0.5-8 keV band. These detections increase the number of X-ray detected AGN at z>4 to ~90; overall, Chandra has detected ~85% of the high-redshift quasars observed with snapshot (few kilosecond) observations. PSS 1506+5220, one of the two X-ray undetected quasars, displays a number of notable features in its rest-frame ultraviolet spectrum, the most prominent being broad, deep SiIV and CIV absorption lines. The average optical-to-X-ray spectral index for the present sample (=-1.88+/-0.05) is steeper than that typically found for z>4 quasars but consistent with the expected value from the known dependence of this spectral index on quasar luminosity. We present joint X-ray spectral fitting for a sample of 48 radio-quiet quasars in the redshift range 3.99-6.28 for which Chandra observations are available. The X-ray spectrum (~870 counts) is well parameterized by a power law with Gamma=1.93+0.10/-0.09 in the rest-frame ~2-40 keV band, and a tight upper limit of N_H~5x10^21 cm^-2 is obtained on any average intrinsic X-ray absorption. There is no indication of any significant evolution in the X-ray properties of quasars between redshifts zero and six, suggesting that the physical processes of accretion onto massive black holes have not changed over the bulk of cosmic time.Comment: 15 pages, 7 figures, accepted for publication in A

Statistical significance of communities in networks

Nodes in real-world networks are usually organized in local modules. These groups, called communities, are intuitively defined as sub-graphs with a larger density of internal connections than of external links. In this work, we introduce a new measure aimed at quantifying the statistical significance of single communities. Extreme and Order Statistics are used to predict the statistics associated with individual clusters in random graphs. These distributions allows us to define one community significance as the probability that a generic clustering algorithm finds such a group in a random graph. The method is successfully applied in the case of real-world networks for the evaluation of the significance of their communities.Comment: 9 pages, 8 figures, 2 tables. The software to calculate the C-score can be found at http://filrad.homelinux.org/cscor

The Time Machine: A Simulation Approach for Stochastic Trees

In the following paper we consider a simulation technique for stochastic trees. One of the most important areas in computational genetics is the calculation and subsequent maximization of the likelihood function associated to such models. This typically consists of using importance sampling (IS) and sequential Monte Carlo (SMC) techniques. The approach proceeds by simulating the tree, backward in time from observed data, to a most recent common ancestor (MRCA). However, in many cases, the computational time and variance of estimators are often too high to make standard approaches useful. In this paper we propose to stop the simulation, subsequently yielding biased estimates of the likelihood surface. The bias is investigated from a theoretical point of view. Results from simulation studies are also given to investigate the balance between loss of accuracy, saving in computing time and variance reduction.Comment: 22 Pages, 5 Figure

Testing linear hypotheses in high-dimensional regressions

For a multivariate linear model, Wilk's likelihood ratio test (LRT) constitutes one of the cornerstone tools. However, the computation of its quantiles under the null or the alternative requires complex analytic approximations and more importantly, these distributional approximations are feasible only for moderate dimension of the dependent variable, say $p\le 20$. On the other hand, assuming that the data dimension $p$ as well as the number $q$ of regression variables are fixed while the sample size $n$ grows, several asymptotic approximations are proposed in the literature for Wilk's \bLa including the widely used chi-square approximation. In this paper, we consider necessary modifications to Wilk's test in a high-dimensional context, specifically assuming a high data dimension $p$ and a large sample size $n$. Based on recent random matrix theory, the correction we propose to Wilk's test is asymptotically Gaussian under the null and simulations demonstrate that the corrected LRT has very satisfactory size and power, surely in the large $p$ and large $n$ context, but also for moderately large data dimensions like $p=30$ or $p=50$. As a byproduct, we give a reason explaining why the standard chi-square approximation fails for high-dimensional data. We also introduce a new procedure for the classical multiple sample significance test in MANOVA which is valid for high-dimensional data.Comment: Accepted 02/2012 for publication in "Statistics". 20 pages, 2 pages and 2 table

Quantum theory of incompatible observations

Maximum likelihood principle is shown to be the best measure for relating the experimental data with the predictions of quantum theory.Comment: 3 page

Quantum homodyne tomography with a priori constraints

I present a novel algorithm for reconstructing the Wigner function from homodyne statistics. The proposed method, based on maximum-likelihood estimation, is capable of compensating for detection losses in a numerically stable way.Comment: 4 pages, REVTeX, 2 figure

Host Galaxy Evolution in Radio-Loud AGN

We investigate the luminosity evolution of the host galaxies of radio-loud AGN through Hubble Space Telescope imaging of 72 BL Lac objects, including new STIS imaging of nine z > 0.6 BL Lacs. With their intrinsically low accretion rates and their strongly beamed jets, BL Lacs provide a unique opportunity to probe host galaxy evolution independent of the biases and ambiguities implicit in quasar studies. We find that the host galaxies of BL Lacs evolve strongly, consistent with passive evolution from a period of active star formation in the range 0.5 <~ z <~ 2.5, and inconsistent with either passive evolution from a high formation redshift or a non-evolving population. This evolution is broadly consistent with that observed in the hosts of other radio-loud AGN, and inconsistent with the flatter luminosity evolution of quiescent early types and radio-quiet hosts. This indicates that active star formation, and hence galaxy interactions, are associated with the formation for radio-loud AGN, and that these host galaxies preferentially accrete less material after their formation epoch than galaxies without powerful radio jets. We discuss possible explanations for the link between merger history and the incidence of a radio jet.Comment: 37 pages, 8 figures, accepted for publication in ApJ, for full PDF incl. figures see http://www.ph.unimelb.edu.au/~modowd/papers/odowdurry2005.pd

Deformation of the N=Z nucleus 76Sr using beta-decay studies

A novel method of deducing the deformation of the N=Z nucleus 76Sr is presented. It is based on the comparison of the experimental Gamow-Teller strength distribution B(GT) from its beta decay with the results of QRPA calculations. This method confirms previous indications of the strong prolate deformation of this nucleus in a totally independent way. The measurement has been carried out with a large Total Absorption gamma Spectrometer, "Lucrecia", newly installed at CERN-ISOLDE.Comment: Accepted in Phys. Rev. Letter

Data Mining and Machine Learning in Astronomy

We review the current state of data mining and machine learning in astronomy. 'Data Mining' can have a somewhat mixed connotation from the point of view of a researcher in this field. If used correctly, it can be a powerful approach, holding the potential to fully exploit the exponentially increasing amount of available data, promising great scientific advance. However, if misused, it can be little more than the black-box application of complex computing algorithms that may give little physical insight, and provide questionable results. Here, we give an overview of the entire data mining process, from data collection through to the interpretation of results. We cover common machine learning algorithms, such as artificial neural networks and support vector machines, applications from a broad range of astronomy, emphasizing those where data mining techniques directly resulted in improved science, and important current and future directions, including probability density functions, parallel algorithms, petascale computing, and the time domain. We conclude that, so long as one carefully selects an appropriate algorithm, and is guided by the astronomical problem at hand, data mining can be very much the powerful tool, and not the questionable black box.Comment: Published in IJMPD. 61 pages, uses ws-ijmpd.cls. Several extra figures, some minor additions to the tex