Deep unsupervised clustering with Gaussian mixture variational autoencoders

We study a variant of the variational autoencoder model with a Gaussian mixture as a prior distribution, with the goal of performing unsupervised clustering through deep generative models. We observe that the standard variational approach in these models is unsuited for unsupervised clustering, and mitigate this problem by leveraging a principled information-theoretic regularisation term known as consistency violation. Adding this term to the standard variational optimisation objective yields networks with both meaningful internal representations and well-defined clusters. We demonstrate the performance of this scheme on synthetic data, MNIST and SVHN, showing that the obtained clusters are distinct, interpretable and result in achieving higher performance on unsupervised clustering classification than previous approaches

Orthogonally Decoupled Variational Gaussian Processes

Gaussian processes (GPs) provide a powerful non-parametric framework for reasoning over functions. Despite appealing theory, its superlinear computational and memory complexities have presented a long-standing challenge. State-of-the-art sparse variational inference methods trade modeling accuracy against complexity. However, the complexities of these methods still scale superlinearly in the number of basis functions, implying that that sparse GP methods are able to learn from large datasets only when a small model is used. Recently, a decoupled approach was proposed that removes the unnecessary coupling between the complexities of modeling the mean and the covariance functions of a GP. It achieves a linear complexity in the number of mean parameters, so an expressive posterior mean function can be modeled. While promising, this approach suffers from optimization difficulties due to ill-conditioning and non-convexity. In this work, we propose an alternative decoupled parametrization. It adopts an orthogonal basis in the mean function to model the residues that cannot be learned by the standard coupled approach. Therefore, our method extends, rather than replaces, the coupled approach to achieve strictly better performance. This construction admits a straightforward natural gradient update rule, so the structure of the information manifold that is lost during decoupling can be leveraged to speed up learning. Empirically, our algorithm demonstrates significantly faster convergence in multiple experiments

Optical coherence tomography- a non-invasive technique applied to conservation of paintings

It is current practice to take tiny samples from a painting to mount and examine in cross-section under a microscope. However, since conservation practice and ethics limit sampling to a minimum and to areas along cracks and edges of paintings, which are often unrepresentative of the whole painting, results from such analyses cannot be taken as representative of a painting as a whole. Recently in a preliminary study, we have demonstrated that near-infrared Optical Coherence Tomography (OCT) can be used directly on paintings to examine the cross-section of paint and varnish layers without contact and the need to take samples. OCT is an optical interferometric technique developed for in vivo imaging of the eye and biological tissues; it is essentially a scanning Michelson’s interferometer with a ‘broadband’ source that has the spatial coherence of a laser. The low temporal coherence and high spatial concentration of the source are the keys to high depth resolution and high sensitivity 3D imaging. The technique is non-invasive and noncontact with a typical working distance of 2 cm. This non-invasive technique enables cross-sections to be examined anywhere on a painting. In this paper, we will report new results on applying near-infrared en-face OCT to paintings conservation and extend the application to the examination of underdrawings, drying processes, and quantitative measurements of optical properties of paint and varnish layers

GPflux: A Library for Deep Gaussian Processes

We introduce GPflux, a Python library for Bayesian deep learning with a strong emphasis on deep Gaussian processes (DGPs). Implementing DGPs is a challenging endeavour due to the various mathematical subtleties that arise when dealing with multivariate Gaussian distributions and the complex bookkeeping of indices. To date, there are no actively maintained, open-sourced and extendable libraries available that support research activities in this area. GPflux aims to fill this gap by providing a library with state-of-the-art DGP algorithms, as well as building blocks for implementing novel Bayesian and GP-based hierarchical models and inference schemes. GPflux is compatible with and built on top of the Keras deep learning eco-system. This enables practitioners to leverage tools from the deep learning community for building and training customised Bayesian models, and create hierarchical models that consist of Bayesian and standard neural network layers in a single coherent framework. GPflux relies on GPflow for most of its GP objects and operations, which makes it an efficient, modular and extensible library, while having a lean codebase

The Morphology of Passively Evolving Galaxies at Z-2 from HST/WFC3 in the Hubble Ultra Deep Field

We discuss near-IR images of six passive galaxies (SSFR< 10(exp -2)/Gyr) at redshift 1.3 < z < 2.4 with stellar mass M approx 10(exp 11) solar mass, selected from the Great Observatories Origins Deep Survey (GOODS), obtained with WFC3/IR and the Hubble Space Telescope (HST). These WFC3 images provide the deepest and highest angular resolution view of the optical rest-frame morphology of such systems to date. We find that the light profile of these; galaxies is generally regular and well described by a Sersic model with index typical of today's spheroids. We confirm the existence of compact and massive early-type galaxies at z approx. 2: four out of six galaxies have T(sub e) approx. 1 kpc or less. The WFC3 images achieve limiting surface brightness mu approx. 26.5 mag/sq arcsec in the F160W bandpass; yet there is no evidence of a faint halo in the five compact galaxies of our sample, nor is a halo observed in their stacked image. We also find very weak "morphological k-correction" in the galaxies between the rest-frame UV (from the ACS z band), and the rest-frame optical (WFC3 H band): the visual classification, Sersic indices and physical sizes of these galaxies are independent or only mildly dependent on the wavelength, within the errors

The Morphology of Passively Evolving Galaxies at Z approximately 2 from HST/WFC3 Deep Imaging in the Hubble Ultradeep Field

We present near-IR images of six passive galaxies (SSFR< 10(exp -2)/ Gyr) at redshift 1.3 < z < 2.4 with stellar mass M approximately 10(exp 11) solar M, selected from the Great Observatories Origins Deep Survey (GOODS), obtained with the Hubble Space Telescope (HST) and the WFC3/IR camera. These images provide the deepest and highest angular resolution view of the optical rest-frame morphology of such systems to date. We find that the light profile of these galaxies is generally regular and well described by a Sersic model with index typical of today's spheroids. We confirm the existence of compact and massive early-type galaxies at z approximately 2: four out of six galaxies have r(sub e) approximately 1 kpc or less. The images reach limiting surface brightness mu approximates 26.5 mag/square arcsec in the F160W bandpass; yet there is no evidence of a faint halo in the galaxies of our sample, even in their stacked image. We also find very weak "morphological k-correction" in the galaxies between the rest-frame UV (from the ACS z-band), and the rest-frame optical (WFC3 H-band): the visual classification, Sersic indices and physical sizes of these galaxies are independent or only mildly dependent on the wavelength, within the errors. The presence of an active nucleus is suspected in two out of six galaxies (33%), opening the intriguing possibility that a large, presently unaccounted population of AGN is hosted in these galaxies, possibly responsible for the cessation of star formation

The PEP survey: clustering of infrared-selected galaxies and structure formation at z~2 in the GOODS South

ABRIDGED-This paper presents the first direct estimate of the 3D clustering properties of far-infrared sources up to z~3. This has been possible thanks to the Pacs Evolutionary Probe (PEP) survey of the GOODS South field performed with the PACS instrument onboard the Herschel Satellite. An analysis of the two-point correlation function over the whole redshift range spanned by the data reports for the correlation length, r_0~6.3 Mpc and r_0~6.7 Mpc, respectively at 100um and 160um, corresponding to dark matter halo masses M>~10^{12.4} M_sun. Objects at z~2 instead seem to be more strongly clustered, with r_0~19 Mpc and r_0~17 Mpc in the two considered PACS channels. This dramatic increase of the correlation length between z~1 and z~2 is connected with the presence of a wide, M>~10^{14} M_sun, filamentary structure which includes more than 50% of the sources detected at z~2. An investigation of the properties of such sources indicates the possibility for boosted star-forming activity in those which reside within the overdense environment with respect of more isolated galaxies found in the same redshift range. Lastly, we also present our results on the evolution of the relationship between luminous and dark matter in star-forming galaxies between z~1 and z~2. We find that the increase of (average) stellar mass in galaxies between z~1 and z~2 is about a factor 10 lower than that of the dark matter haloes hosting such objects ([z~1]/[z~2] ~ 0.4 vs M_{halo}[z~1]/M_{halo}[z~2] ~ 0.04). Our findings agree with the evolutionary picture of downsizing whereby massive galaxies at z~2 were more actively forming stars than their z~1 counterparts, while at the same time contained a lower fraction of their mass in the form of luminous matter.Comment: 14 pages, 8 figures, MNRAS accepte

The relative abundance of compact and normal massive early-type galaxies and its evolution from redshift z~2 to the present

We report on the evolution of the number density and size of early-type galaxies from z~2 to z~0. We select a sample of 563 massive (M>10^{10} Msun), passively evolving (SSFR<10^{-2} Gyr^{-1}) and morphologically spheroidal galaxies at 0<z<2.5, using the panchromatic photometry and spectroscopic redshifts available in the GOODS fields. We combine ACS and WFC3 HST images to study the morphology of our galaxies in their optical rest-frame in the whole 0<z<2.5 range. We find that throughout the explored redshift range the passive galaxies selected with our criteria have weak morphological K-correction, with size being slightly smaller in the optical than in the UV rest-frame (by ~20 and ~10 at z>1.2 and z<1.2, respectively). We measure a significant evolution of the mass-size relation of early-type galaxies, with the fractional increment that is almost independent on the stellar mass. Early-type galaxies (ETGs) formed at z>1 appear to be preferentially small, and the evolution of the mass-size relation at z<1 is driven by both the continuous size growth of the compact galaxies and the appearance of new ETGs with large sizes. We also find that the number density of all passive early-type galaxies increases rapidly, by a factor of 5, from z~2 to z~1, and then more mildly by another factor of 1.5 from z~1 to z~0. We interpret these results as the evidence that the bulk of the ETGs are formed at 1<z<3 through a mechanism that leaves very compact remnants. At z<1 the compact ETGs grow gradually in size, becoming normal size galaxies, and at the same time new ETGs with normal-large sizes are formed.Comment: accepted for publication in Ap

Multi-objective optimization using Deep Gaussian Processes: Application to Aerospace Vehicle Design

International audienceThis paper is focused on the problem of constrained multi-objective design optimization of aerospace vehicles. The design of such vehicles often involves disciplinary legacy models considered as black-box and computationally expensive simulations characterized by a possible non-stationary behavior (an abrupt change in the response or a different smoothness along the design space). The expensive cost of an exact function evaluation makes the use of classical evolutionary multi-objective algorithms not tractable. While Bayesian Optimization based on Gaussian Process regression can handle the expensive cost of the evaluations, the non-stationary behavior of the functions can make it inefficient. A recent approach consisting of coupling Bayesian Optimization with Deep Gaussian Processes showed promising results for single-objective non-stationary problems. This paper presents an extension of this approach to the multi-objective context. The efficiency of the proposed approach is assessed with respect to classical optimization methods on an analytical test-case and on an aerospace design problem

H-alpha emitters in z~2 proto-clusters: evidence for faster evolution in dense environments

This is a study of H-alpha emitters in two dense galaxy proto-clusters surrounding radio galaxies at z~2. We show that the proto-cluster surrounding MRC 1138-262 contains 14+/-2 times more H-alpha candidates than the average field (9 sigma significance), and the z=2.35 radio galaxy 4C+10.48 is surrounded by 12+/-2 times more emitters than the field (5 sigma), so it is also likely to reside in a dense proto-cluster environment. We compared these H-alpha emitters, situated in dense environments, to a control field sample selected from 3 separate fields forming a total area of 172 arcmin^2. We constructed and compared H-alpha and rest-frame R continuum luminosity functions of the emitters in both environments. The star formation density is on average 13 times greater in the proto-clusters than the field at z~2, so the total star formation rate within the central 1.5Mpc of the proto-clusters exceeds 3000Msun/yr. However, we found no significant difference in the shape of the H-alpha luminosity functions, implying that environment does not substantially affect the strength of the H-alpha line from strongly star forming galaxies. The proto-cluster emitters are typically 0.8mag brighter in rest-frame R continuum than field emitters, implying they are twice as massive as their field counterparts at the same redshift. We also show the proto-cluster galaxies have lower specific star formation rates than field galaxies, meaning the emitters in the dense environments formed more of their stars earlier than the field galaxies. We conclude that galaxy growth in the early Universe was accelerated in dense environments, and that cluster galaxies differed from field galaxies even before the cluster had fully formed.Comment: Accepted for publication in MNRA