Geometric deep learning: going beyond Euclidean data

Many scientific fields study data with an underlying structure that is a non-Euclidean space. Some examples include social networks in computational social sciences, sensor networks in communications, functional networks in brain imaging, regulatory networks in genetics, and meshed surfaces in computer graphics. In many applications, such geometric data are large and complex (in the case of social networks, on the scale of billions), and are natural targets for machine learning techniques. In particular, we would like to use deep neural networks, which have recently proven to be powerful tools for a broad range of problems from computer vision, natural language processing, and audio analysis. However, these tools have been most successful on data with an underlying Euclidean or grid-like structure, and in cases where the invariances of these structures are built into networks used to model them. Geometric deep learning is an umbrella term for emerging techniques attempting to generalize (structured) deep neural models to non-Euclidean domains such as graphs and manifolds. The purpose of this paper is to overview different examples of geometric deep learning problems and present available solutions, key difficulties, applications, and future research directions in this nascent field

TextureGAN: Controlling Deep Image Synthesis with Texture Patches

In this paper, we investigate deep image synthesis guided by sketch, color, and texture. Previous image synthesis methods can be controlled by sketch and color strokes but we are the first to examine texture control. We allow a user to place a texture patch on a sketch at arbitrary locations and scales to control the desired output texture. Our generative network learns to synthesize objects consistent with these texture suggestions. To achieve this, we develop a local texture loss in addition to adversarial and content loss to train the generative network. We conduct experiments using sketches generated from real images and textures sampled from a separate texture database and results show that our proposed algorithm is able to generate plausible images that are faithful to user controls. Ablation studies show that our proposed pipeline can generate more realistic images than adapting existing methods directly.Comment: CVPR 2018 spotligh

Medical image synthesis using generative adversarial networks: towards photo-realistic image synthesis

This proposed work addresses the photo-realism for synthetic images. We introduced a modified generative adversarial network: StencilGAN. It is a perceptually-aware generative adversarial network that synthesizes images based on overlaid labelled masks. This technique can be a prominent solution for the scarcity of the resources in the healthcare sector

Towards Precision LSST Weak-Lensing Measurement - I: Impacts of Atmospheric Turbulence and Optical Aberration

The weak-lensing science of the LSST project drives the need to carefully model and separate the instrumental artifacts from the intrinsic lensing signal. The dominant source of the systematics for all ground based telescopes is the spatial correlation of the PSF modulated by both atmospheric turbulence and optical aberrations. In this paper, we present a full FOV simulation of the LSST images by modeling both the atmosphere and the telescope optics with the most current data for the telescope specifications and the environment. To simulate the effects of atmospheric turbulence, we generated six-layer phase screens with the parameters estimated from the on-site measurements. For the optics, we combined the ray-tracing tool ZEMAX and our simulated focal plane data to introduce realistic aberrations and focal plane height fluctuations. Although this expected flatness deviation for LSST is small compared with that of other existing cameras, the fast f-ratio of the LSST optics makes this focal plane flatness variation and the resulting PSF discontinuities across the CCD boundaries significant challenges in our removal of the systematics. We resolve this complication by performing PCA CCD-by-CCD, and interpolating the basis functions using conventional polynomials. We demonstrate that this PSF correction scheme reduces the residual PSF ellipticity correlation below 10^-7 over the cosmologically interesting scale. From a null test using HST/UDF galaxy images without input shear, we verify that the amplitude of the galaxy ellipticity correlation function, after the PSF correction, is consistent with the shot noise set by the finite number of objects. Therefore, we conclude that the current optical design and specification for the accuracy in the focal plane assembly are sufficient to enable the control of the PSF systematics required for weak-lensing science with the LSST.Comment: Accepted to PASP. High-resolution version is available at http://dls.physics.ucdavis.edu/~mkjee/LSST_weak_lensing_simulation.pd

CayleyNets: Graph Convolutional Neural Networks with Complex Rational Spectral Filters

The rise of graph-structured data such as social networks, regulatory networks, citation graphs, and functional brain networks, in combination with resounding success of deep learning in various applications, has brought the interest in generalizing deep learning models to non-Euclidean domains. In this paper, we introduce a new spectral domain convolutional architecture for deep learning on graphs. The core ingredient of our model is a new class of parametric rational complex functions (Cayley polynomials) allowing to efficiently compute spectral filters on graphs that specialize on frequency bands of interest. Our model generates rich spectral filters that are localized in space, scales linearly with the size of the input data for sparsely-connected graphs, and can handle different constructions of Laplacian operators. Extensive experimental results show the superior performance of our approach, in comparison to other spectral domain convolutional architectures, on spectral image classification, community detection, vertex classification and matrix completion tasks