Discriminate-and-Rectify Encoders: Learning from Image Transformation Sets

The complexity of a learning task is increased by transformations in the input space that preserve class identity. Visual object recognition for example is affected by changes in viewpoint, scale, illumination or planar transformations. While drastically altering the visual appearance, these changes are orthogonal to recognition and should not be reflected in the representation or feature encoding used for learning. We introduce a framework for weakly supervised learning of image embeddings that are robust to transformations and selective to the class distribution, using sets of transforming examples (orbit sets), deep parametrizations and a novel orbit-based loss. The proposed loss combines a discriminative, contrastive part for orbits with a reconstruction error that learns to rectify orbit transformations. The learned embeddings are evaluated in distance metric-based tasks, such as one-shot classification under geometric transformations, as well as face verification and retrieval under more realistic visual variability. Our results suggest that orbit sets, suitably computed or observed, can be used for efficient, weakly-supervised learning of semantically relevant image embeddings.This material is based upon work supported by the Center for Brains, Minds and Machines (CBMM), funded by NSF STC award CCF-1231216

A new code for orbit analysis and Schwarzschild modelling of triaxial stellar systems

We review the methods used to study the orbital structure and chaotic properties of various galactic models and to construct self-consistent equilibrium solutions by Schwarzschild's orbit superposition technique. These methods are implemented in a new publicly available software tool, SMILE, which is intended to be a convenient and interactive instrument for studying a variety of 2D and 3D models, including arbitrary potentials represented by a basis-set expansion, a spherical-harmonic expansion with coefficients being smooth functions of radius (splines), or a set of fixed point masses. We also propose two new variants of Schwarzschild modelling, in which the density of each orbit is represented by the coefficients of the basis-set or spline spherical-harmonic expansion, and the orbit weights are assigned in such a way as to reproduce the coefficients of the underlying density model. We explore the accuracy of these general-purpose potential expansions and show that they may be efficiently used to approximate a wide range of analytic density models and serve as smooth representations of discrete particle sets (e.g. snapshots from an N-body simulation), for instance, for the purpose of orbit analysis of the snapshot. For the variants of Schwarzschild modelling, we use two test cases - a triaxial Dehnen model containing a central black hole, and a model re-created from an N-body snapshot obtained by a cold collapse. These tests demonstrate that all modelling approaches are capable of creating equilibrium models.Comment: MNRAS, 24 pages, 18 figures. Software is available at http://td.lpi.ru/~eugvas/smile

A Foliated View of Transfer Learning

Transfer learning considers a learning process where a new task is solved by transferring relevant knowledge from known solutions to related tasks. While this has been studied experimentally, there lacks a foundational description of the transfer learning problem that exposes what related tasks are, and how they can be exploited. In this work, we present a definition for relatedness between tasks and identify foliations as a mathematical framework to represent such relationships.Comment: 14 pages, 6 figure

SAR-NeRF: Neural Radiance Fields for Synthetic Aperture Radar Multi-View Representation

SAR images are highly sensitive to observation configurations, and they exhibit significant variations across different viewing angles, making it challenging to represent and learn their anisotropic features. As a result, deep learning methods often generalize poorly across different view angles. Inspired by the concept of neural radiance fields (NeRF), this study combines SAR imaging mechanisms with neural networks to propose a novel NeRF model for SAR image generation. Following the mapping and projection pinciples, a set of SAR images is modeled implicitly as a function of attenuation coefficients and scattering intensities in the 3D imaging space through a differentiable rendering equation. SAR-NeRF is then constructed to learn the distribution of attenuation coefficients and scattering intensities of voxels, where the vectorized form of 3D voxel SAR rendering equation and the sampling relationship between the 3D space voxels and the 2D view ray grids are analytically derived. Through quantitative experiments on various datasets, we thoroughly assess the multi-view representation and generalization capabilities of SAR-NeRF. Additionally, it is found that SAR-NeRF augumented dataset can significantly improve SAR target classification performance under few-shot learning setup, where a 10-type classification accuracy of 91.6\% can be achieved by using only 12 images per class