EFEM: Equivariant Neural Field Expectation Maximization for 3D Object Segmentation Without Scene Supervision

We introduce Equivariant Neural Field Expectation Maximization (EFEM), a simple, effective, and robust geometric algorithm that can segment objects in 3D scenes without annotations or training on scenes. We achieve such unsupervised segmentation by exploiting single object shape priors. We make two novel steps in that direction. First, we introduce equivariant shape representations to this problem to eliminate the complexity induced by the variation in object configuration. Second, we propose a novel EM algorithm that can iteratively refine segmentation masks using the equivariant shape prior. We collect a novel real dataset Chairs and Mugs that contains various object configurations and novel scenes in order to verify the effectiveness and robustness of our method. Experimental results demonstrate that our method achieves consistent and robust performance across different scenes where the (weakly) supervised methods may fail. Code and data available at https://www.cis.upenn.edu/~leijh/projects/efemComment: Accepted by CVPR2023, project page https://www.cis.upenn.edu/~leijh/projects/efe

EVALUATING MONOCULAR DEPTH ESTIMATION METHODS

Depth estimation from monocular images has become a prominent focus in photogrammetry and computer vision research. Monocular Depth Estimation (MDE), which involves determining depth from a single RGB image, offers numerous advantages, including applications in simultaneous localization and mapping (SLAM), scene comprehension, 3D modeling, robotics, and autonomous driving. Depth information retrieval becomes especially crucial in situations where other sources like stereo images, optical flow, or point clouds are not available. In contrast to traditional stereo or multi-view methods, MDE techniques require fewer computational resources and smaller datasets. This research work presents a comprehensive analysis and evaluation of some state-of-the-art MDE methods, considering their ability to infer depth information in terrestrial images. The evaluation includes quantitative assessments using ground truth data, including 3D analyses and inference time

POINT SPREAD FUNCTION ESTIMATION AND UNCERTAINTY QUANTIFICATION

An important component of analyzing images quantitatively is modeling image blur due to eects from the system for image capture. When the eect of image blur is assumed to be translation invariant and isotropic, it can be generally modeled as convolution with a radially symmetric kernel, called the point spread function (PSF). Standard techniques for estimating the PSF involve imaging a bright point source, but this is not always feasible (e.g. high energy radiography). This work provides a novel non-parametric approach to estimating the PSF from a calibration image of a vertical edge. Moreover, the approach is within a hierarchical Bayesian framework that in addition to providing a method for estimation, also gives a quantification of uncertainty in the estimate by Markov Chain Monte Carlo (MCMC) methods. In the development, we employ a recently developed enhancement to Gibbs sampling, referred to as partial collapse. The improved algorithm has been independently derived in several other works, however, it has been shown that partial collapse may be improperly implemented resulting in a sampling algorithm that that no longer converges to the desired posterior. The algorithm we present is proven to satisfy invariance with respect to the target density. This work and its implementation on radiographic data from the U.S. Department of Energy\u27s Cygnus high-energy X-ray diagnostic system have culminated in a paper titled \Partially Collapsed Gibbs Samplers for Linear Inverse Problems and Applications to X-ray Imaging. The other component of this work is mainly theoretical and develops the requisite functional analysis to make the integration based model derived in the first chapter rigorous. The literature source is from functional analysis related to distribution theory for linear partial differential equations, and briefly addresses infinite dimensional probability theory for Hilbert space-valued stochastic processes, a burgeoning and very active research area for the analysis of inverse problems. To our knowledge, this provides a new development of a notion of radial symmetry for L2 based distributions. This work results in defining an L2 complete space of radially symmetric distributions, which is an important step toward rigorously placing the PSF estimation problem in the infinite dimensional framework and is part of ongoing work toward that end

HALSIE - Hybrid Approach to Learning Segmentation by Simultaneously Exploiting Image and Event Modalities

Standard frame-based algorithms fail to retrieve accurate segmentation maps in challenging real-time applications like autonomous navigation, owing to the limited dynamic range and motion blur prevalent in traditional cameras. Event cameras address these limitations by asynchronously detecting changes in per-pixel intensity to generate event streams with high temporal resolution, high dynamic range, and no motion blur. However, event camera outputs cannot be directly used to generate reliable segmentation maps as they only capture information at the pixels in motion. To augment the missing contextual information, we postulate that fusing spatially dense frames with temporally dense events can generate semantic maps with fine-grained predictions. To this end, we propose HALSIE, a hybrid approach to learning segmentation by simultaneously leveraging image and event modalities. To enable efficient learning across modalities, our proposed hybrid framework comprises two input branches, a Spiking Neural Network (SNN) branch and a standard Artificial Neural Network (ANN) branch to process event and frame data respectively, while exploiting their corresponding neural dynamics. Our hybrid network outperforms the state-of-the-art semantic segmentation benchmarks on DDD17 and MVSEC datasets and shows comparable performance on the DSEC-Semantic dataset with upto 33.23$\times$ reduction in network parameters. Further, our method shows upto 18.92$\times$ improvement in inference cost compared to existing SOTA approaches, making it suitable for resource-constrained edge applications

Trustworthy Representation Learning Across Domains

As AI systems have obtained significant performance to be deployed widely in our daily live and human society, people both enjoy the benefits brought by these technologies and suffer many social issues induced by these systems. To make AI systems good enough and trustworthy, plenty of researches have been done to build guidelines for trustworthy AI systems. Machine learning is one of the most important parts for AI systems and representation learning is the fundamental technology in machine learning. How to make the representation learning trustworthy in real-world application, e.g., cross domain scenarios, is very valuable and necessary for both machine learning and AI system fields. Inspired by the concepts in trustworthy AI, we proposed the first trustworthy representation learning across domains framework which includes four concepts, i.e, robustness, privacy, fairness, and explainability, to give a comprehensive literature review on this research direction. Specifically, we first introduce the details of the proposed trustworthy framework for representation learning across domains. Second, we provide basic notions and comprehensively summarize existing methods for the trustworthy framework from four concepts. Finally, we conclude this survey with insights and discussions on future research directions.Comment: 38 pages, 15 figure

Scaling behaviour of braided active channels: a Taylor’s power law approach

none9At a channel (reach) scale, braided channels are fluvial, geomorphological, complex systems that are characterized by a shift of bars during flood events. In such events water flows are channeled in multiple and mobile channels across a gravel floodplain that remain in unmodified conditions. From a geometrical point of view, braided patterns of the active hydraulic channels are characterized by multicursal nature with structures that are spatially developed by either simple- and multi-scaling behavior. Since current studies do not take into account a general procedure concerning scale measurements, the latter behavior is still not well understood. The aim of our investigation is to analyze directly, through a general procedure, the scaling behavior of hydraulically active channels per transect and per reach analyzed. Our generalized stochastic approach is based on Taylor’s law, and the theory of exponential dispersion distributions. In particular, we make use of a power law, based on the variance and mean of the active channel fluctuations. In this way we demonstrate that the number of such fluctuations with respect to the unicursal behavior of the braided rivers, follows a jump-process of Poisson and compound Poisson–Gamma distributions. Furthermore, a correlation is also provided between the scaling fractal exponents obtained by Taylor’s law and the Hurst exponents.Samuele De Bartolo, Stefano Rizzello, Ennio Ferrari, Ferdinando Frega, Gaetano Napoli, Raffaele Vitolo, Michele Scaraggi, Carmine Fallico, Gerardo SeverinoDE BARTOLO, Samuele; Rizzello, Stefano; Ferrari, Ennio; Frega, Ferdinando; Napoli, Gaetano; Vitolo, Raffaele; Scaraggi, Michele; Fallico, Carmine; Severino, Gerard

Scaling behaviour of braided active channels: a Taylor’s power law approach

none9At a channel (reach) scale, braided channels are fluvial, geomorphological, complex systems that are characterized by a shift of bars during flood events. In such events water flows are channeled in multiple and mobile channels across a gravel floodplain that remain in unmodified conditions. From a geometrical point of view, braided patterns of the active hydraulic channels are characterized by multicursal nature with structures that are spatially developed by either simple- and multi-scaling behavior. Since current studies do not take into account a general procedure concerning scale measurements, the latter behavior is still not well understood. The aim of our investigation is to analyze directly, through a general procedure, the scaling behavior of hydraulically active channels per transect and per reach analyzed. Our generalized stochastic approach is based on Taylor’s law, and the theory of exponential dispersion distributions. In particular, we make use of a power law, based on the variance and mean of the active channel fluctuations. In this way we demonstrate that the number of such fluctuations with respect to the unicursal behavior of the braided rivers, follows a jump-process of Poisson and compound Poisson–Gamma distributions. Furthermore, a correlation is also provided between the scaling fractal exponents obtained by Taylor’s law and the Hurst exponents.Samuele De Bartolo, Stefano Rizzello, Ennio Ferrari, Ferdinando Frega, Gaetano Napoli, Raffaele Vitolo, Michele Scaraggi, Carmine Fallico, Gerardo SeverinoDE BARTOLO, Samuele; Rizzello, Stefano; Ferrari, Ennio; Frega, Ferdinando; Napoli, Gaetano; Vitolo, Raffaele; Scaraggi, Michele; Fallico, Carmine; Severino, Gerard

The Crystal Collimation System of the Relativistic Heavy Ion Collider

N/

Sampling theory in shift-invariant spaces: generalizations

Roughly speaking sampling theory deals with determining whether we can or can not recover a continuous function from some discrete set of its values. The most important result and main pillar of this theory is the well-known Shannon’s sampling theorem which states that: If a signal f(t) contains no frequencies higher than 1/2 cycles per second, it is completely determined by giving its ordinates at a sequence of points spaced one second apart….A grandes rasgos la teoría de muestreo estudia el problema de recuperar una función continua a partir de un conjunto discreto de sus valores. El resultado más importante y pilar fundamental de esta teoría es el conocido teorema de muestreo de Shannon que afirma que: Si una señal f(t) no contiene frecuencias mayores que 1/2 ciclos por segundo entonces está completamente determinada por sus ordenadas en una sucesión de puntos espaciados en un segundo….Proyecto de investigación MTM2009–08345 del Ministerio de Ciencia e Innovación de España.Programa Oficial de Doctorado en Ingeniería MatemáticaPresidente: Luis Alberto Ibort Latre.- Secretario: Eugenio Hernández Rodríguez.- Vocal: Ole Christense