Nonlinear Independent Component Analysis for Principled Disentanglement in Unsupervised Deep Learning

A central problem in unsupervised deep learning is how to find useful representations of high-dimensional data, sometimes called "disentanglement". Most approaches are heuristic and lack a proper theoretical foundation. In linear representation learning, independent component analysis (ICA) has been successful in many applications areas, and it is principled, i.e. based on a well-defined probabilistic model. However, extension of ICA to the nonlinear case has been problematic due to the lack of identifiability, i.e. uniqueness of the representation. Recently, nonlinear extensions that utilize temporal structure or some auxiliary information have been proposed. Such models are in fact identifiable, and consequently, an increasing number of algorithms have been developed. In particular, some self-supervised algorithms can be shown to estimate nonlinear ICA, even though they have initially been proposed from heuristic perspectives. This paper reviews the state-of-the-art of nonlinear ICA theory and algorithms

Fundamental limits upon the measurement of state vectors

Using the Shannon information theory and the Bayesian methodology for inverting quantum data [K. R. W. Jones, Ann. Phys. (N.Y.) 207, 140 (1991)] we prove a fundamental bound upon the measurability of finite-dimensional quantum states. To do so we imagine a thought experiment for the quantum communication of a pure state , known to one experimenter, to his colleague via the transmission of N identical copies of it in the limit of zero temperature. Initial information available to the second experimenter is merely that of the allowed manifold of superpositions upon which the chosen may lie. Her efforts to determine it, in an optimal way, subject to the fundamental constraints imposed by quantum noise, define a statistical uncertainty principle. This limits the accuracy with which can be measured according to the number N of transmitted copies. The general result is illustrated in the physically realizable case of polarized photons

Robust Techniques for Signal Processing: A Survey

Coordinated Science Laboratory was formerly known as Control Systems LaboratoryU.S. Army Research Office / DAAG29-81-K-0062U.S. Air Force Office of Scientific Research / AFOSR 82-0022Joint Services Electronics Program / N00014-84-C-0149National Science Foundation / ECS-82-12080U.S. Office of Naval Research / N00014-80-K-0945 and N00014-81-K-001

SymmetryGAN: Symmetry Discovery with Deep Learning

What are the symmetries of a dataset? Whereas the symmetries of an individual data element can be characterized by its invariance under various transformations, the symmetries of an ensemble of data elements are ambiguous due to Jacobian factors introduced while changing coordinates. In this paper, we provide a rigorous statistical definition of the symmetries of a dataset, which involves inertial reference densities, in analogy to inertial frames in classical mechanics. We then propose SymmetryGAN as a novel and powerful approach to automatically discover symmetries using a deep learning method based on generative adversarial networks (GANs). When applied to Gaussian examples, SymmetryGAN shows excellent empirical performance, in agreement with expectations from the analytic loss landscape. SymmetryGAN is then applied to simulated dijet events from the Large Hadron Collider (LHC) to demonstrate the potential utility of this method in high energy collider physics applications. Going beyond symmetry discovery, we consider procedures to infer the underlying symmetry group from empirical data.Comment: 19 pages, 17 figure

Collaborative Information Processing in Wireless Sensor Networks for Diffusive Source Estimation

In this dissertation, we address the issue of collaborative information processing for diffusive source parameter estimation using wireless sensor networks (WSNs) capable of sensing in dispersive medium/environment, from signal processing perspective. We begin the dissertation by focusing on the mathematical formulation of a special diffusion phenomenon, i.e., an underwater oil spill, along with statistical algorithms for meaningful analysis of sensor data leading to efficient estimation of desired parameters of interest. The objective is to obtain an analytical solution to the problem, rather than using non-model based sophisticated numerical techniques. We tried to make the physical diffusion model as much appropriate as possible, while maintaining some pragmatic and reasonable assumptions for the simplicity of exposition and analytical derivation. The dissertation studies both source localization and tracking for static and moving diffusive sources respectively. For static diffusive source localization, we investigate two parametric estimation techniques based on the maximum-likelihood (ML) and the best linear unbiased estimator (BLUE) for a special case of our obtained physical dispersion model. We prove the consistency and asymptotic normality of the obtained ML solution when the number of sensor nodes and samples approach infinity, and derive the Cramer-Rao lower bound (CRLB) on its performance. In case of a moving diffusive source, we propose a particle filter (PF) based target tracking scheme for moving diffusive source, and analytically derive the posterior Cramer-Rao lower bound (PCRLB) for the moving source state estimates as a theoretical performance bound. Further, we explore nonparametric, machine learning based estimation technique for diffusive source parameter estimation using Dirichlet process mixture model (DPMM). Since real data are often complicated, no parametric model is suitable. As an alternative, we exploit the rich tools of nonparametric Bayesian methods, in particular the DPMM, which provides us with a flexible and data-driven estimation process. We propose DPMM based static diffusive source localization algorithm and provide analytical proof of convergence. The proposed algorithm is also extended to the scenario when multiple diffusive sources of same kind are present in the diffusive field of interest. Efficient power allocation can play an important role in extending the lifetime of a resource constrained WSN. Resource-constrained WSNs rely on collaborative signal and information processing for efficient handling of large volumes of data collected by the sensor nodes. In this dissertation, the problem of collaborative information processing for sequential parameter estimation in a WSN is formulated in a cooperative game-theoretic framework, which addresses the issue of fair resource allocation for estimation task at the Fusion center (FC). The framework allows addressing either resource allocation or commitment for information processing as solutions of cooperative games with underlying theoretical justifications. Different solution concepts found in cooperative games, namely, the Shapley function and Nash bargaining are used to enforce certain kinds of fairness among the nodes in a WSN

Modelling Locally Changing Variance Structured Time Series Data By Using Breakpoints Bootstrap Filtering

Stochastic processes have applications in many areas such as oceanography and engineering. Special classes of such processes deal with time series of sparse data. Studies in such cases focus in the analysis, construction and prediction in parametric models. Here, we assume several non-linear time series with additive noise components, and the model fitting is proposed in two stages. The first stage identifies the density using all the clusters information, without specifying any prior knowledge of the underlying distribution function of the time series. The effect of covariates is controlled by fitting the linear regression model with serially correlated errors. In the second stage, we partition the time series into consecutive non-overlapping intervals of quasi stationary increments where the coefficients shift from one stable regression relationship to a different one using a breakpoints detection algorithm. These breakpoints are estimated by minimizing the likelihood from the residuals. We approach time series prediction through the mixture distribution of combined error components. Parameter estimation of mixture distribution is done by using the EM algorithm. We apply the method to fish otolith data influenced by various environmental conditions and get estimation of parameters for the model

Untangling hotel industry’s inefficiency: An SFA approach applied to a renowned Portuguese hotel chain

The present paper explores the technical efficiency of four hotels from Teixeira Duarte Group - a renowned Portuguese hotel chain. An efficiency ranking is established from these four hotel units located in Portugal using Stochastic Frontier Analysis. This methodology allows to discriminate between measurement error and systematic inefficiencies in the estimation process enabling to investigate the main inefficiency causes. Several suggestions concerning efficiency improvement are undertaken for each hotel studied.info:eu-repo/semantics/publishedVersio