Conditional Sum-Product Networks: Imposing Structure on Deep Probabilistic Architectures

Probabilistic graphical models are a central tool in AI; however, they are generally not as expressive as deep neural models, and inference is notoriously hard and slow. In contrast, deep probabilistic models such as sum-product networks (SPNs) capture joint distributions in a tractable fashion, but still lack the expressive power of intractable models based on deep neural networks. Therefore, we introduce conditional SPNs (CSPNs), conditional density estimators for multivariate and potentially hybrid domains which allow harnessing the expressive power of neural networks while still maintaining tractability guarantees. One way to implement CSPNs is to use an existing SPN structure and condition its parameters on the input, e.g., via a deep neural network. This approach, however, might misrepresent the conditional independence structure present in data. Consequently, we also develop a structure-learning approach that derives both the structure and parameters of CSPNs from data. Our experimental evidence demonstrates that CSPNs are competitive with other probabilistic models and yield superior performance on multilabel image classification compared to mean field and mixture density networks. Furthermore, they can successfully be employed as building blocks for structured probabilistic models, such as autoregressive image models.Comment: 13 pages, 6 figure

Evaluation of bistable systems versus matched filters in detecting bipolar pulse signals

This paper presents a thorough evaluation of a bistable system versus a matched filter in detecting bipolar pulse signals. The detectability of the bistable system can be optimized by adding noise, i.e. the stochastic resonance (SR) phenomenon. This SR effect is also demonstrated by approximate statistical detection theory of the bistable system and corresponding numerical simulations. Furthermore, the performance comparison results between the bistable system and the matched filter show that (a) the bistable system is more robust than the matched filter in detecting signals with disturbed pulse rates, and (b) the bistable system approaches the performance of the matched filter in detecting unknown arrival times of received signals, with an especially better computational efficiency. These significant results verify the potential applicability of the bistable system in signal detection field.Comment: 15 pages, 9 figures, MikTex v2.

On the Estimation of Nonrandom Signal Coefficients from Jittered Samples

This paper examines the problem of estimating the parameters of a bandlimited signal from samples corrupted by random jitter (timing noise) and additive iid Gaussian noise, where the signal lies in the span of a finite basis. For the presented classical estimation problem, the Cramer-Rao lower bound (CRB) is computed, and an Expectation-Maximization (EM) algorithm approximating the maximum likelihood (ML) estimator is developed. Simulations are performed to study the convergence properties of the EM algorithm and compare the performance both against the CRB and a basic linear estimator. These simulations demonstrate that by post-processing the jittered samples with the proposed EM algorithm, greater jitter can be tolerated, potentially reducing on-chip ADC power consumption substantially.Comment: 11 pages, 8 figure

Target Tracking in Non-Gaussian Environment

Masreliez filter which is a Kalman type of recursive filter is implemented and validated. The main computation in Masreliez filter is to evaluate the score function which directly influences the estimates of the target states. Scalar approximation for score function evaluation is extended to vector observations, implemented and validated. The simulation studies have shown that the performance of the Masreliez filter is relatively better than that of the conventional Kalman filter in the presence of significant glint noise in the observation