Finding Non-overlapping Clusters for Generalized Inference Over Graphical Models

Graphical models use graphs to compactly capture stochastic dependencies amongst a collection of random variables. Inference over graphical models corresponds to finding marginal probability distributions given joint probability distributions. In general, this is computationally intractable, which has led to a quest for finding efficient approximate inference algorithms. We propose a framework for generalized inference over graphical models that can be used as a wrapper for improving the estimates of approximate inference algorithms. Instead of applying an inference algorithm to the original graph, we apply the inference algorithm to a block-graph, defined as a graph in which the nodes are non-overlapping clusters of nodes from the original graph. This results in marginal estimates of a cluster of nodes, which we further marginalize to get the marginal estimates of each node. Our proposed block-graph construction algorithm is simple, efficient, and motivated by the observation that approximate inference is more accurate on graphs with longer cycles. We present extensive numerical simulations that illustrate our block-graph framework with a variety of inference algorithms (e.g., those in the libDAI software package). These simulations show the improvements provided by our framework.Comment: Extended the previous version to include extensive numerical simulations. See http://www.ima.umn.edu/~dvats/GeneralizedInference.html for code and dat

Short Packets over Block-Memoryless Fading Channels: Pilot-Assisted or Noncoherent Transmission?

We present nonasymptotic upper and lower bounds on the maximum coding rate achievable when transmitting short packets over a Rician memoryless block-fading channel for a given requirement on the packet error probability. We focus on the practically relevant scenario in which there is no \emph{a priori} channel state information available at the transmitter and at the receiver. An upper bound built upon the min-max converse is compared to two lower bounds: the first one relies on a noncoherent transmission strategy in which the fading channel is not estimated explicitly at the receiver; the second one employs pilot-assisted transmission (PAT) followed by maximum-likelihood channel estimation and scaled mismatched nearest-neighbor decoding at the receiver. Our bounds are tight enough to unveil the optimum number of diversity branches that a packet should span so that the energy per bit required to achieve a target packet error probability is minimized, for a given constraint on the code rate and the packet size. Furthermore, the bounds reveal that noncoherent transmission is more energy efficient than PAT, even when the number of pilot symbols and their power is optimized. For example, for the case when a coded packet of $168$ symbols is transmitted using a channel code of rate $0.48$ bits/channel use, over a block-fading channel with block size equal to $8$ symbols, PAT requires an additional $1.2$ dB of energy per information bit to achieve a packet error probability of $10^{-3}$ compared to a suitably designed noncoherent transmission scheme. Finally, we devise a PAT scheme based on punctured tail-biting quasi-cyclic codes and ordered statistics decoding, whose performance are close ($1$ dB gap at $10^{-3}$ packet error probability) to the ones predicted by our PAT lower bound. This shows that the PAT lower bound provides useful guidelines on the design of actual PAT schemes.Comment: 30 pages, 5 figures, journa

Fast linear algebra is stable

In an earlier paper, we showed that a large class of fast recursive matrix multiplication algorithms is stable in a normwise sense, and that in fact if multiplication of $n$-by-$n$ matrices can be done by any algorithm in $O(n^{\omega + \eta})$ operations for any $\eta > 0$, then it can be done stably in $O(n^{\omega + \eta})$ operations for any $\eta > 0$. Here we extend this result to show that essentially all standard linear algebra operations, including LU decomposition, QR decomposition, linear equation solving, matrix inversion, solving least squares problems, (generalized) eigenvalue problems and the singular value decomposition can also be done stably (in a normwise sense) in $O(n^{\omega + \eta})$ operations.Comment: 26 pages; final version; to appear in Numerische Mathemati

One- and two-level filter-bank convolvers

In a recent paper, it was shown in detail that in the case of orthonormal and biorthogonal filter banks we can convolve two signals by directly convolving the subband signals and combining the results. In this paper, we further generalize the result. We also derive the statistical coding gain for the generalized subband convolver. As an application, we derive a novel low sensitivity structure for FIR filters from the convolution theorem. We define and derive a deterministic coding gain of the subband convolver over direct convolution for a fixed wordlength implementation. This gain serves as a figure of merit for the low sensitivity structure. Several numerical examples are included to demonstrate the usefulness of these ideas. By using the generalized polyphase representation, we show that the subband convolvers, linear periodically time varying systems, and digital block filtering can be viewed in a unified manner. Furthermore, the scheme called IFIR filtering is shown to be a special case of the convolver

Generalized feedback detection for spatial multiplexing multi-antenna systems

We present a unified detection framework for spatial multiplexing multiple-input multiple-output (MIMO) systems by generalizing Heller’s classical feedback decoding algorithm for convolutional codes. The resulting generalized feedback detector (GFD) is characterized by three parameters: window size, step size and branch factor. Many existing MIMO detectors are turned out to be special cases of the GFD. Moreover, different parameter choices can provide various performance-complexity tradeoffs. The connection between MIMO detectors and tree search algorithms is also established. To reduce redundant computations in the GFD, a shared computation technique is proposed by using a tree data structure. Using a union bound based analysis of the symbol error rates, the diversity order and signal-to-noise ratio (SNR) gain are derived analytically as functions of the three parameters; for example, the diversity order of the GFD varies between 1 and N. The complexity of the GFD varies between those of the maximum-likelihood (ML) detector and the zero-forcing decision feedback detector (ZFDFD). Extensive computer simulation results are also provided