Linear and Order Statistics Combiners for Pattern Classification

Several researchers have experimentally shown that substantial improvements can be obtained in difficult pattern recognition problems by combining or integrating the outputs of multiple classifiers. This chapter provides an analytical framework to quantify the improvements in classification results due to combining. The results apply to both linear combiners and order statistics combiners. We first show that to a first order approximation, the error rate obtained over and above the Bayes error rate, is directly proportional to the variance of the actual decision boundaries around the Bayes optimum boundary. Combining classifiers in output space reduces this variance, and hence reduces the "added" error. If N unbiased classifiers are combined by simple averaging, the added error rate can be reduced by a factor of N if the individual errors in approximating the decision boundaries are uncorrelated. Expressions are then derived for linear combiners which are biased or correlated, and the effect of output correlations on ensemble performance is quantified. For order statistics based non-linear combiners, we derive expressions that indicate how much the median, the maximum and in general the ith order statistic can improve classifier performance. The analysis presented here facilitates the understanding of the relationships among error rates, classifier boundary distributions, and combining in output space. Experimental results on several public domain data sets are provided to illustrate the benefits of combining and to support the analytical results.Comment: 31 page

Automatic Quality Estimation for ASR System Combination

Recognizer Output Voting Error Reduction (ROVER) has been widely used for system combination in automatic speech recognition (ASR). In order to select the most appropriate words to insert at each position in the output transcriptions, some ROVER extensions rely on critical information such as confidence scores and other ASR decoder features. This information, which is not always available, highly depends on the decoding process and sometimes tends to over estimate the real quality of the recognized words. In this paper we propose a novel variant of ROVER that takes advantage of ASR quality estimation (QE) for ranking the transcriptions at "segment level" instead of: i) relying on confidence scores, or ii) feeding ROVER with randomly ordered hypotheses. We first introduce an effective set of features to compensate for the absence of ASR decoder information. Then, we apply QE techniques to perform accurate hypothesis ranking at segment-level before starting the fusion process. The evaluation is carried out on two different tasks, in which we respectively combine hypotheses coming from independent ASR systems and multi-microphone recordings. In both tasks, it is assumed that the ASR decoder information is not available. The proposed approach significantly outperforms standard ROVER and it is competitive with two strong oracles that e xploit prior knowledge about the real quality of the hypotheses to be combined. Compared to standard ROVER, the abs olute WER improvements in the two evaluation scenarios range from 0.5% to 7.3%

Supervised Classification: Quite a Brief Overview

The original problem of supervised classification considers the task of automatically assigning objects to their respective classes on the basis of numerical measurements derived from these objects. Classifiers are the tools that implement the actual functional mapping from these measurements---also called features or inputs---to the so-called class label---or output. The fields of pattern recognition and machine learning study ways of constructing such classifiers. The main idea behind supervised methods is that of learning from examples: given a number of example input-output relations, to what extent can the general mapping be learned that takes any new and unseen feature vector to its correct class? This chapter provides a basic introduction to the underlying ideas of how to come to a supervised classification problem. In addition, it provides an overview of some specific classification techniques, delves into the issues of object representation and classifier evaluation, and (very) briefly covers some variations on the basic supervised classification task that may also be of interest to the practitioner

Multiuser MIMO-OFDM for Next-Generation Wireless Systems

This overview portrays the 40-year evolution of orthogonal frequency division multiplexing (OFDM) research. The amelioration of powerful multicarrier OFDM arrangements with multiple-input multiple-output (MIMO) systems has numerous benefits, which are detailed in this treatise. We continue by highlighting the limitations of conventional detection and channel estimation techniques designed for multiuser MIMO OFDM systems in the so-called rank-deficient scenarios, where the number of users supported or the number of transmit antennas employed exceeds the number of receiver antennas. This is often encountered in practice, unless we limit the number of users granted access in the base station’s or radio port’s coverage area. Following a historical perspective on the associated design problems and their state-of-the-art solutions, the second half of this treatise details a range of classic multiuser detectors (MUDs) designed for MIMO-OFDM systems and characterizes their achievable performance. A further section aims for identifying novel cutting-edge genetic algorithm (GA)-aided detector solutions, which have found numerous applications in wireless communications in recent years. In an effort to stimulate the cross pollination of ideas across the machine learning, optimization, signal processing, and wireless communications research communities, we will review the broadly applicable principles of various GA-assisted optimization techniques, which were recently proposed also for employment inmultiuser MIMO OFDM. In order to stimulate new research, we demonstrate that the family of GA-aided MUDs is capable of achieving a near-optimum performance at the cost of a significantly lower computational complexity than that imposed by their optimum maximum-likelihood (ML) MUD aided counterparts. The paper is concluded by outlining a range of future research options that may find their way into next-generation wireless systems

Exact MIMO Zero-Forcing Detection Analysis for Transmit-Correlated Rician Fading

We analyze the performance of multiple input/multiple output (MIMO) communications systems employing spatial multiplexing and zero-forcing detection (ZF). The distribution of the ZF signal-to-noise ratio (SNR) is characterized when either the intended stream or interfering streams experience Rician fading, and when the fading may be correlated on the transmit side. Previously, exact ZF analysis based on a well-known SNR expression has been hindered by the noncentrality of the Wishart distribution involved. In addition, approximation with a central-Wishart distribution has not proved consistently accurate. In contrast, the following exact ZF study proceeds from a lesser-known SNR expression that separates the intended and interfering channel-gain vectors. By first conditioning on, and then averaging over the interference, the ZF SNR distribution for Rician-Rayleigh fading is shown to be an infinite linear combination of gamma distributions. On the other hand, for Rayleigh-Rician fading, the ZF SNR is shown to be gamma-distributed. Based on the SNR distribution, we derive new series expressions for the ZF average error probability, outage probability, and ergodic capacity. Numerical results confirm the accuracy of our new expressions, and reveal effects of interference and channel statistics on performance.Comment: 14 pages, two-colum, 1 table, 10 figure

Performance Analysis of Dual-User Macrodiversity MIMO Systems with Linear Receivers in Flat Rayleigh Fading

The performance of linear receivers in the presence of co-channel interference in Rayleigh channels is a fundamental problem in wireless communications. Performance evaluation for these systems is well-known for receive arrays where the antennas are close enough to experience equal average SNRs from a source. In contrast, almost no analytical results are available for macrodiversity systems where both the sources and receive antennas are widely separated. Here, receive antennas experience unequal average SNRs from a source and a single receive antenna receives a different average SNR from each source. Although this is an extremely difficult problem, progress is possible for the two-user scenario. In this paper, we derive closed form results for the probability density function (pdf) and cumulative distribution function (cdf) of the output signal to interference plus noise ratio (SINR) and signal to noise ratio (SNR) of minimum mean squared error (MMSE) and zero forcing (ZF) receivers in independent Rayleigh channels with arbitrary numbers of receive antennas. The results are verified by Monte Carlo simulations and high SNR approximations are also derived. The results enable further system analysis such as the evaluation of outage probability, bit error rate (BER) and capacity.Comment: 24 pages, 7 figures; IEEE Transaction of Wireless Communication 2012 Corrected typo