Robust detail-preserving signal extraction

We discuss robust filtering procedures for signal extraction from noisy time series. Particular attention is paid to the preservation of relevant signal details like abrupt shifts. moving averages and running medians are widely used but have shortcomings when large spikes (outliers) or trends occur. Modifications like modified trimmed means and linear median hybrid filters combine advantages of both approaches, but they do not completely overcome the difficulties. Better solutions can be based on robust regression techniques, which even work in real time because of increased computational power and faster algorithms. Reviewing previous work we present filters for robust signal extraction and discuss their merits for preserving trends, abrupt shifts and local extremes as well as for the removal of outliers. --

Computing the Least Quartile Difference Estimator in the Plane

A common problem in linear regression is that largely aberrant values can strongly influence the results. The least quartile difference (LQD) regression estimator is highly robust, since it can resist up to almost 50% largely deviant data values without becoming extremely biased. Additionally, it shows good behavior on Gaussian data – in contrast to many other robust regression methods. However, the LQD is not widely used yet due to the high computational effort needed when using common algorithms, e.g. the subset algorithm of Rousseeuw and Leroy. For computing the LQD estimator for n data points in the plane, we propose a randomized algorithm with expected running time O(n2 log2 n) and an approximation algorithm with a running time of roughly O(n2 log n). It can be expected that the practical relevance of the LQD estimator will strongly increase thereby. --

Dialogue Act Modeling for Automatic Tagging and Recognition of Conversational Speech

We describe a statistical approach for modeling dialogue acts in conversational speech, i.e., speech-act-like units such as Statement, Question, Backchannel, Agreement, Disagreement, and Apology. Our model detects and predicts dialogue acts based on lexical, collocational, and prosodic cues, as well as on the discourse coherence of the dialogue act sequence. The dialogue model is based on treating the discourse structure of a conversation as a hidden Markov model and the individual dialogue acts as observations emanating from the model states. Constraints on the likely sequence of dialogue acts are modeled via a dialogue act n-gram. The statistical dialogue grammar is combined with word n-grams, decision trees, and neural networks modeling the idiosyncratic lexical and prosodic manifestations of each dialogue act. We develop a probabilistic integration of speech recognition with dialogue modeling, to improve both speech recognition and dialogue act classification accuracy. Models are trained and evaluated using a large hand-labeled database of 1,155 conversations from the Switchboard corpus of spontaneous human-to-human telephone speech. We achieved good dialogue act labeling accuracy (65% based on errorful, automatically recognized words and prosody, and 71% based on word transcripts, compared to a chance baseline accuracy of 35% and human accuracy of 84%) and a small reduction in word recognition error.Comment: 35 pages, 5 figures. Changes in copy editing (note title spelling changed

Least mean M -estimate algorithms for robust adaptive filtering in impulse noise

This paper proposes two gradient-based adaptive algorithms, called the least mean M-estimate and the transform domain least mean M -estimate (TLMM) algorithms, for robust adaptive filtering in impulse noise. A robust M -estimator is used as the objective function to suppress the adverse effects of impulse noise on the filter weights. They have a computational complexity of order O(N) and can be viewed, respectively, as the generalization of the least mean square and the transform-domain least mean square algorithms. A robust method for estimating the required thresholds in the M -estimator is also given. Simulation results show that the TLMM algorithm, in particular, is more robust and effective than other commonly used algorithms in suppressing the adverse effects of the impulses. © 2000 IEEE.published_or_final_versio

Extension of Impulse Detectors to Spatial Dimension and their Utilization as Switch in the LMS L-SD Filter

In this paper, one kind of adaptive LMS filters based on order statistics is used for two-dimensional filtration of noisy greyscale images degraded by mixed noise. The signal-dependent adaptive LMS L-filter (L-SD) consists of two normalized constrained adaptive LMS L-filters, because they have better convergence properties than simple LMS algorithm. Moreover, first filter suppresses the noise in homogeneous regions and second filter preserves the high components of filtered image. Some versions of spatial order statistic detectors were developed from the impulse detectors and were employed as switch between output these filters

Rank discriminants for predicting phenotypes from RNA expression

Statistical methods for analyzing large-scale biomolecular data are commonplace in computational biology. A notable example is phenotype prediction from gene expression data, for instance, detecting human cancers, differentiating subtypes and predicting clinical outcomes. Still, clinical applications remain scarce. One reason is that the complexity of the decision rules that emerge from standard statistical learning impedes biological understanding, in particular, any mechanistic interpretation. Here we explore decision rules for binary classification utilizing only the ordering of expression among several genes; the basic building blocks are then two-gene expression comparisons. The simplest example, just one comparison, is the TSP classifier, which has appeared in a variety of cancer-related discovery studies. Decision rules based on multiple comparisons can better accommodate class heterogeneity, and thereby increase accuracy, and might provide a link with biological mechanism. We consider a general framework ("rank-in-context") for designing discriminant functions, including a data-driven selection of the number and identity of the genes in the support ("context"). We then specialize to two examples: voting among several pairs and comparing the median expression in two groups of genes. Comprehensive experiments assess accuracy relative to other, more complex, methods, and reinforce earlier observations that simple classifiers are competitive.Comment: Published in at http://dx.doi.org/10.1214/14-AOAS738 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org

Analysis of Multipath Mitigation Techniques with Land Mobile Satellite Channel Model

Multipath is undesirable for Global Navigation Satellite System (GNSS) receivers, since the reception of multipath can create a significant distortion to the shape of the correlation function leading to an error in the receivers’ position estimate. Many multipath mitigation techniques exist in the literature to deal with the multipath propagation problem in the context of GNSS. The multipath studies in the literature are often based on optimistic assumptions, for example, assuming a static two-path channel or a fading channel with a Rayleigh or a Nakagami distribution. But, in reality, there are a lot of channel modeling issues, for example, satellite-to-user geometry, variable number of paths, variable path delays and gains, Non Line-Of-Sight (NLOS) path condition, receiver movements, etc. that are kept out of consideration when analyzing the performance of these techniques. Therefore, this is of utmost importance to analyze the performance of different multipath mitigation techniques in some realistic measurement-based channel models, for example, the Land Multipath is undesirable for Global Navigation Satellite System (GNSS) receivers, since the reception of multipath can create a significant distortion to the shape of the correlation function leading to an error in the receivers’ position estimate. Many multipath mitigation techniques exist in the literature to deal with the multipath propagation problem in the context of GNSS. The multipath studies in the literature are often based on optimistic assumptions, for example, assuming a static two-path channel or a fading channel with a Rayleigh or a Nakagami distribution. But, in reality, there are a lot of channel modeling issues, for example, satellite-to-user geometry, variable number of paths, variable path delays and gains, Non Line-Of-Sight (NLOS) path condition, receiver movements, etc. that are kept out of consideration when analyzing the performance of these techniques. Therefore, this is of utmost importance to analyze the performance of different multipath mitigation techniques in some realistic measurement-based channel models, for example, the Land Mobile Satellite (LMS) channel model [1]-[4], developed at the German Aerospace Center (DLR). The DLR LMS channel model is widely used for simulating the positioning accuracy of mobile satellite navigation receivers in urban outdoor scenarios. The main objective of this paper is to present a comprehensive analysis of some of the most promising techniques with the DLR LMS channel model in varying multipath scenarios. Four multipath mitigation techniques are chosen herein for performance comparison, namely, the narrow Early-Minus-Late (nEML), the High Resolution Correlator, the C/N0-based two stage delay tracking technique, and the Reduced Search Space Maximum Likelihood (RSSML) delay estimator. The first two techniques are the most popular and traditional ones used in nowadays GNSS receivers, whereas the later two techniques are comparatively new and are advanced techniques, recently proposed by the authors. In addition, the implementation of the RSSML is optimized here for a narrow-bandwidth receiver configuration in the sense that it now requires a significantly less number of correlators and memory than its original implementation. The simulation results show that the reduced-complexity RSSML achieves the best multipath mitigation performance in moderate-to-good carrier-to-noise density ratio with the DLR LMS channel model in varying multipath scenarios