Reduced complexity on-line estimation of hidden Markov model parameters

In this paper we propose and study low complexity algorithms for on-line estimation of hidden Markov model (HMM) parameters. The estimates approach the true model parameters as the measurement noise approaches zero, but otherwise give improved estimates, albeit with bias. On a nite data set in the high noise case, the bias may not be signi cantly more severe than for a higher complexity asymptotically optimal scheme. Our algorithms require O(N3) calculations per time instant, where N is the number of states. Previous algorithms based on earlier hidden Markov model signal processing methods, including the expectation-maximumisation (EM) algorithm require O(N4) calculations per time instant

Online topology free Gaussian HMM parameter estimation based on clustering

Tese de Mestrado Integrado. Engenharia Informática e Computação. Faculdade de Engenharia. Universidade do Porto. 201

An investigation of multi-dimensional evolutionary algorithms for virtual reality scenario development

Virtual reality (VR) has emerged as a powerful visualization tool for design, simulation, and analysis in modem complex industrial systems. The primary motivation for this thesis is to develop a framework for the effective use of VR in design-simulation-analysis cycles, particularly in situations involving large, complex, multi-dimensional data-sets. This thesis develops a framework that is intended to support not only the integration of such data for visual, interactive, and immersive displays, but also provides a method for performing risk analysis. Previously static VR environments are enhanced with time-evolutionary capabilities. Four candidate algorithms are evaluated for this purpose – deterministic modeling, auto-regressive moving average modeling, genetic algorithm modeling, and hidden Markov modeling. Benefits, drawbacks, and trade-offs are evaluated with reference to their suitability for development in a VR environment. The methods developed in this research work are demonstrated by applying them to multi-sensor data obtained during the in-line, nondestructive evaluation of gas transmission pipelines

Robust decision-making with model uncertainty in aerospace systems

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Aeronautics and Astronautics, 2008.Includes bibliographical references (p. 161-168).Actual performance of sequential decision-making problems can be extremely sensitive to errors in the models, and this research addressed the role of robustness in coping with this uncertainty. The first part of this thesis presents a computationally efficient sampling methodology, Dirichlet Sigma Points, for solving robust Markov Decision Processes with transition probability uncertainty. A Dirichlet prior is used to model the uncertainty in the transition probabilities. This approach uses the first two moments of the Dirichlet to generates samples of the uncertain probabilities and uses these samples to find the optimal robust policy. The Dirichlet Sigma Point method requires a much smaller number of samples than conventional Monte Carlo approaches, and is empirically demonstrated to be a very good approximation to the robust solution obtained with a very large number of samples. The second part of this thesis discusses the area of robust hybrid estimation. Model uncertainty in hybrid estimation can result in significant covariance mismatches and inefficient estimates. The specific problem of covariance underestimation is addressed, and a new robust estimator is developed that finds the largest covariance admissible within a prescribed uncertainty set. The robust estimator can be found by solving a small convex optimization problem in conjunction with Monte Carlo sampling, and reduces estimation errors in the presence of transition probability uncertainty. The Dirichlet Sigma Points are extended to this problem to reduce the computational requirements of the estimator. In the final part of the thesis, the Dirichlet Sigma Points are extended for real-time adaptation. Using insight from estimation theory, a modified version of the Dirichlet Sigma Points is presented that significantly improves the response time of classical estimators. The thesis is concluded with hardware implementation of these robust and adaptive algorithms on the RAVEN testbed, demonstrating their applicability to real-life UAV missions.by Luca Francesco Bertuccelli.Ph.D

Towards adaptive anomaly detection systems using boolean combination of hidden Markov models

Anomaly detection monitors for significant deviations from normal system behavior. Hidden Markov Models (HMMs) have been successfully applied in many intrusion detection applications, including anomaly detection from sequences of operating system calls. In practice, anomaly detection systems (ADSs) based on HMMs typically generate false alarms because they are designed using limited representative training data and prior knowledge. However, since new data may become available over time, an important feature of an ADS is the ability to accommodate newly-acquired data incrementally, after it has originally been trained and deployed for operations. Incremental re-estimation of HMM parameters raises several challenges. HMM parameters should be updated from new data without requiring access to the previously-learned training data, and without corrupting previously-learned models of normal behavior. Standard techniques for training HMM parameters involve iterative batch learning, and hence must observe the entire training data prior to updating HMM parameters. Given new training data, these techniques must restart the training procedure using all (new and previously-accumulated) data. Moreover, a single HMM system for incremental learning may not adequately approximate the underlying data distribution of the normal process, due to the many local maxima in the solution space. Ensemble methods have been shown to alleviate knowledge corruption, by combining the outputs of classifiers trained independently on successive blocks of data. This thesis makes contributions at the HMM and decision levels towards improved accuracy, efficiency and adaptability of HMM-based ADSs. It first presents a survey of techniques found in literature that may be suitable for incremental learning of HMM parameters, and assesses the challenges faced when these techniques are applied to incremental learning scenarios in which the new training data is limited and abundant. Consequently, An efficient alternative to the Forward-Backward algorithm is first proposed to reduce the memory complexity without increasing the computational overhead of HMM parameters estimation from fixed-size abundant data. Improved techniques for incremental learning of HMM parameters are then proposed to accommodate new data over time, while maintaining a high level of performance. However, knowledge corruption caused by a single HMM with a fixed number of states remains an issue. To overcome such limitations, this thesis presents an efficient system to accommodate new data using a learn-and-combine approach at the decision level. When a new block of training data becomes available, a new pool of base HMMs is generated from the data using a different number of HMM states and random initializations. The responses from the newly-trained HMMs are then combined to those of the previously-trained HMMs in receiver operating characteristic (ROC) space using novel Boolean combination (BC) techniques. The learn-and-combine approach allows to select a diversified ensemble of HMMs (EoHMMs) from the pool, and adapts the Boolean fusion functions and thresholds for improved performance, while it prunes redundant base HMMs. The proposed system is capable of changing its desired operating point during operations, and this point can be adjusted to changes in prior probabilities and costs of errors. During simulations conducted for incremental learning from successive data blocks using both synthetic and real-world system call data sets, the proposed learn-and-combine approach has been shown to achieve the highest level of accuracy than all related techniques. In particular, it can sustain a significantly higher level of accuracy than when the parameters of a single best HMM are re-estimated for each new block of data, using the reference batch learning and the proposed incremental learning techniques. It also outperforms static fusion techniques such as majority voting for combining the responses of new and previously-generated pools of HMMs. Ensemble selection techniques have been shown to form compact EoHMMs for operations, by selecting diverse and accurate base HMMs from the pool while maintaining or improving the overall system accuracy. Pruning has been shown to prevents pool sizes from increasing indefinitely with the number of data blocks acquired over time. Therefore, the storage space for accommodating HMMs parameters and the computational costs of the selection techniques are reduced, without negatively affecting the overall system performance. The proposed techniques are general in that they can be employed to adapt HMM-based systems to new data, within a wide range of application domains. More importantly, the proposed Boolean combination techniques can be employed to combine diverse responses from any set of crisp or soft one- or two-class classifiers trained on different data or features or trained according to different parameters, or from different detectors trained on the same data. In particular, they can be effectively applied when training data is limited and test data is imbalanced

Adaptive estimation of HMM transition probabilities

This paper presents new schemes for recursive estimation of the state transition probabilities for hidden Markov models (HMM's) via extended least squares (ELS) and recursive state prediction error (RSPE) methods. Local convergence analysis for the proposed RSPE algorithm is shown using the ordinary differential equation (ODE) approach developed for the more familiar recursive output prediction error (RPE) methods. The presented scheme converges and is relatively well conditioned compared with the ..

Adaptive estimation of HMM transition probabilities

This paper presents new schemes for recursive estimation of the state transition probabilities for hidden Markov models (HMMs) via Extended Least Squares (ELS) and Recursive Prediction Error (RPE) methods. These new schemes are designed to be consistent and well conditioned, compared to the previous RPE schemes which are known to be ill-conditioned in low noise environments. The ELS algorithms presented in this paper are computationally of order N 2, each time instant, where N is the number of Markov states, compared with the computationally effort of N 4, each time instant, required to implement the previous RPE scheme. However, the RPE algorithms proposed in this paper, although requiring less computational effort are still of order N 4. An consistent algorithm for simultaneous estimation of the state output levels and the state transition probabilities is also presented and discussed. Implementation aspects of all proposed algorithms are discussed, and simulation studies are presented to illustrate convergence and convergence rates.</p

Adaptive estimation of HMM transition probabilities

This paper presents new schemes for recursive estimation of the state transition probabilities for hidden Markov models (HMMs) via Recursive Prediction Error (RPE) methods. These new schemes are designed to be consistent and well conditioned, compared to the previous RPE schemes which are known to be ill-conditioned in low noise environments. The RPE algorithms proposed in this paper, although requiring less computational effort than the previous algorithms are still of order N 4, each time instant, where N is the number of Markov states. Extended least squares (ELS) algorithms are also presented which less computational effort (order N 2 per time instant) but for which no convergence results are presented. An consistent algorithm for simultaneous estimation of the state output levels and the state transition probabilities is also presented and discussed. Implementation aspects of all proposed algorithms are discussed, and simulation studies are presented to illustrate convergence and convergence rates.</p