Hidden hybrid Markov/semi-Markov chains.

http://www.sciencedirect.com/science?ₒb=IssueURL&_tockey=%23TOC%235880%232005%23999509996%23596026%23FLA%23&ₐuth=y&view=c&ₐcct=C000056834&_version=1&_urlVersion=0&_userid=2292769&md5=87e7f8be94f92a8574da566c600ce631International audienceModels that combine Markovian states with implicit geometric state occupancy distributions and semi-Markovian states with explicit state occupancy distributions, are investigated. This type of model retains the flexibility of hidden semi-Markov chains for the modeling of short or medium size homogeneous zones along sequences but also enables the modeling of long zones with Markovian states. The forward-backward algorithm, which in particular enables to implement efficiently the E-step of the EM algorithm, and the Viterbi algorithm for the restoration of the most likely state sequence are derived. It is also shown that macro-states, i.e. series-parallel networks of states with common observation distribution, are not a valid alternative to semi-Markovian states but may be useful at a more macroscopic level to combine Markovian states with semi-Markovian states. This statistical modeling approach is illustrated by the analysis of branching and flowering patterns in plants

Animal vocal sequences: not the Markov chains we thought they were.

Many animals produce vocal sequences that appear complex. Most researchers assume that these sequences are well characterized as Markov chains (i.e. that the probability of a particular vocal element can be calculated from the history of only a finite number of preceding elements). However, this assumption has never been explicitly tested. Furthermore, it is unclear how language could evolve in a single step from a Markovian origin, as is frequently assumed, as no intermediate forms have been found between animal communication and human language. Here, we assess whether animal taxa produce vocal sequences that are better described by Markov chains, or by non-Markovian dynamics such as the 'renewal process' (RP), characterized by a strong tendency to repeat elements. We examined vocal sequences of seven taxa: Bengalese finches Lonchura striata domestica, Carolina chickadees Poecile carolinensis, free-tailed bats Tadarida brasiliensis, rock hyraxes Procavia capensis, pilot whales Globicephala macrorhynchus, killer whales Orcinus orca and orangutans Pongo spp. The vocal systems of most of these species are more consistent with a non-Markovian RP than with the Markovian models traditionally assumed. Our data suggest that non-Markovian vocal sequences may be more common than Markov sequences, which must be taken into account when evaluating alternative hypotheses for the evolution of signalling complexity, and perhaps human language origins.This is the author's accepted manuscript and will be under embargo until the 20th of August 2015. This final version is published by Royal Society Publishing here: http://dx.doi.org/10.1098/rspb.2014.1370

Some aspects of traffic control and performance evaluation of ATM networks

The emerging high-speed Asynchronous Transfer Mode (ATM) networks are expected to integrate through statistical multiplexing large numbers of traffic sources having a broad range of statistical characteristics and different Quality of Service (QOS) requirements. To achieve high utilisation of network resources while maintaining the QOS, efficient traffic management strategies have to be developed. This thesis considers the problem of traffic control for ATM networks. The thesis studies the application of neural networks to various ATM traffic control issues such as feedback congestion control, traffic characterization, bandwidth estimation, and Call Admission Control (CAC). A novel adaptive congestion control approach based on a neural network that uses reinforcement learning is developed. It is shown that the neural controller is very effective in providing general QOS control. A Finite Impulse Response (FIR) neural network is proposed to adaptively predict the traffic arrival process by learning the relationship between the past and future traffic variations. On the basis of this prediction, a feedback flow control scheme at input access nodes of the network is presented. Simulation results demonstrate significant performance improvement over conventional control mechanisms. In addition, an accurate yet computationally efficient approach to effective bandwidth estimation for multiplexed connections is investigated. In this method, a feed forward neural network is employed to model the nonlinear relationship between the effective bandwidth and the traffic situations and a QOS measure. Applications of this approach to admission control, bandwidth allocation and dynamic routing are also discussed. A detailed investigation has indicated that CAC schemes based on effective bandwidth approximation can be very conservative and prevent optimal use of network resources. A modified effective bandwidth CAC approach is therefore proposed to overcome the drawback of conventional methods. Considering statistical multiplexing between traffic sources, we directly calculate the effective bandwidth of the aggregate traffic which is modelled by a two-state Markov modulated Poisson process via matching four important statistics. We use the theory of large deviations to provide a unified description of effective bandwidths for various traffic sources and the associated ATM multiplexer queueing performance approximations, illustrating their strengths and limitations. In addition, a more accurate estimation method for ATM QOS parameters based on the Bahadur-Rao theorem is proposed, which is a refinement of the original effective bandwidth approximation and can lead to higher link utilisation

Convergence of adaptive morphological filters in the context of Markov chains

A typical parameterized r-opening *r is a filter defined as a union of openings by a collection of compact, convex structuring elements, each of which is governed by a parameter vector r. It reduces to a single parameter r-opening filter by a set of structuring elements when r is a scalar sizing parameter. The parameter vector is adjusted by a set of adaptation rules according to whether the re construction Ar derived from r correctly or incorrectly passes the signal and noise grains sampled from the image. Applied to the signal-union-noise model, the optimization problem is to find the vector of r that minimizes the Mean-Absolute-Error between the filtered and ideal image processes. The adaptive r-opening filter fits into the framework of Markov processes, the adaptive parameter being the state of the process. For a single parameter r-opening filter, we proved that there exists a stationary distribution governing the parameter in the steady state and convergence is characterized in terms of the steady-state distribution. Key filter properties such as parameter mean, parameter variance, and expected error in the steady state are characterized via the stationary distribution. Steady-state behavior is compared to the optimal solution for the uniform model, for which it is possible to derive a closed-form solution for the optimal filter. We also developed the Markov adaptation system for multiparameter opening filters and provided numerical solutions to some special cases. For multiparameter r-opening filters, various adaptive models derived from various assumptions on the form of the filter have been studied. Although the state-probability increment equations can be derived from the appropriate Chapman-Kolmogorov equations, the closed-form representation of steady-state distributions is mathematically problematic due to the support geometry of the boundary states and their transitions. Therefore, numerical methods are employed to approximate for steady state probability distributions. The technique developed for conventional opening filters is also applied to bandpass opening filters. In present thesis study, the concept of signal and noise pass sets plays a central role throughout the adaptive filter analysis. The pass set reduces to the granulometric measure (or {&r}-measure) of the signal and noise grain. Optimization and adaptation are characterized in terms of the distribution of the granulometric measures for single parameter filters, or in terms of the multivariate distribution of the signal and noise pass sets. By introducing these concepts, this thesis study also provides some optimal opening filter error equations. It has been shown in the case of the uniform distribution of single sizing parameter that there is a strong agreement between the adaptive filter and optimal filter based on analytic error minimization. This agreement has been also demonstrated in various r-opening filters. Furthermore, the probabilistic interpretation has a close connection to traditional linear adaptive filter theory. The method has been applied to the classical grain separation (clutter removal) problem. *See content for correct numerical representation

Learning and Management for Internet-of-Things: Accounting for Adaptivity and Scalability

Internet-of-Things (IoT) envisions an intelligent infrastructure of networked smart devices offering task-specific monitoring and control services. The unique features of IoT include extreme heterogeneity, massive number of devices, and unpredictable dynamics partially due to human interaction. These call for foundational innovations in network design and management. Ideally, it should allow efficient adaptation to changing environments, and low-cost implementation scalable to massive number of devices, subject to stringent latency constraints. To this end, the overarching goal of this paper is to outline a unified framework for online learning and management policies in IoT through joint advances in communication, networking, learning, and optimization. From the network architecture vantage point, the unified framework leverages a promising fog architecture that enables smart devices to have proximity access to cloud functionalities at the network edge, along the cloud-to-things continuum. From the algorithmic perspective, key innovations target online approaches adaptive to different degrees of nonstationarity in IoT dynamics, and their scalable model-free implementation under limited feedback that motivates blind or bandit approaches. The proposed framework aspires to offer a stepping stone that leads to systematic designs and analysis of task-specific learning and management schemes for IoT, along with a host of new research directions to build on.Comment: Submitted on June 15 to Proceeding of IEEE Special Issue on Adaptive and Scalable Communication Network

Aggregate matrix-analytic techniques and their applications

The complexity of computer systems affects the complexity of modeling techniques that can be used for their performance analysis. In this dissertation, we develop a set of techniques that are based on tractable analytic models and enable efficient performance analysis of computer systems. Our approach is three pronged: first, we propose new techniques to parameterize measurement data with Markovian-based stochastic processes that can be further used as input into queueing systems; second, we propose new methods to efficiently solve complex queueing models; and third, we use the proposed methods to evaluate the performance of clustered Web servers and propose new load balancing policies based on this analysis.;We devise two new techniques for fitting measurement data that exhibit high variability into Phase-type (PH) distributions. These techniques apply known fitting algorithms in a divide-and-conquer fashion. We evaluate the accuracy of our methods from both the statistics and the queueing systems perspective. In addition, we propose a new methodology for fitting measurement data that exhibit long-range dependence into Markovian Arrival Processes (MAPs).;We propose a new methodology, ETAQA, for the exact solution of M/G/1-type processes, (GI/M/1-type processes, and their intersection, i.e., quasi birth-death (QBD) processes. ETAQA computes an aggregate steady state probability distribution and a set of measures of interest. E TAQA is numerically stable and computationally superior to alternative solution methods. Apart from ETAQA, we propose a new methodology for the exact solution of a class of GI/G/1-type processes based on aggregation/decomposition.;Finally, we demonstrate the applicability of the proposed techniques by evaluating load balancing policies in clustered Web servers. We address the high variability in the service process of Web servers by dedicating the servers of a cluster to requests of similar sizes and propose new, content-aware load balancing policies. Detailed analysis shows that the proposed policies achieve high user-perceived performance and, by continuously adapting their scheduling parameters to the current workload characteristics, provide good performance under conditions of transient overload

Towards faster numerical solution of Continuous Time Markov Chains stored by symbolic data structures

This work considers different aspects of model-based performance- and dependability analysis. This research area analyses systems (e.g. computer-, telecommunication- or production-systems) in order to quantify their performance and reliability. Such an analysis can be carried out already in the planning phase, without a physically existing system. All aspects treated in this work are based on finite state spaces (i.e. the models only have finitely many states) and a representation of the state graphs by Multi-Terminal Binary Decision Diagrams (MTBDDs). Currently, there are many tools that transform high-level model specifications (e.g. process algebra or Petri-Net) to low-level models (e.g. Markov chains). Markov chains can be represented by sparse matrices. For complex models very large state spaces may occur (this phenomenon is called state space explosion in the literature) and accordingly very large matrices representing the state graphs. The problem of building the model from the specification and storing the state graph can be regarded as solved: There are heuristics for compactly storing the state graph by MTBDD or Kronecker data structure and there are efficient algorithms for the model generation and functional analysis. For the quantitative analysis there are still problems due to the size of the underlying state space. This work provides some methods to alleviate the problems in case of MTBDD-based storage of the state graph. It is threefold: 1. For the generation of smaller state graphs in the model generation phase (which usually are easier to solve) a symbolic elimination algorithm is developed. 2. For the calculation of steady-state probabilities of Markov chains a multilevel algorithm is developed which allows for faster solutions. 3. For calculating the most probable paths in a state graph, the mean time to the first failure of a system and related measures, a path-based solver is developed

Statistical inverse problems for population processes

Population processes are stochastic processes that record the dynamics of the number of individuals in a population, and have many different applications in a broad range of areas. Population processes are often modelled as Markov processes, and have the important feature that transitions correspond either to an increase or a decrease in the population size. These two types of transitions are often referred to as births and deaths. A specific class of population processes is the class of birth-death processes, where transitions can only increase or decrease the population by one at a time. In many real-life situations the dynamics of a population is affected by exogenous, often unobservable, factors. Therefore, this thesis considers population processes of which the parameters are affected by an underlying stochastic process, referred to as the background process. The aim is to find reliable inference techniques to estimate the parameters, including those related to the background process, from discrete-time observations of the population size. The statistical inference is complicated severely by the fact that a substantial part of the process is unobserved. First, the underlying background process is not observed. Second, only the population size is observed, which is the net effect of all the transitions in the dynamics of the population. Last, the population size is observed in discrete time, hence the transitions in between two consecutive observations are not observed. In this thesis we show a collection of techniques to overcome these complications for a variety of population processes. The aspects in which the models differ, ask for specific inference techniques. For a certain class of Markov-modulated population processes, we show how the well-known EM algorithm can be used to estimate the model parameters. In these models, the background process is a finite, continuous-time Markov chain and the parameters of the population process switch between distinct values at the jump times of this Markov chain. An algorithm is presented that iteratively maximizes the likelihood function and at the same time updates the parameter estimates. A generalization of the conventional birth-death process, involving a background process, is the quasi birth-death process. We use the Erlangization technique to evaluate the likelihood function for this kind of processes, which can then be maximized numerically to obtain maximum likelihood estimates. A specific model in the class of quasi birth-death processes is a birth-death process of which the births follow a hypoexponential distribution with L phases and are controlled by an on/off mechanism. We call this the on/off-seq-L process, and use it to model the dynamics of populations of mRNA molecules in single living cells. Numerical complications related to the likelihood maximization are analyzed and solutions are presented. Based on real-life data, we illustrate the estimation method, and perform a model selection procedure on the number of phases and on the on/off mechanism. Last, we consider a class of discrete-time multivariate population processes under Markov-modulation. In these models, the population process is defined on a network with finitely many nodes. In addition to the births and deaths that can occur at each of the nodes, the individuals follow a probabilistic route through the network. We introduce the saddlepoint technique and show how it can be used to evaluate the likelihood function based on observations of the network population vector. The likelihood function can again be maximized numerically to obtain maximum likelihood estimates. Throughout the thesis, the accuracy of the inference methods is investigated by extensive simulation studies