Learning Markov Decision Processes for Model Checking

Constructing an accurate system model for formal model verification can be both resource demanding and time-consuming. To alleviate this shortcoming, algorithms have been proposed for automatically learning system models based on observed system behaviors. In this paper we extend the algorithm on learning probabilistic automata to reactive systems, where the observed system behavior is in the form of alternating sequences of inputs and outputs. We propose an algorithm for automatically learning a deterministic labeled Markov decision process model from the observed behavior of a reactive system. The proposed learning algorithm is adapted from algorithms for learning deterministic probabilistic finite automata, and extended to include both probabilistic and nondeterministic transitions. The algorithm is empirically analyzed and evaluated by learning system models of slot machines. The evaluation is performed by analyzing the probabilistic linear temporal logic properties of the system as well as by analyzing the schedulers, in particular the optimal schedulers, induced by the learned models.Comment: In Proceedings QFM 2012, arXiv:1212.345

Toward a more economical cluster state quantum computation

We assess the effects of an intrinsic model for imperfections in cluster states by introducing {\it noisy cluster states} and characterizing their role in the one-way model for quantum computation. The action of individual dephasing channels on cluster qubits is also studied. We show that the effect of non-idealities is limited by using small clusters, which requires compact schemes for computation. In light of this, we address an experimentally realizable four-qubit linear cluster which simulates a controlled-{\sf NOT} ({\sf CNOT}).Comment: 4 pages, 2 figures, RevTeX4; proposal for experimental setup include

Excitation of the ionized nitrogen molecule in the aurora

Thesis (M.S.) University of Alaska Fairbanks, 2002An understanding of the excitation mechanism of the ionospheric molecules during auroral activity is of vital importance for the overall ionospheric understanding including its interaction with the magnetosphere. In this thesis we study two emissions originating from the excited nitrogen molecule ion. The first negative (0,1) emission at 4278 Å originating from the B state, and Meinel (2,0) emission at 7852 Å originating from the A state during moderate to strong aurora have been observed with an imaging spectrograph at Poker Flat, Alaska. The B state has a short lifetime compared to the inverse collision frequency at auroral altitudes, while the A state can be deactivated during collisions at altitudes near 95 km. The B state can be populated by an up-welling of N₂ into sunlit regions. Both processes are expected to depend on auroral activity. If none of the processes are present we expect a constant ratio between the two emissions. Data for three nights have been studied and a constant ratio was found at all times. Thus neither deactivation of the A state or up-welling of the ion seem to appear during the observations presented here. The values of the ratio for the three nights are 2.53 plus-minus 0.38, 3.05 plus minus 0.22, and 3.40 plus minus 1.10, respectively

Finding the K shortest hyperpaths using reoptimization

The shortest hyperpath problem is an extension of the classical shortest path problem and has applications in many different areas. Recently, algorithms for finding the K shortest hyperpaths in a directed hypergraph have been developed by Andersen, Nielsen and Pretolani. In this paper we improve the worst-case computational complexity of an algorithm for finding the K shortest hyperpaths in an acyclic hypergraph. This result is obtained by applying new reoptimization techniques for shortest hyperpaths. The algorithm turns out to be quite effective in practice and has already been successfully applied in the context of stochastic time-dependent networks, for finding the K best strategies and for solving bicriterion problems.Network programming; Directed hypergraphs; K shortest hyperpaths; K shortest paths

K shortest paths in stochastic time-dependent networks

A substantial amount of research has been devoted to the shortest path problem in networks where travel times are stochastic or (deterministic and) time-dependent. More recently, a growing interest has been attracted by networks that are both stochastic and time-dependent. In these networks, the best route choice is not necessarily a path, but rather a time-adaptive strategy that assigns successors to nodes as a function of time. In some particular cases, the shortest origin-destination path must nevertheless be chosen a priori, since time-adaptive choices are not allowed. Unfortunately, finding the a priori shortest path is NP-hard, while the best time-adaptive strategy can be found in polynomial time. In this paper, we propose a solution method for the a priori shortest path problem, and we show that it can be easily adapted to the ranking of the first K shortest paths. Moreover, we present a computational comparison of time-adaptive and a priori route choices, pointing out the effect of travel time and cost distributions. The reported results show that, under realistic distributions, our solution methods are effectiveShortest paths; K shortest paths; stochastic time-dependent networks; routing; directed hypergraphs

A note on “Multicriteria adaptive paths in stochastic, time-varying networks”

In a recent paper, Opasanon and Miller-Hooks study multicriteria adaptive paths in stochastic time-varying networks. They propose a label correcting algorithm for finding the full set of efficient strategies. In this note we show that their algorithm is not correct, since it is based on a property that does not hold in general. Opasanon and Miller-Hooks also propose an algorithm for solving a parametric problem. We give a simplified algorithm which is linear in the input size.Multiple objective programming; shortest paths; stochastic time-dependent networks; time-adaptive strategies

Bicriterion a priori route choice in stochastic time-dependent networks.

In recent years there has been a growing interest in using stochastic time-dependent (STD) networks as a modelling tool for a number of applications within such areas as transportation and telecommunications. It is known that an optimal routing policy does not necessarily correspond to a path, but rather to a time-adaptive strategy. In some applications, however, it makes good sense to require that the routing policy corresponds to a loopless path in the network, that is, the time-adaptive aspect disappears and a priori route choice is considered. In this paper we consider bicriterion a priori route choice in STD networks, i.e. the problem of finding the set of efficient paths. Both expectation and min-max criteria are considered and a solution method based on the two-phase approach is devised. Experimental results reveal that the full set of efficient solutions can be determined on rather large test instances, which is in contrast to previously reported results for the time-adaptive caseStochastic time-dependent networks; Bicriterion shortest path; A priori route choice; Two-phase method