Learning Markov Decision Processes for Model Checking

Constructing an accurate system model for formal model verification can be both resource demanding and time-consuming. To alleviate this shortcoming, algorithms have been proposed for automatically learning system models based on observed system behaviors. In this paper we extend the algorithm on learning probabilistic automata to reactive systems, where the observed system behavior is in the form of alternating sequences of inputs and outputs. We propose an algorithm for automatically learning a deterministic labeled Markov decision process model from the observed behavior of a reactive system. The proposed learning algorithm is adapted from algorithms for learning deterministic probabilistic finite automata, and extended to include both probabilistic and nondeterministic transitions. The algorithm is empirically analyzed and evaluated by learning system models of slot machines. The evaluation is performed by analyzing the probabilistic linear temporal logic properties of the system as well as by analyzing the schedulers, in particular the optimal schedulers, induced by the learned models.Comment: In Proceedings QFM 2012, arXiv:1212.345

Shale heavy metal isotope records of low environmental O2 between two Archean Oxidation Events

© The Author(s), 2022. This article is distributed under the terms of the Creative Commons Attribution License. The definitive version was published in Ostrander, C. M., Kendall, B., Gordon, G. W., Nielsen, S. G., Zheng, W., & Anbar, A. D. Shale heavy metal isotope records of low environmental O2 between two Archean Oxidation Events. Frontiers in Earth Science, 10, (2022): 833609, https://doi.org/10.3389/feart.2022.833609.Evidence of molecular oxygen (O2) accumulation at Earth’s surface during the Archean (4.0–2.5 billion years ago, or Ga) seems to increase in its abundance and compelling nature toward the end of the eon, during the runup to the Great Oxidation Event. Yet, many details of this late-Archean O2 story remain under-constrained, such as the extent, tempo, and location of O2 accumulation. Here, we present a detailed Fe, Tl, and U isotope study of shales from a continuous sedimentary sequence deposited between ∼2.6 and ∼2.5 Ga and recovered from the Pilbara Craton of Western Australia (the Wittenoom and Mt. Sylvia formations preserved in drill core ABDP9). We find a progressive decrease in bulk-shale Fe isotope compositions moving up core (as low as δ56Fe = –0.78 ± 0.08‰; 2SD) accompanied by invariant authigenic Tl isotope compositions (average ε205TlA = –2.0 ± 0.6; 2SD) and bulk-shale U isotope compositions (average δ238U = –0.30 ± 0.05‰; 2SD) that are both not appreciably different from crustal rocks or bulk silicate Earth. While there are multiple possible interpretations of the decreasing δ56Fe values, many, to include the most compelling, invoke strictly anaerobic processes. The invariant and near-crustal ε205TlA and δ238U values point even more strongly to this interpretation, requiring reducing to only mildly oxidizing conditions over ten-million-year timescales in the late-Archean. For the atmosphere, our results permit either homogenous and low O2 partial pressures (between 10−6.3 and 10−6 present atmospheric level) or heterogeneous and spatially restricted O2 accumulation nearest the sites of O2 production. For the ocean, our results permit minimal penetration of O2 in marine sediments over large areas of the seafloor, at most sufficient for the burial of Fe oxide minerals but insufficient for the burial of Mn oxide minerals. The persistently low background O2 levels implied by our dataset between ∼2.6 and ∼2.5 Ga contrast with the timeframes immediately before and after, where strong evidence is presented for transient Archean Oxidation Events. Viewed in this broader context, our data support the emerging narrative that Earth’s initial oxygenation was a dynamic process that unfolded in fits-and-starts over many hundreds-of-millions of years.This work was supported financially by the NSF Frontiers in Earth System Dynamics program award NSF EAR-1338810 (AA), a Woods Hole Oceanographic Institution Postdoctoral Scholarship (CO), a NSERC Discovery Grant (RGPIN-435930) and the Canada Research Chair program (BK), and a NASA Exobiology award 80NSSC20K0615 (SN)

NMR multiple quantum coherences in quasi-one-dimensional spin systems: Comparison with ideal spin-chain dynamics

The 19F spins in a crystal of fluorapatite have often been used to experimentally approximate a one-dimensional spin system. Under suitable multi-pulse control, the nuclear spin dynamics may be modeled to first approximation by a double-quantum one-dimensional Hamiltonian, which is analytically solvable for nearest-neighbor couplings. Here, we use solid-state nuclear magnetic resonance techniques to investigate the multiple quantum coherence dynamics of fluorapatite, with an emphasis on understanding the region of validity for such a simplified picture. Using experimental, numerical, and analytical methods, we explore the effects of long-range intra-chain couplings, cross-chain couplings, as well as couplings to a spin environment, all of which tend to damp the oscillations of the multiple quantum coherence signal at sufficiently long times. Our analysis characterizes the extent to which fluorapatite can faithfully simulate a one-dimensional quantum wire.Comment: 14 pages, 11 eps color figure

Quantum teleportation between light and matter

Quantum teleportation is an important ingredient in distributed quantum networks, and can also serve as an elementary operation in quantum computers. Teleportation was first demonstrated as a transfer of a quantum state of light onto another light beam; later developments used optical relays and demonstrated entanglement swapping for continuous variables. The teleportation of a quantum state between two single material particles (trapped ions) has now also been achieved. Here we demonstrate teleportation between objects of a different nature - light and matter, which respectively represent 'flying' and 'stationary' media. A quantum state encoded in a light pulse is teleported onto a macroscopic object (an atomic ensemble containing 10^12 caesium atoms). Deterministic teleportation is achieved for sets of coherent states with mean photon number (n) up to a few hundred. The fidelities are 0.58+-0.02 for n=20 and 0.60+-0.02 for n=5 - higher than any classical state transfer can possibly achieve. Besides being of fundamental interest, teleportation using a macroscopic atomic ensemble is relevant for the practical implementation of a quantum repeater. An important factor for the implementation of quantum networks is the teleportation distance between transmitter and receiver; this is 0.5 metres in the present experiment. As our experiment uses propagating light to achieve the entanglement of light and atoms required for teleportation, the present approach should be scalable to longer distances.Comment: 23 pages, 8 figures, incl. supplementary informatio

Learning deterministic probabilistic automata from a model checking perspective

Probabilistic automata models play an important role in the formal design and analysis of hard- and software systems. In this area of applications, one is often interested in formal model-checking procedures for verifying critical system properties. Since adequate system models are often difficult to design manually, we are interested in learning models from observed system behaviors. To this end we adopt techniques for learning finite probabilistic automata, notably the Alergia algorithm. In this paper we show how to extend the basic algorithm to also learn automata models for both reactive and timed systems. A key question of our investigation is to what extent one can expect a learned model to be a good approximation for the kind of probabilistic properties one wants to verify by model checking. We establish theoretical convergence properties for the learning algorithm as well as for probability estimates of system properties expressed in linear time temporal logic and linear continuous stochastic logic. We empirically compare the learning algorithm with statistical model checking and demonstrate the feasibility of the approach for practical system verification

The Affective Impact of Financial Skewness on Neural Activity and Choice

Few finance theories consider the influence of “skewness” (or large and asymmetric but unlikely outcomes) on financial choice. We investigated the impact of skewed gambles on subjects' neural activity, self-reported affective responses, and subsequent preferences using functional magnetic resonance imaging (FMRI). Neurally, skewed gambles elicited more anterior insula activation than symmetric gambles equated for expected value and variance, and positively skewed gambles also specifically elicited more nucleus accumbens (NAcc) activation than negatively skewed gambles. Affectively, positively skewed gambles elicited more positive arousal and negatively skewed gambles elicited more negative arousal than symmetric gambles equated for expected value and variance. Subjects also preferred positively skewed gambles more, but negatively skewed gambles less than symmetric gambles of equal expected value. Individual differences in both NAcc activity and positive arousal predicted preferences for positively skewed gambles. These findings support an anticipatory affect account in which statistical properties of gambles—including skewness—can influence neural activity, affective responses, and ultimately, choice

Characterisation of the Faecal Bacterial Community in Adult and Elderly Horses Fed a High Fibre, High Oil or High Starch Diet Using 454 Pyrosequencing

Faecal samples were collected from seventeen animals, each fed three different diets (high fibre, high fibre with a starch rich supplement and high fibre with an oil rich supplement). DNA was extracted and the V1–V2 regions of 16SrDNA were 454-pyrosequenced to investigate the faecal microbiome of the horse. The effect of age was also considered by comparing mature (8 horses aged 5–12) versus elderly horses (9 horses aged 19–28). A reduction in diversity was found in the elderly horse group. Significant differences between diets were found at an OTU level (52 OTUs at corrected Q<0.1). The majority of differences found were related to the Firmucutes phylum (37) with some changes in Bacteroidetes (6), Proteobacteria (3), Actinobacteria (2) and Spirochaetes (1). For the forage only diet,with no added starch or oil, we found 30/2934 OTUs (accounting for 15.9% of sequences) present in all horses. However the core (i.e. present in all horses) associated with the oil rich supplemented diet was somewhat smaller (25/3029 OTUs, 10.3% ) and the core associated with the starch rich supplemented diet was even smaller (15/2884 OTUs, 5.4% ). The core associated with samples across all three diets was extremely small (6/5689 OTUs accounting for only 2.3% of sequences) and dominated by the order Clostridiales, with the most abundant family being Lachnospiraceae. In conclusion, forage based diets plus starch or oil rich complementary feeds were associated with differences in the faecal bacterial community compared with the forage alone. Further, as observed in people, ageing is associated with a reduction in bacterial diversity. However there was no change in the bacterial community structure in these healthy animals associated with age

Mapping coherence in measurement via full quantum tomography of a hybrid optical detector

Quantum states and measurements exhibit wave-like --- continuous, or particle-like --- discrete, character. Hybrid discrete-continuous photonic systems are key to investigating fundamental quantum phenomena, generating superpositions of macroscopic states, and form essential resources for quantum-enhanced applications, e.g. entanglement distillation and quantum computation, as well as highly efficient optical telecommunications. Realizing the full potential of these hybrid systems requires quantum-optical measurements sensitive to complementary observables such as field quadrature amplitude and photon number. However, a thorough understanding of the practical performance of an optical detector interpolating between these two regions is absent. Here, we report the implementation of full quantum detector tomography, enabling the characterization of the simultaneous wave and photon-number sensitivities of quantum-optical detectors. This yields the largest parametrization to-date in quantum tomography experiments, requiring the development of novel theoretical tools. Our results reveal the role of coherence in quantum measurements and demonstrate the tunability of hybrid quantum-optical detectors.Comment: 7 pages, 3 figure

Capsule carbohydrate structure determines virulence in Acinetobacter baumannii

Acinetobacter baumannii is a highly antibiotic-resistant bacterial pathogen for which novel therapeutic approaches are needed. Unfortunately, the drivers of virulence in A. baumannii remain uncertain. By comparing genomes among a panel of A. baumannii strains we identified a specific gene variation in the capsule locus that correlated with altered virulence. While less virulent strains possessed the intact gene gtr6, a hypervirulent clinical isolate contained a spontaneous transposon insertion in the same gene, resulting in the loss of a branchpoint in capsular carbohydrate structure. By constructing isogenic gtr6 mutants, we confirmed that gtr6-disrupted strains were protected from phagocytosis in vitro and displayed higher bacterial burden and lethality in vivo. Gtr6+ strains were phagocytized more readily and caused lower bacterial burden and no clinical illness in vivo. We found that the CR3 receptor mediated phagocytosis of gtr6+, but not gtr6-, strains in a complement-dependent manner. Furthermore, hypovirulent gtr6+ strains demonstrated increased virulence in vivo when CR3 function was abrogated. In summary, loss-of-function in a single capsule assembly gene dramatically altered virulence by inhibiting complement deposition and recognition by phagocytes across multiple A. baumannii strains. Thus, capsular structure can determine virulence among A. baumannii strains by altering bacterial interactions with host complement-mediated opsonophagocytosis