Learning Markov Decision Processes for Model Checking

Constructing an accurate system model for formal model verification can be both resource demanding and time-consuming. To alleviate this shortcoming, algorithms have been proposed for automatically learning system models based on observed system behaviors. In this paper we extend the algorithm on learning probabilistic automata to reactive systems, where the observed system behavior is in the form of alternating sequences of inputs and outputs. We propose an algorithm for automatically learning a deterministic labeled Markov decision process model from the observed behavior of a reactive system. The proposed learning algorithm is adapted from algorithms for learning deterministic probabilistic finite automata, and extended to include both probabilistic and nondeterministic transitions. The algorithm is empirically analyzed and evaluated by learning system models of slot machines. The evaluation is performed by analyzing the probabilistic linear temporal logic properties of the system as well as by analyzing the schedulers, in particular the optimal schedulers, induced by the learned models.Comment: In Proceedings QFM 2012, arXiv:1212.345

Machine learning enabled multiple illumination quantitative optoacoustic oximetry imaging in humans.

Optoacoustic (OA) imaging is a promising modality for quantifying blood oxygen saturation (sO2) in various biomedical applications - in diagnosis, monitoring of organ function, or even tumor treatment planning. We present an accurate and practically feasible real-time capable method for quantitative imaging of sO2 based on combining multispectral (MS) and multiple illumination (MI) OA imaging with learned spectral decoloring (LSD). For this purpose we developed a hybrid real-time MI MS OA imaging setup with ultrasound (US) imaging capability; we trained gradient boosting machines on MI spectrally colored absorbed energy spectra generated by generic Monte Carlo simulations and used the trained models to estimate sO2 on real OA measurements. We validated MI-LSD in silico and on in vivo image sequences of radial arteries and accompanying veins of five healthy human volunteers. We compared the performance of the method to prior LSD work and conventional linear unmixing. MI-LSD provided highly accurate results in silico and consistently plausible results in vivo. This preliminary study shows a potentially high applicability of quantitative OA oximetry imaging, using our method

Direct Evidence in Risk Attitudes and Migration

It has long been hypothesized that individuals'' migration propensities depend on their attitudes towards risk, but the empirical evidence, to the extent that it exists, has been indirect. In this paper, we use newly available data from the German Socio-Economic Panel to measure directly the relationship between migration propensities and attitudes towards risk. We find that individuals who are more willing to take risks are more likely to migrate between labor markets in Germany. This result is robust to stratifying by age, sex, education, national origin, and a variety of other demographic characteristics. The effect is substantial relative to the unconditional migration propensity and compared to the conventional determinants of migration. We find no evidence that these findings are the result of reverse causality.education, training and the labour market;

Direct Evidence on Risk Attitudes and Migration

Geographic mobility is important for the functioning of labor markets because it brings labor resources to where they can be most efficiently used. It has long been hypothesized that individuals' migration propensities depend on their attitudes towards risk, but the empirical evidence, to the extent that it exists, has been indirect. In this paper, we use newly available data from the German Socio-Economic Panel to measure directly the relationship between migration propensities and attitudes towards risk. We find that individuals who are more willing to take risks are more likely to migrate between labor markets in Germany. This result is robust to stratifying by age, sex, education, national origin, and a variety of other demographic characteristics, as well as to the level of aggregation used to define geographic mobility. The effect is substantial relative to the unconditional migration propensity and compared to the conventional determinants of migration. We also find that being more willing to take risks is more important for the extensive than for the intensive margin of migration.Migration, attitudes, risk