The Dreaming Variational Autoencoder for Reinforcement Learning Environments

Reinforcement learning has shown great potential in generalizing over raw sensory data using only a single neural network for value optimization. There are several challenges in the current state-of-the-art reinforcement learning algorithms that prevent them from converging towards the global optima. It is likely that the solution to these problems lies in short- and long-term planning, exploration and memory management for reinforcement learning algorithms. Games are often used to benchmark reinforcement learning algorithms as they provide a flexible, reproducible, and easy to control environment. Regardless, few games feature a state-space where results in exploration, memory, and planning are easily perceived. This paper presents The Dreaming Variational Autoencoder (DVAE), a neural network based generative modeling architecture for exploration in environments with sparse feedback. We further present Deep Maze, a novel and flexible maze engine that challenges DVAE in partial and fully-observable state-spaces, long-horizon tasks, and deterministic and stochastic problems. We show initial findings and encourage further work in reinforcement learning driven by generative exploration.Comment: Best Student Paper Award, Proceedings of the 38th SGAI International Conference on Artificial Intelligence, Cambridge, UK, 2018, Artificial Intelligence XXXV, 201

Trace metals distribution and uptake in soil and rice grown on a 3-year vermicompost amended soil

This study was designed to investigate the influence of vermicompost (VC) on trace metals distribution and uptake in soil and rice plant in research field as split plot arrangement based on randomized complete block design with three replications in 2008. Main-plot was VC and chemical fertilizer (CF) that were added to soil in 6 levels (20 and 40 ton/ha VC, 20 and 40 ton/ha VC + 1/2 CF, CF and control). Application years considered as sub-plot comprised 1, 2 and 3 years. The results indicated thatfertilizers and application periods treatments influenced micronutrients in soil and rice. Available copper (Cu) had no significant difference under different treatments. The highest available iron (Fe) was found in the 40 ton treatment group. During the 3 years, application of 20 ton and enriched 40 ton gave the most available zinc (Zn) and manganese (Mn). In VC and enriched VC, treatments happened to give the highest Zn uptake by rice. Under the 3 years, application of 40 ton/ha VC, the highest Fe (91.19 ppm) and Cu (13.66 ppm) concentration was seen in flag leaf, while Fe (31.35 ppm) and Mn (27.56 ppm) was seen in grain. With the application of enriched 20 ton VC, the maximum uptake of Mn by flag leaf and Cu by grain was obtained

Life Cycle Assessment of Municipal Waste Management System (Case Study: Karaj, Iran)

LCA has been defined as a tool for evaluating the environmental burdens and potential impacts that can be applied to municipal solid waste management systems for determine the optimum municipal solid waste (MSW) management strategy.To investigate the Waste Management system strategyof Karaj City we used LCA method. Three scenarios were defined and compared based on environmental burden include water pollution, air pollution, consumed energy and waste residues.. For each of these scenarios, an ecological indicator was achieved from checklist values. From the environmental point of view, results show that recycling is one of the best alternatives for Waste Management. Furthermore, composting has an important role in alleviating the load of pollutants and energy usage in the Waste Management system. ©JASEMKeywords: Waste Management system, LCA, Kara

The Dreaming Variational Autoencoder for Reinforcement Learning Environments

Reinforcement learning has shown great potential in generalizing over raw sensory data using only a single neural network for value optimization. There are several challenges in the current state-of-the-art reinforcement learning algorithms that prevent them from converging towards the global optima. It is likely that the solution to these problems lies in short- and long-term planning, exploration and memory management for reinforcement learning algorithms. Games are often used to benchmark reinforcement learning algorithms as they provide a flexible, reproducible, and easy to control environment. Regardless, few games feature a state-space where results in exploration, memory, and planning are easily perceived. This paper presents The Dreaming Variational Autoencoder (DVAE), a neural network based generative modeling architecture for exploration in environments with sparse feedback. We further present Deep Maze, a novel and flexible maze engine that challenges DVAE in partial and fully-observable state-spaces, long-horizon tasks, and deterministic and stochastic problems. We show initial findings and encourage further work in reinforcement learning driven by generative exploration.The Dreaming Variational Autoencoder for Reinforcement Learning EnvironmentsacceptedVersionNivå

Novel linear piezo‐resistive auxetic meta‐sensors with low Young's modulus by a core–shell conceptual design and electromechanical modelling

Production of piezo-resistive auxetic sensors is usually carried out through mixing and coating methods. Although these methods are beneficial, Young's modulus of mixed sensors becomes high because of using a high percentage of sensing elements while the durability of coated sensors gets low due to the separation of sensing elements from the sensor surface. This article presents a new core–shell metamaterial model to address the mentioned problems. The shell and the core are produced of polydimethylsiloxane (PDMS) rubber and a mixture of PDMS/graphite powders (73.45 wt% graphite powders), respectively. A finite element model is developed via COMSOL software to predict the electromechanical behaviors of the created sensor and verified by an experimental study. Scanning electron microscope imaging is conducted to detect the separations of the graphite particles. The main important feature of this meta-sensor is to possess a linear sensitivity due to having zero Poisson's ratio. The advantage of this method is that Young's modulus of the sensor does not decrease (unlike the mixing method), and the sensor-coated particles do not separate from the sensor after a while (unlike the coating method). The introduced model has advantages that promote potential applications such as using sensory gloves to detect, for instance, human hand movements

Global, regional, and national incidence, prevalence, and mortality of HIV, 1980–2017, and forecasts to 2030, for 195 countries and territories: a systematic analysis for the Global Burden of Diseases, Injuries, and Risk Factors Study 2017

Background Understanding the patterns of HIV/AIDS epidemics is crucial to tracking and monitoring the progress of prevention and control efforts in countries. We provide a comprehensive assessment of the levels and trends of HIV/AIDS incidence, prevalence, mortality, and coverage of antiretroviral therapy (ART) for 1980–2017 and forecast these estimates to 2030 for 195 countries and territories. Methods We determined a modelling strategy for each country on the basis of the availability and quality of data. For countries and territories with data from population-based seroprevalence surveys or antenatal care clinics, we estimated prevalence and incidence using an open-source version of the Estimation and Projection Package—a natural history model originally developed by the UNAIDS Reference Group on Estimates, Modelling, and Projections. For countries with cause-specific vital registration data, we corrected data for garbage coding (ie, deaths coded to an intermediate, immediate, or poorly defined cause) and HIV misclassification. We developed a process of cohort incidence bias adjustment to use information on survival and deaths recorded in vital registration to back-calculate HIV incidence. For countries without any representative data on HIV, we produced incidence estimates by pulling information from observed bias in the geographical region. We used a re-coded version of the Spectrum model (a cohort component model that uses rates of disease progression and HIV mortality on and off ART) to produce age-sex-specific incidence, prevalence, and mortality, and treatment coverage results for all countries, and forecast these measures to 2030 using Spectrum with inputs that were extended on the basis of past trends in treatment scale-up and new infections. Findings Global HIV mortality peaked in 2006 with 1·95 million deaths (95% uncertainty interval 1·87–2·04) and has since decreased to 0·95 million deaths (0·91–1·01) in 2017. New cases of HIV globally peaked in 1999 (3·16 million, 2·79–3·67) and since then have gradually decreased to 1·94 million (1·63–2·29) in 2017. These trends, along with ART scale-up, have globally resulted in increased prevalence, with 36·8 million (34·8–39·2) people living with HIV in 2017. Prevalence of HIV was highest in southern sub-Saharan Africa in 2017, and countries in the region had ART coverage ranging from 65·7% in Lesotho to 85·7% in eSwatini. Our forecasts showed that 54 countries will meet the UNAIDS target of 81% ART coverage by 2020 and 12 countries are on track to meet 90% ART coverage by 2030. Forecasted results estimate that few countries will meet the UNAIDS 2020 and 2030 mortality and incidence targets. Interpretation Despite progress in reducing HIV-related mortality over the past decade, slow decreases in incidence, combined with the current context of stagnated funding for related interventions, mean that many countries are not on track to reach the 2020 and 2030 global targets for reduction in incidence and mortality. With a growing population of people living with HIV, it will continue to be a major threat to public health for years to come. The pace of progress needs to be hastened by continuing to expand access to ART and increasing investments in proven HIV prevention initiatives that can be scaled up to have population-level impact

Prediction of breast self-examination in a sample of Iranian women: an application of the Health Belief Model

<p>Abstract</p> <p>Background</p> <p>Iranian women, many of whom live in small cities, have limited access to mammography and clinical breast examinations. Thus, breast self examination (BSE) becomes an important and necessary approach to detecting this disease in its early stages in order to limit its resultant morbidity and mortality. This study examined constructs arising from the Health Belief Model as predictors of breast self examination behavior in a sample of women living in Bandar Abbas, Iran.</p> <p>Methods</p> <p>This study was conducted in eight health centers located in Bandar Abbas, Iran. The sample consisted of 240 eligible women who were selected from referrals to the centers. The inclusion criteria were as follows: aged 30 years and over; and able to read and write Farsi. Women with breast cancer, who were pregnant, or breast feeding, were excluded from the study. Data were collected by using a self administered questionnaire which included demographic characteristics and Champion's Health Belief Model Scale. This instrument measures the concepts of disease susceptibility (3 items), seriousness (6 items), benefits (4 items), barriers (8 items) and self-efficacy (10 items).</p> <p>Results</p> <p>The subjects' mean age was 37.2 (SD = 6.1) years. Just under a third of the subjects (31.7%) had performed BSE in the past and 7.1% of them performed it at least monthly. Perceived benefits and perceived self-efficacy of the women who performed BSE were significantly higher compared with women who did not practice BSE (p < 0.03). Furthermore, perceived barriers were lower among those who had performed BSE (p < 0.001). Logistic regression analysis indicated that women who perceived fewer barriers (OR: 0.70, 95% CI: 0.63-0.77, p < 0.001) and had higher self-efficacy (OR: 1.08, 95% CI: 1.02-1.13, p = 0.003) were more likely to perform BSE (R²= 0.52).</p> <p>Conclusion</p> <p>Findings from this study indicated that perceived barriers and perceived self-efficacy could be predictors of BSE behavior among the sample of women. Therefore, BSE training programs that emphasize self-efficacy and address perceived barriers are recommended.</p

Deep Reinforcement Learning: An Overview

In recent years, a specific machine learning method called deep learning has gained huge attraction, as it has obtained astonishing results in broad applications such as pattern recognition, speech recognition, computer vision, and natural language processing. Recent research has also been shown that deep learning techniques can be combined with reinforcement learning methods to learn useful representations for the problems with high dimensional raw data input. This chapter reviews the recent advances in deep reinforcement learning with a focus on the most used deep architectures such as autoencoders, convolutional neural networks and recurrent neural networks which have successfully been come together with the reinforcement learning framework.Comment: Proceedings of SAI Intelligent Systems Conference (IntelliSys) 201

LNCS

A controller is a device that interacts with a plant. At each time point,it reads the plant’s state and issues commands with the goal that the plant oper-ates optimally. Constructing optimal controllers is a fundamental and challengingproblem. Machine learning techniques have recently been successfully applied totrain controllers, yet they have limitations. Learned controllers are monolithic andhard to reason about. In particular, it is difficult to add features without retraining,to guarantee any level of performance, and to achieve acceptable performancewhen encountering untrained scenarios. These limitations can be addressed bydeploying quantitative run-timeshieldsthat serve as a proxy for the controller.At each time point, the shield reads the command issued by the controller andmay choose to alter it before passing it on to the plant. We show how optimalshields that interfere as little as possible while guaranteeing a desired level ofcontroller performance, can be generated systematically and automatically usingreactive synthesis. First, we abstract the plant by building a stochastic model.Second, we consider the learned controller to be a black box. Third, we mea-surecontroller performanceandshield interferenceby two quantitative run-timemeasures that are formally defined using weighted automata. Then, the problemof constructing a shield that guarantees maximal performance with minimal inter-ference is the problem of finding an optimal strategy in a stochastic2-player game“controller versus shield” played on the abstract state space of the plant with aquantitative objective obtained from combining the performance and interferencemeasures. We illustrate the effectiveness of our approach by automatically con-structing lightweight shields for learned traffic-light controllers in various roadnetworks. The shields we generate avoid liveness bugs, improve controller per-formance in untrained and changing traffic situations, and add features to learnedcontrollers, such as giving priority to emergency vehicles