Prioritizing starting states for reinforcement learning

Online, off-policy reinforcement learning algorithms are able to use an experience memory to remember and replay past experiences. In prior work, this approach was used to stabilize training by breaking the temporal correlations of the updates and avoiding the rapid forgetting of possibly rare experiences. In this work, we propose a conceptually simple framework that uses an experience memory to help exploration by prioritizing the starting states from which the agent starts acting in the environment, importantly, in a fashion that is also compatible with on-policy algorithms. Given the capacity to restart the agent in states corresponding to its past observations, we achieve this objective by (i) enabling the agent to restart in states belonging to significant past experiences (e.g., nearby goals), and (ii) promoting faster coverage of the state space through starting from a more diverse set of states. While, using a good priority measure to identify significant past transitions, we expect case (i) to more considerably help exploration in certain domains (e.g., sparse reward tasks), we hypothesize that case (ii) will generally be beneficial, even without any prioritization. We show empirically that our approach improves learning performance for both off-policy and on-policy deep reinforcement learning methods, with most notable gains in highly sparse reward tasks

Mycobacterial Disease Overlooked in a Frail Diabetic Male Treated for Pneumonia

Mycobacterial disease has been characterized to cause human illness since the 1950’s but practitioners whom address pulmonary disease can overlook its insidious presentation. This case report documents the presentation of a mycobacterial illness treated as a severe pneumonia. This is the case of a critically ill 43-year-old male, with poorly treated diabetes who presented with severe, long-standing respiratory distress in DKA. He had a history of weight loss, night sweats, decreased oral intake, and weakness for about a month. He was treated for euglycemia and his pulmonary disease was characterized as a bacterial pneumonia. Common pathogens were ruled out in an RIDP and he was treated with broad-spectrum antibiotics and discharged. On hospital follow up, careful examination of imaging report and history taking consisted with a timely screening for TB. A quantiferon gold test was positive and our patient was admitted to isolation on our inpatient service for evaluation. His AFB smear being positive, he was started on RIPE therapy. He was discharged with three negative sputum smears on the same therapy with cultures pending final characterization. This report documents a common treatment algorithm, based on a differential focused on typical culprits for pulmonary infection. Latent mycobacterial infections in our community are on the rise, especially in individuals whom are immunocompromised such as HIV+ or diabetics. First, these individuals must be identified using a screening tool such as a questionnaire. Second, obtaining screening labwork, even in non-compliant individuals are easily obtainable. Quantiferon gold assays could be administered to those identified as high risk even with no previous testing or prior negative test results. As these tests result, evaluation with a chest X-ray will stratify our groups to either further evaluation for latent mycobacterial infections or for treatment based on a positive chest X-ray. There needs to be a focus on identifying patients at risk for exposure to TB and providing diagnosis and treatment for latent infection or active disease. This will decrease the spread of tuberculosis

A collection of wet beam models for wave–ice interaction

Theoretical models for the prediction of decay rate and dispersion process of gravity waves traveling into an integrated ice cover expanded over a long way are introduced. The term “wet beam” is chosen to refer to these models as they are developed by incorporating water-based damping and added mass forces. Presented wet beam models differ from each other according to the rheological behavior considered for the ice cover. Two-parameter viscoelastic solid models accommodating Kelvin–Voigt (KV) and Maxwell mechanisms along with a one-parameter elastic solid model are used to describe the rheological behavior of the ice layer. Quantitative comparison between the landfast ice field data and model predictions suggests that wet beam models, adopted with both KV and Maxwell mechanisms, predict the decay rate more accurately compared to a dry beam model. Furthermore, the wet beam models, adopted with both KV and Maxwell mechanisms, are found to construct decay rates of disintegrated ice fields, though they are built for a continuous ice field. Finally, it is found that wet beam models can accurately construct decay rate curves of freshwater ice, though they cannot predict the dispersion process of waves accurately. To overcome this limitation, three-parameter solid models, termed standard linear solid (SLS) mechanisms, are suggested to be used to re-formulate the dispersion relationship of wet beam models, which were seen to construct decay rates and dispersion curves of freshwater ice with an acceptable level of accuracy. Overall, the two-parameter wet beam dispersion relationships presented in this research are observed to predict decay rates and dispersion process of waves traveling into actual ice covers, though three-parameter wet beam models were seen to reconstruct the those of freshwater ice formed in a wave flume. The wet beam models presented in this research can be implemented in spectral models on a large geophysical scale.</p

Lorentz Symmetry in QFT on Quantum Bianchi I Space-Time

We develop the quantum theory of a scalar field on LQC Bianchi I geometry. In particular, we focus on single modes of the field: the evolution equation is derived from the quantum scalar constraint, and it is shown that the same equation can be obtained from QFT on an "classical" effective geometry. We investigate the dependence of this effective space-time on the wavevector of the mode (which could in principle generate a deformation in local Lorentz-symmetry), focusing our attention on the dispersion relation. We prove that when we disregard backreaction no Lorentz-violation is present, despite the effective metric being different than the classical Bianchi I one. A preliminary analysis of the correction due to inclusion of backreaction is briefly discussed in the context of Born-Oppenheimer approximation.Comment: 14 pages, v3. Corrected a reference in the bibliograph

Prevalence of human respiratory syncytial virus circulating in Iran

Respiratory syncytial virus (RSV) is a leading cause of acute respiratory infection during early childhood and is associated with a great burden on patients, parents, and society. While no treatment is yet available, results from recent phase 2 clinical trials of cell-entry inhibitors and RSV vaccines are promising. To prepare for introduction of these novel therapeutics, good understanding of its molecular epidemiology and continuous RSV surveillance data are necessary. This paper provides an overview of RSV prevalence and genotype distribution in Iran from 1996 to 2013. This meta-analysis includes 21 published studies. In total, 775 (18.7) of 4140 respiratory specimens were positive for RSV infection. The male-female ratio of RSV-positive patients was 1.5:1. Significant peaks of RSV infection were detected during the cold season (November-March). RSV infection was mainly observed in patients <2 years of age. Phylogenetic studies showed that genotypes GA1, GA2, GA5, and BA co-circulated in Iran in 2007-2013. This review highlights the necessity of introducing standard molecular surveillance programs to inform the epidemiological, clinical, and pathological characteristics of various RSV genotypes. Improved understanding of the molecular epidemiology will be useful for development of novel RSV therapeutics. © 2015 King Saud Bin Abdulaziz University for Health Sciences

Transcranial alternating current stimulation (tACS) mechanisms and protocols

Perception, cognition and consciousness can be modulated as a function of oscillating neural activity, while ongoing neuronal dynamics are influenced by synaptic activity and membrane potential. Consequently, transcranial alternating current stimulation (tACS) may be used for neurological intervention. The advantageous features of tACS include the biphasic and sinusoidal tACS currents, the ability to entrain large neuronal populations, and subtle control over somatic effects. Through neuromodulation of phasic, neural activity, tACS is a powerful tool to investigate the neural correlates of cognition. The rapid development in this area requires clarity about best practices. Here we briefly introduce tACS and review the most compelling findings in the literature to provide a starting point for using tACS. We suggest that tACS protocols be based on functional brain mechanisms and appropriate control experiments, including active sham and condition blinding

New multi-proxy record shows potential impacts of precipitation on the rise and ebb of Bronze Age and imperial Persian societies in southeastern Iran

The Achaemenids and Sasanian ‘Persian’ Empires were significant political, economic, and social forces in the Late Bronze Age and Late Antiquity Eurasia, respectively, which have left marks on the heritage of the Mediterranean and Middle Eastern world. While attention is often focused on military and political conditions when discussing the prosperity and decline of these imperial powers, their realms, which crossed a variety of environmental settings, were highly dependent on the predictability of rainfall that drove agriculture and effective provisioning. Here, we present a multi-proxy sedimentological, geochemical, and palynological record from a 2.5-m long peat deposit near the excavation site in Konar Sandal near Jiroft in southeastern Iran, covering 4000-850 cal yr BP. Around 3950 cal yr BP a wet period prevailed based on elemental ratios, stable C isotope, pollen, and diagnostic lipids. Between 3900 and 3300 cal yr BP, wet/semi-wet conditions developed with the appearance of Cerealia-type pollen. Dry and windy conditions followed (ca. 3300-2900 cal yr BP), which coincided with the Siberian anticyclones and climatic shifts developing in the Eastern Mediterranean region. Consequently, the Bronze Age settlements around Jiroft, dependent on agriculture, underwent a steady decline. A prolonged wet period followed (ca. 2900-2300 cal yr BP) with the abundance of Sparganium-type pollen and the aquatic lipid proxy (Paq). This change coincided with intensive agricultural practices and the flourishing of the powerful Median and Achaemenid empires. The shift to high Ti/Al ratios coeval with the lowest δ13COM values suggests an increase in aeolian activity and dry conditions ca. 2100-1650 cal yr BP. The Jiroft valley again experienced wet conditions between 1550 and 1300 cal yr BP, which overlapped with the economic prosperity of the middle to late Sasanian empire. The paleoenvironmental reconstruction indicates that wet periods and intensive agriculture coincide with the Persian empires' zenith, political influence, and economic affluence. Therefore, contextualized and detailed paleoenvironmental records are desirable to explore the interplay of political and climatic factors in the development and fragmentation of the ancient settlements and imperial powers in Eurasian history.1. Introduction 2. Geographical setting 3. Materials and methods 3.1. Sampling, magnetic susceptibility, and grain size analysis 3.2. Radiocarbon analyses 3.3. Elemental and mineralogical analyses 3.4. C/N and stable isotope analyses 3.4.1. Lipid extraction 3.5. Pollen analysis 4. Results 4.1. Age-depth model and chronology 4.2. Units 4.2.1. Unit 1 (U 1; 250-189 cm; ca. 4000-3550 cal yr BP) 4.2.2. Unit 2 (U 2; 189-164 cm; ca. 3550-3300 cal yr BP) 4.2.3. Unit 3 (U 3; 164-134 cm; ca. 3300-2900 cal yr BP) 4.2.4. Unit 4 (U 4; 134-106 cm; ca. 2900-2300 cal yr BP) 4.2.5. Unit 5 (U 5; 106-61 cm; ca. 2300-1550 cal yr BP) 4.2.6. Unit 6 (U 6; 61-10 cm; ca. 1550-850 cal yr BP) 5. Discussion 5.1. Paleoclimate preceding the decline of the Jiroft civilization 5.2. The Late Bronze Age decline in Jiroft 5.3. Rise and fall of the Persian Empire 5.3.1. The pre-Islamic period and recent times 6. Conclusion

Integrated approach for zonation of a mid-Cenomanian carbonate reservoir in a sequence stratigraphic framework

The mid-Cenomanian Mishrif Formation (Fm.) is considered as one of the most important rudist-bearing reservoir horizons in the Sirri Oil Fields of the Persian Gulf. Due to the general heterogeneity of carbonate reservoirs, the use of an integrated approach is helpful for investigating porosity and permeability distribution along with recognizing controlling pore system factors in the reservoir. Thus, for the reservoir characterization of the Mishrif Fm., an integrated approach including facies analysis, diagenetic history and sequence stratigraphic analysis is considered. Detailed petrographic studies showed a total of eight microfacies and seven facies belts, related to inner ramp to the basin of a homoclinal carbonate ramp. Humid climatic condition and tectonic activity, associated with eustatic sea-level fluctuations during the mid-Cretaceous, led to meteoric diagenesis of the Mishrif carbonates during subaerial exposures (mid-Cenomanian and Cenomanian-Turonian disconformities). General diagenetic overprints and modifications include micritization, cementation, dissolution, compaction, dolomitization, pyritization and fracturing. Considering this reservoir in the sequence stratigraphic framework reveals that the reservoir zones development is basically related to the Cenomanian–Turonian sequence boundary, recognized in the three studied wells, and also to the mid-Cenomanian boundary, identified only in one well. In addition, pore system properties were inspected by differentiation of Hydraulic Flow Units (HFUs) within the reservoir. The identified flow units, based on their capability for fluid flow, can be classified into four main rock types with very high- (HFUD), high- (HFUC), medium- (HFUB) and low-quality (HFUA). Accordingly, this study shows that the main part of the Mishrif Reservoir is affected by diagenetic processes related to subaerial exposures, resulting in zones with higher storage capacity and fluid flow rates. So, the study of depositional and diagenetic characteristics of the Mishrif carbonates in the sequence stratigraphy framework is essential to unravel the reservoir heterogeneity, and to describe the reservoir zones and their distribution in the field and regional scale. In addition, observed changes in the thickness of hydrocarbon column are attributed to the different location of the studied wells on the anticline structures, which show a tilted oil-water contact with a slope to the North