Improving Asynchronous Advantage Actor Critic with a More Intelligent Exploration Strategy

We propose a simple and efficient modification to the Asynchronous Advantage Actor Critic (A3C) algorithm that improves training. In 2016 Google’s DeepMind set a new standard for state-of-theart reinforcement learning performance with the introduction of the A3C algorithm. The goal of this research is to show that A3C can be improved by the use of a new novel exploration strategy we call “Follow then Forage Exploration” (FFE). FFE forces the agents to follow the best known path at the beginning of a training episode and then later in the episode the agent is forced to “forage” and explores randomly. In tests against A3C implemented using OpenAI’s Universe-Starter-Agent, FFE was able to show on average that it reached the maximum score faster

Earthquake forecasting and its verification

No proven method is currently available for the reliable short time prediction of earthquakes (minutes to months). However, it is possible to make probabilistic hazard assessments for earthquake risk. These are primarily based on the association of small earthquakes with future large earthquakes. In this paper we discuss a new approach to earthquake forecasting. This approach is based on a pattern informatics (PI) method which quantifies temporal variations in seismicity. The output is a map of areas in a seismogenic region (``hotspots'') where earthquakes are forecast to occur in a future 10-year time span. This approach has been successfully applied to California, to Japan, and on a worldwide basis. These forecasts are binary--an earthquake is forecast either to occur or to not occur. The standard approach to the evaluation of a binary forecast is the use of the relative operating characteristic (ROC) diagram, which is a more restrictive test and less subject to bias than maximum likelihood tests. To test our PI method, we made two types of retrospective forecasts for California. The first is the PI method and the second is a relative intensity (RI) forecast based on the hypothesis that future earthquakes will occur where earthquakes have occurred in the recent past. While both retrospective forecasts are for the ten year period 1 January 2000 to 31 December 2009, we performed an interim analysis 5 years into the forecast. The PI method out performs the RI method under most circumstances.Comment: 10(+1) pages, 5 figures, 2 tables. Submitted to Nonlinearl Processes in Geophysics on 5 August 200

Space-Time Clustering and Correlations of Major Earthquakes

Earthquake occurrence in nature is thought to result from correlated elastic stresses, leading to clustering in space and time. We show that occurrence of major earthquakes in California correlates with time intervals when fluctuations in small earthquakes are suppressed relative to the long term average. We estimate a probability of less than 1% that this coincidence is due to random clustering.Comment: 5 pages, 3 figures. Submitted to PR

AC voltage sag-swell compensator based on unified non-inverting and inverting output voltage AC chopper

This paper proposes a unified AC chopper topology that can generate both non-inverting and inverting AC voltages. It is therefore suitable for compensating utility voltage sag-swell in domestic consumer and industry infeeds. The proposed converter is able to save on the passive device footprint and improve dynamic performance compared to other similar candidates. The topology derivation, operating principle and performance evaluation are analysed in this paper. Simulation results are presented to assess the proposed scheme with in-phase and out-phase AC voltages generated from a fixed grid input to compensate the voltage sag-swell disturbances in the utility

Electroencephalogram of Healthy Horses During Inhaled Anesthesia.

BackgroundPrevious study of the diagnostic validity of electroencephalography (EEG) to detect abnormalities in equine cerebral cortical function relied on the administration of various drugs for sedation, induction, and maintenance of general anesthesia but used identical criteria to interpret recordings.ObjectivesTo determine the effects of 2 inhalation anesthetics on the EEG of healthy horses.AnimalsSix healthy horses.MethodsProspective study. After the sole administration of one of either isoflurane or halothane at 1.2, 1.4, and 1.6 times the minimum alveolar concentration, EEG was recorded during controlled ventilation, spontaneous ventilation, and nerve stimulation.ResultsBurst suppression was observed with isoflurane, along with EEG events that resembled epileptiform discharges. Halothane results were variable between horses, with epileptiform-like discharges and bursts of theta, alpha, and beta recorded intermittently. One horse died and 2 were euthanized as the result of anesthesia-related complications.Conclusions and clinical importanceThe results of this study indicate that the effects of halothane and isoflurane on EEG activity in the normal horse can be quite variable, even when used in the absence of other drugs. It is recommended that equine EEG be performed without the use of these inhalation anesthetics and that general anesthesia be induced and maintained by other contemporary means

Qualitative and Quantitative Characteristics of the Electroencephalogram in Normal Horses during Administration of Inhaled Anesthesia.

BackgroundThe effects of anesthesia on the equine electroencephalogram (EEG) after administration of various drugs for sedation, induction, and maintenance are known, but not that the effect of inhaled anesthetics alone for EEG recording.ObjectiveTo determine the effects of isoflurane and halothane, administered as single agents at multiple levels, on the EEG and quantitative EEG (qEEG) of normal horses.AnimalsSix healthy horses.MethodsProspective study. Digital EEG with video and quantitative EEG (qEEG) were recorded after the administration of one of the 2 anesthetics, isoflurane or halothane, at 3 alveolar doses (1.2, 1.4 and 1.6 MAC). Segments of EEG during controlled ventilation (CV), spontaneous ventilation (SV), and with peroneal nerve stimulation (ST) at each MAC multiple for each anesthetic were selected, analyzed, and compared. Multiple non-EEG measurements were also recorded.ResultsSpecific raw EEG findings were indicative of changes in the depth of anesthesia. However, there was considerable variability in EEG between horses at identical MAC multiples/conditions and within individual horses over segments of a given epoch. Statistical significance for qEEG variables differed between anesthetics with bispectral index (BIS) CV MAC and 95% spectral edge frequency (SEF95) SV MAC differences in isoflurane only and median frequency (MED) differences in SV MAC with halothane only.Conclusions and clinical importanceUnprocessed EEG features (background and transients) appear to be beneficial for monitoring the depth of a particular anesthetic, but offer little advantage over the use of changes in mean arterial pressure for this purpose