Explore, Exploit or Listen: Combining Human Feedback and Policy Model to Speed up Deep Reinforcement Learning in 3D Worlds

We describe a method to use discrete human feedback to enhance the performance of deep learning agents in virtual three-dimensional environments by extending deep-reinforcement learning to model the confidence and consistency of human feedback. This enables deep reinforcement learning algorithms to determine the most appropriate time to listen to the human feedback, exploit the current policy model, or explore the agent's environment. Managing the trade-off between these three strategies allows DRL agents to be robust to inconsistent or intermittent human feedback. Through experimentation using a synthetic oracle, we show that our technique improves the training speed and overall performance of deep reinforcement learning in navigating three-dimensional environments using Minecraft. We further show that our technique is robust to highly innacurate human feedback and can also operate when no human feedback is given

Data and analyses associated with "Dermal mycobacteriosis and warming sea surface temperatures are associated with elevated mortality of striped bass in Chesapeake Bay"

<div>The following data and analyses (R-code) are associated with the manuscript:</div><div><br></div><div>Groner ML, Hoenig JM, Pradel R, Choquet R, Vogelbein, WK, Gauthier, DT, Friedrichs MAM. Dermal mycobacteriosis and warming sea surface temperatures are associated with elevated mortality of striped bass in Chesapeake Bay. 2018 Ecology and Evolution</div><div><br></div><div><br></div><div><b>Data</b></div><div>1) SB recap data for esurge.txt: Dataset on striped bass mark-recapture from 2005-2013 as described in the manuscript. Formatted for analysis in ESURGE<br></div><div> $COV:tag- tag number of unique fish</div><div> H:year- year of study 1-2005, 2-2006, etc.</div><div> COV:rel_fl- fork length at the time of tagging</div><div>$COV:realage- age of fish at the time of tagging</div><div> RC: indicates of data is right-censored (i.e., fish was removed at capture)</div><div><br></div><div> *Note that this data was modified from the dataset available on dryad (https://datadryad.org/resource/doi:10.5061/dryad.f56v8/3), in order to fit the E-SURGE format.</div><div><br></div><div>2) SB Survival best-fit MMSMR.csv: Estimated survival of striped bass by year and disease state from the best-fit MMSMR model</div><div><br></div><div>3) SB state Transitions best-fit MMSMR.csv: Estimated transitions betwee disease states from the best-fit MMSMR model</div><div><br></div><div>4) SB Encounter rates best-fit MMSMR.csv: Estimated catchability of striped bass by year and disease state from the best-fit MMSMR model</div><div><br></div><div>5) SB Survival temp-dependent.csv: Survival rates of striped bass as a function of disease state and average summer SST</div><div><br></div><div>6) Environmental covariates.csv: yearly averages for average summer SST, #days > 25 C, #days < 5.1 mg/L Dissolved oxygen, average water flow</div><div><b><br></b></div><div><b>R-code:</b></div><div><div>1) Graph SB best-fit MMSMR output.R: graphs results from best fit MMSMR (not including environmental covariates) uses datasets 2,3,4 listed above</div><div><br></div><div>2) MMSMR results with environmental covariates.R: </div><div>graphs results from best model in table 3 (include environmental covariates of survival) uses datasets 5, 6 listed above</div><div><br></div><div>3) Temperature-dependent population projection.R: </div><div>projects and graphs a cohort of 10,000 3+ year old striped bass given different average summer SSTs</div></div><div><br></div><div><br></div

American lobster Recapture data- LIS - Epizootic shell disease

Data on recaptures only from American lobsters tagged and recaptured by the Millstone Environmental Lab in Millstone, CT. Data file includes: tag_num: unique tag identifier, rel_month: month of tagging, rel_day: day of tagging, rel_year: year of tagging, sex, rel_carapace_length: carapace length at tagging, male:1 if male, ovig: 1 if ovigerous female at tagging, female: 1 if non-ovigerous female at tagging, dal: days at large between tagging and release, disease_status: disease status at tagging (0 is healthy, 1 is mildly diseased (<10% of carapace with lesions), 2 is moderately diseased (10-50% of carapace with lesions), 3 is severely diseased (>50% of carapace with lesions)), recap_date: date a recapture. Data on animals that were tagged and not recaptured or were recaptured in other locations are not included in this dataset

Snow crab recapture data 2006-2008, Conception Bay

Snow crabs from Conception Bay that were recaptured after a tagging study initiated in 2006. DAL= days at large, M= disease status at tagging (0=healthy, 1=diseased), disease (bitter crab disease) was diagnosed by visual assessment of the carapace. Data on crabs that were not recaptured is excluded

Design of Substituted Imidazolidinylpiperidinylbenzoic Acids as Chemokine Receptor 5 Antagonists: Potent Inhibitors of R5 HIV‑1 Replication

The redesign of the previously reported thiophene-3-yl-methyl urea series, as a result of potential cardiotoxicity, was successfully accomplished, resulting in the identification of a novel potent series of CCR5 antagonists containing the imidazolidinylpiperidinyl scaffold. The main redesign criteria were to reduce the number of rotatable bonds and to maintain an acceptable lipophilicity to mitigate hERG inhibition. The structure–activity relationship (SAR) that was developed was used to identify compounds with the best pharmacological profile to inhibit HIV-1. As a result, five advanced compounds, <b>6d</b>, <b>6e</b>, <b>6i</b>, <b>6h</b>, and <b>6k</b>, were further evaluated for receptor selectivity, antiviral activity against CCR5 using (R5) HIV-1 clinical isolates, and in vitro and in vivo safety. On the basis of these results, <b>6d</b> and <b>6h</b> were selected for further development