Explore, Exploit or Listen: Combining Human Feedback and Policy Model to Speed up Deep Reinforcement Learning in 3D Worlds

We describe a method to use discrete human feedback to enhance the performance of deep learning agents in virtual three-dimensional environments by extending deep-reinforcement learning to model the confidence and consistency of human feedback. This enables deep reinforcement learning algorithms to determine the most appropriate time to listen to the human feedback, exploit the current policy model, or explore the agent's environment. Managing the trade-off between these three strategies allows DRL agents to be robust to inconsistent or intermittent human feedback. Through experimentation using a synthetic oracle, we show that our technique improves the training speed and overall performance of deep reinforcement learning in navigating three-dimensional environments using Minecraft. We further show that our technique is robust to highly innacurate human feedback and can also operate when no human feedback is given

EUREGIO data coverage on countries.

(DOCX)</p

Visualization and description of GF brokerage roles.

Note: The square is the broker, node color indicates group. Authors’ elaboration based on Gould and Fernandez [4].</p

Organizational network of the R&D section of a manufacturing company.

Organizational network of the R&D section of a manufacturing company.</p

Ranking of the top five regions in terms of largest difference between WNGF and backbone.

Ranking of the top five regions in terms of largest difference between WNGF and backbone.</p

mCherry expression in flat mounts of TM in AdPLAT-injected Eyes.

<p>Flatmount of anterior segments of mouse eyes from (<b>A</b>) animal injected with AdPLAT that showed robust mCherry expression (TA+/+ AdPLAT) (<b>B</b>) animal with minimal mCherry expression despite injection with AdPLAT (TA+/-AdPLAT), and (<b>C</b>) treatment-naïve mouse. (<b>D</b>) qRT-PCR quantification of transgenic PLAT expression in mouse eyes. Levels of PLAT are undetectable in all eyes except those showing robust mCherry expression (TA+/+ AdPLAT). CP = ciliary processes, TM = trabecular meshwork. Arrows indicate mCherry-positive cells.</p

Additional file 2: of Detection, prevalence, and transmission of avian hematozoa in waterfowl at the Arctic/sub-Arctic interface: co-infections, viral interactions, and sources of variation

This supporting file contains estimates of Leucocytozoon prevalence relative to co-infection with Haemoproteus (Table S13), estimated Haemoproteus prevalence relative to co-infection with Leucocytozoon (Table S14), estimated Haemoproteus prevalence relative to body condition (Table S15), and estimated Leucocytozoon prevalence relative to body condition (Table S16). (PDF 66 kb

Experimental Design.

<p>12 week-old C57BL/6 female mice received bilateral injections with 20 ul of triamcinolone acetonide (TA) suspension subconjunctivally. Animals were then divided into three treatment groups: 1) animals that also received unilateral intracameral injection with 2 ul of AdPLAT concurrently with TA injection while the contralateral eye remained uninjected, 2) animals that received bilateral intracameral injection with 2 ul of AdNull concurrently with TA injection, and 3) animals that received unilateral intracameral injection with 2 ul of AdPLAT one week after TA injection while the contralateral eye remained uninjected. Animals were sacrificed for outflow facility determination one week after AdPLAT or AdNull treatment.</p

Gene expression changes.

<p>Normalized fold change (mean ± SD) of (<b>A</b>) PAI-1 and (<b>B</b>) MMP-2 expression and normalized fold difference of (<b>C</b>) MMP-9 and (<b>D</b>) MMP-13 expression in TA eyes (n=10), TA+/+ AdPLAT eyes (n=9), and TA+/-AdPLAT eyes (n=10) from animals receiving AdPLAT one week after TA injection. Group means are significantly different (ANOVA PAI-1, p<0.01, MMP-2, p<0.001). Asterisks indicate differences on post hoc analysis, **p<0.01, ***p<0.001. No mRNA was detected for MMP-9 and MMP-13 in the TA or TA+/-AdPLAT groups.</p

Outflow facility in TA and AdPLAT concurrently treated mouse eyes.

<p>Mean ± SD outflow facility in eyes receiving TA alone (TA, n=22), eyes injected with AdPLAT showing robust mCherry expression (TA+/+ AdPLAT, n=12), eyes injected with AdPLAT showing minimal or no mCherry expression (TA+/-AdPLAT, n=8), and eyes injected with AdNull (TA+AdNull, n=30). Group means are significantly different (ANOVA, p<0.01). Asterisks indicate significant differences on post hoc analysis, *p<0.05, **p<0.01.</p