LETTER Robot Task Learning based on Reinforcement Learning in Virtual Space

Abstract − As a novel learning method, reinforced learning by which a robot acquires control rules through trial and error has gotten a lot of attention. However, it is quite difficult for robots to acquire control rules by reinforcement learning in real space because many learning trials are needed to achieve the control rules; the robot itself may lose control, or there may be safety problems with the control objects. In this paper, we propose a method in which a robot in real space learns a virtual task; then the task is transferred from virtual to real space. The robot eventually acquires the task in a real environment. We show that a real robot can acquire a task in virtual space with an input device by an example of an inverted pendulum. Next, we verify availability that the acquired task in virtual space can be applied to a real world task. We emphasize the utilization of virtual space to effectively obtain the real world task

Pharmacological activity of C10-substituted analogs of the high-affinity kainate receptor agonist dysiherbaine

Kainate receptor antagonists have potential as therapeutic agents in a number of neuropathologies. Synthetic modification of the convulsant marine toxin neodysiherbaine A (NDH) previously yielded molecules with a diverse set of pharmacological actions on kainate receptors. Here we characterize three new synthetic analogs of NDH that contain substituents at the C10 position in the pyran ring of the marine toxin. The analogs exhibited high-affinity binding to the GluK1 (GluR5) subunit and lower affinity binding to GluK2 (GluR6) and GluK3 (GluR7) subunits in radioligand displacement assays with recombinant kainate and AMPA receptors. As well, the natural toxin NDH exhibited ∼100-fold selectivity for GluK2 over GluK3 subunits, which was attributable to the C8 hydroxyl group in NDH. We used molecular dynamic simulations to determine the specific interactions between NDH and residues within the ligand-binding domains of these two kainate receptor subunits that contribute to the divergent apparent affinities for the compound. These data demonstrate that interactions with the GluK1 subunit are preserved in analogs with substitutions at C10 in NDH and further reveal the determinants of selectivity and pharmacological activity of molecules acting on kainate receptor subunits, which could aid in design of additional compounds that target these receptors.peerReviewe