PRM-RL: Long-range Robotic Navigation Tasks by Combining Reinforcement Learning and Sampling-based Planning

We present PRM-RL, a hierarchical method for long-range navigation task completion that combines sampling based path planning with reinforcement learning (RL). The RL agents learn short-range, point-to-point navigation policies that capture robot dynamics and task constraints without knowledge of the large-scale topology. Next, the sampling-based planners provide roadmaps which connect robot configurations that can be successfully navigated by the RL agent. The same RL agents are used to control the robot under the direction of the planning, enabling long-range navigation. We use the Probabilistic Roadmaps (PRMs) for the sampling-based planner. The RL agents are constructed using feature-based and deep neural net policies in continuous state and action spaces. We evaluate PRM-RL, both in simulation and on-robot, on two navigation tasks with non-trivial robot dynamics: end-to-end differential drive indoor navigation in office environments, and aerial cargo delivery in urban environments with load displacement constraints. Our results show improvement in task completion over both RL agents on their own and traditional sampling-based planners. In the indoor navigation task, PRM-RL successfully completes up to 215 m long trajectories under noisy sensor conditions, and the aerial cargo delivery completes flights over 1000 m without violating the task constraints in an environment 63 million times larger than used in training.Comment: 9 pages, 7 figure

Resveratrol Prevents High Fluence Red Light-Emitting Diode Reactive Oxygen Species-Mediated Photoinhibition of Human Skin Fibroblast Migration.

BackgroundSkin fibrosis is a significant medical problem that leads to a functional, aesthetic, and psychosocial impact on quality-of-life. Light-emitting diode-generated 633-nm red light (LED-RL) is part of the visible light spectrum that is not known to cause DNA damage and is considered a safe, non-invasive, inexpensive, and portable potential alternative to ultraviolet phototherapy that may change the treatment paradigm of fibrotic skin disease.ObjectiveThe goal of our study was to investigate the how reactive oxygen species (ROS) free radicals generated by high fluence LED-RL inhibit the migration of skin fibroblasts, the main cell type involved in skin fibrosis. Fibroblast migration speed is increased in skin fibrosis, and we studied cellular migration speed of cultured human skin fibroblasts as a surrogate measure of high fluence LED-RL effect on fibroblast function. To ascertain the inhibitory role of LED-RL generated ROS on migration speed, we hypothesized that resveratrol, a potent antioxidant, could prevent the photoinhibitory effects of high fluence LED-RL on fibroblast migration speed.MethodsHigh fluence LED-RL generated ROS were measured by flow cytometry analysis using dihydrorhodamine (DHR). For purposes of comparison, we assessed the effects of ROS generated by hydrogen peroxide (H2O2) on fibroblast migration speed and the ability of resveratrol, a well known antioxidant, to prevent LED-RL and H2O2 generated ROS-associated changes in fibroblast migration speed. To determine whether resveratrol could prevent the high fluence LED-RL ROS-mediated photoinhibition of human skin fibroblast migration, treated cells were incubated with resveratrol at concentrations of 0.0001% and 0.001% for 24 hours, irradiated with high fluences LED-RL of 480, 640, and 800 J/cm2.ResultsHigh fluence LED-RL increases intracellular fibroblast ROS and decreases fibroblast migration speed. LED-RL at 480, 640 and 800 J/cm2 increased ROS levels to 132.8%, 151.0%, and 158.4% relative to matched controls, respectively. These LED-RL associated increases in ROS were prevented by pretreating cells with 0.0001% or 0.001% resveratrol. Next, we quantified the effect of hydrogen peroxide (H2O2)-associated ROS on fibroblast migration speed, and found that while H2O2-associated ROS significantly decreased relative fibroblast migration speed, pretreatment with 0.0001% or 0.001% resveratrol significantly prevented the decreases in migration speed. Furthermore, we found that LED-RL at 480, 640 and 800 J/cm2 decreased fibroblast migration speed to 83.0%, 74.4%, and 68.6% relative to matched controls, respectively. We hypothesized that these decreases in fibroblast migration speed were due to associated increases in ROS generation. Pretreatment with 0.0001% and 0.001% resveratrol prevented the LED-RL associated decreases in migration speed.ConclusionHigh fluence LED-RL increases ROS and is associated with decreased fibroblast migration speed. We provide mechanistic support that the decreased migration speed associated with high fluence LED-RL is mediated by ROS, by demonstrating that resveratrol prevents high fluence LED-RL associated migration speed change. These data lend support to an increasing scientific body of evidence that high fluence LED-RL has anti-fibrotic properties. We hypothesize that our findings may result in a greater understanding of the fundamental mechanisms underlying visible light interaction with skin and we anticipate clinicians and other researchers may utilize these pathways for patient benefit