Bipedal Walking Energy Minimization by Reinforcement Learning with Evolving Policy Parameterization

We present a learning-based approach for minimizing the electric energy consumption during walking of a passively-compliant bipedal robot. The energy consumption is reduced by learning a varying-height center-of-mass trajectory which uses efficiently the robots passive compliance. To do this, we propose a reinforcement learning method which evolves the policy parameterization dynamically during the learning process and thus manages to find better policies faster than by using fixed parameterization. The method is first tested on a function approximation task, and then applied to the humanoid robot COMAN where it achieves significant energy reduction. © 2011 IEEE

Bio­-inspired approaches to the control and modelling of an anthropomimetic robot

Introducing robots into human environments requires them to handle settings designed specifically for human size and morphology, however, large, conventional humanoid robots with stiff, high powered joint actuators pose a significant danger to humans. By contrast, “anthropomimetic” robots mimic both human morphology and internal structure; skeleton, muscles, compliance and high redundancy. Although far safer, their resultant compliant structure presents a formidable challenge to conventional control. Here we review, and seek to address, characteristic control issues of this class of robot, whilst exploiting their biomimetic nature by drawing upon biological motor control research. We derive a novel learning controller for discovering effective reaching actions created through sustained activation of one or more muscle synergies, an approach which draws upon strong, recent evidence from animal and humans studies, but is almost unexplored to date in musculoskeletal robot literature. Since the best synergies for a given robot will be unknown, we derive a deliberately simple reinforcement learning approach intended to allow their emergence, in particular those patterns which aid linearization of control. We also draw upon optimal control theories to encourage the emergence of smoother movement by incorporating signal dependent noise and trial repetition. In addition, we argue the utility of developing a detailed dynamic model of a complete robot and present a stable, physics-­‐‑based model, of the anthropomimetic ECCERobot, running in real time with 55 muscles and 88 degrees of freedom. Using the model, we find that effective reaching actions can be learned which employ only two sequential motor co-­‐‑activation patterns, each controlled by just a single common driving signal. Factor analysis shows the emergent muscle co-­‐‑activations can be reconstructed to significant accuracy using weighted combinations of only 13 common fragments, labelled “candidate synergies”. Using these synergies as drivable units the same controller learns the same task both faster and better, however, other reaching tasks perform less well, proportional to dissimilarity; we therefore propose that modifications enabling emergence of a more generic set of synergies are required. Finally, we propose a continuous controller for the robot, based on model predictive control, incorporating our model as a predictive component for state estimation, delay-­‐‑ compensation and planning, including merging of the robot and sensed environment into a single model. We test the delay compensation mechanism by controlling a second copy of the model acting as a proxy for the real robot, finding that performance is significantly improved if a precise degree of compensation is applied and show how rapidly an un-­‐‑compensated controller fails as the model accuracy degrades

ロボティクスのための深層強化学習による運動シナジー発現と冗長性定量化に関する研究

Tohoku University林部充宏課

Upper limb movement control after stroke and in healthy ageing: does intensive upper limb neurorehabilitation improve motor control and reduce motor impairment in the chronic phase of stroke?

Stroke affects people of all ages, but many are in the elderly population. 75% of stroke survivors have residual upper limb motor impairment and resultant disability. This thesis firstly examines upper limb motor control in chronic stroke. Evidence is emerging that high dose, high intensity complex neurorehabilitation interventions in chronic stroke patients produce unprecedented gains on clinical outcome scores of motor impairment, function and activity. But whether these clinical improvements represent behavioural repair or merely behavioural compensation remains undetermined. To address this question, upper limb movement kinematics, strength and joint range and clinical scores were measured in 52 chronic stroke patients before and after an intensive three-week treatment intervention. 29 chronic stroke patients who had not undergone treatment were similarly assessed, three-weeks apart. Significant improvements in motor control, arm strength and joint range in addition to gains on clinical scores were observed in the impaired arm of the intervention group. Crucially, changes in motor control occurred independently of changes in strength and joint range. Improvements in motor control were retained in a cohort of 28 patients in the intervention group, also assessed 6-weeks and 6-months after treatment had ended, demonstrating persistent changes in motor behaviour. These results suggest that behavioural restitution has occurred. Secondly, knowledge of the effects of normal healthy ageing on upper limb motor control is essential to informing research and delivery of clinical services. To this end, movement kinematics were measured in both arms of 57 healthy adults aged 22 to 82 years. A decline in motor control was observed as age increased, particularly in the non-dominant arm. However, motor control in healthy adults of all ages remained significantly better than in chronic stroke patients pre- and post-intervention. This thesis provides new evidence that treatment-driven improvements in motor control are achievable in the chronic post-stroke upper limb, which strongly suggests that motor control should remain a therapeutic target well beyond the current three to six-month post-stroke window. It will inform the continued development and delivery of high dose, high intensity upper limb neurorehabilitation treatment interventions for stroke patients of all ages

Dose in stroke rehabilitation trials

Background: the dose and the length of rehabilitative interventions for optimal motor recovery after stroke are unknown. Dose optimization studies are required as precursors to efficacy trials, but are rarely conducted in stroke rehabilitation research. Objective: to overcome the knowledge gap on appropriate dose and length of rehabilitative interventions guiding the implementation of novel effective approaches to dose optimization in stroke rehabilitation research. Method: two systematic reviews on dose optimization in exercise-based training and pharmaceutical clinical research guided the development of a new approach to dose-finding suitable for physical interventions. The feasibility of a novel phase I 3+3 rule-based, outcome-adaptive dose-finding design was assessed with stroke survivors with moderate upper limb paresis. Moreover, the feasibility of a repetitive assessment procedure to identify the appropriate length of motor interventions was explored in stroke rehabilitation research. Results: the first literature review showed a lack of reliable approaches to dose optimization in exercise-based training. The review of pharmaceutical research highlighted dose optimization “gold” standard approaches, and helped in devising the dose-finding study for physical intervention. The dose-finding study was feasible using the applied model-task intervention. Preliminary explorations on the dose-response relationship were possible indicating a maximum tolerable dose and a potential recommended dose of 209 and 162 repetitions respectively of the applied intervention-task. The repetitive assessment procedure was found feasible in a clinical efficacy stroke rehabilitative trial. The repetitive assessment procedure provided relevant data on the therapy effect over-time showing that more than six weeks of the applied upper limb intervention may be necessary to reach maximal therapy effects. Whereas, five weeks of intervention appeared enough to exploit therapy effects for the lower limb. Conclusions: results are promising on identifying relevant dose and protocol endpoints implementing dose-finding and repetitive assessments approaches in stroke rehabilitation. Further confirmative data are needed to validate these findings

The design of active workspaces

Thesis (S.M.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2001.Includes bibliographical references (p. 89-93).This thesis investigates the creation of context-specific environments designed to assist people engaged in productive tasks. These Active Workspaces are an emerging class of applications that combine interaction design, innovative input and output techniques and multi-media coaching methodologies to teach people to create things in the physical domain. The design and implementation of two systems that exemplify the features, process and goals of Active Workspace design are detailed and evaluated. CounterActive is an interactive kitchen counter that guides users of varying age and skill level through the preparation of several recipes. CounterActive enlivens the experience of cooking with instructional videos, illustrative photos, entertaining music and sounds. Origami Desk is an exhibit that teaches people how to fold paper into beautiful boxes and cranes. The desk not only projects animations and videos for the user to mimic, but also monitors the folding of the paper to enable performance feedback. These project descriptions are accompanied by generalized principles for the design of Active Workspaces, process guidelines, and as well as analysis of relevant technologies and ruminations on possible future applications in this arena.by Wendy Guang-wen Ju.S.M

Hollywood superheroes : the aesthetics of comic book to film adaptation

This thesis develops a theoretically-informed approach with which to analyse the aesthetics of the adaptation of superhero comic books into blockbuster films. Pervasive modes of thinking present superhero blockbusters as artistically degraded products that are not worthy of aesthetic analysis. I demonstrate that exploring the ways in which superhero blockbusters adapt comic book style and form reveals aesthetic sophistication and multiplicities of meaning. Engaging with comic book and film history also enables me to identify ways in which superhero blockbusters have contributed to the development of Hollywood’s blockbuster filmmaking paradigm. My approach combines models and concepts from studies of adaptation that employ poststructuralist theory. This theoretical framework explains transformations that content may undergo as it is adapted between the different forms available to comics and film, and enables examination of dialogues occurring in the vast networks of intertexts in which superhero blockbusters are situated. After my review of literature establishes the thesis’ theoretical underpinnings, my chapters undertake close textual analysis of three distinct case studies. The selection of case studies allows me to continue to develop my approach by examining different superhero archetypes, alongside significant contexts, trends and technologies that impact Hollywood blockbusters. Chapter one looks at the first superhero blockbuster, Superman: The Movie (1978). I begin by outlining, and exploring relations between, the range of Superman texts released prior to the film. Doing so reveals the qualities of the intertextual networks that comprise a superhero franchise. I then analyse the strategies that Superman: The Movie deploys to adapt and enter the network of Superman texts, before situating the film in the context of the emerging blockbuster paradigm in 1970s Hollywood. Chapters two and three analyse films produced in the twenty-first century, as superhero blockbusters gained a central position in Hollywood production. Chapter two evaluates the aesthetics of the Spider-Man trilogy (2002, 2004 and 2007) in relation to two contexts that are often considered to have facilitated the superhero blockbuster’s twenty-first century success: the increasing use and sophistication of digital filmmaking technologies in Hollywood, and the contemporary sociopolitical climate. Looking at the representation of bodies and space elucidates the ways in which the films incorporate digital filmmaking technologies into their adaptive practices and offer a sociopolitical commentary. Chapter three examines the strategies that films produced by Marvel Studios, with particular focus on team film The Avengers (2012), deploy to adapt the model of seriality that superhero comic books use to interconnect multiple series in a shared diegesis. The analysis focuses on ways in which The Avengers uses bodies and space to compress the expansive diegetic universe into a single film, and interrogates how these strategies shape the film’s sociopolitical meanings. My case studies demonstrate that the approach developed in this thesis illuminates the complex and equivocal meanings that the adaptive practices of superhero blockbusters generate