Robot-assisted gait self-training: assessing the level achieved

This paper presents the technological status of robot-assisted gait self-training under real clinical environment conditions. A successful rehabilitation after surgery in hip endoprosthetics comprises self-training of the lessons taught by physiotherapists. While doing this, immediate feedback to the patient about deviations from the expected physiological gait pattern during training is important. Hence, the Socially Assistive Robot (SAR) developed for this type of training employs task-specific, user-centered navigation and autonomous, real-time gait feature classification techniques to enrich the self-training through companionship and timely corrective feedback. The evaluation of the system took place during user tests in a hospital from the point of view of technical benchmarking, considering the therapists’ and patients’ point of view with regard to training motivation and from the point of view of initial findings on medical efficacy as a prerequisite from an economic perspective. In this paper, the following research questions were primarily considered: Does the level of technology achieved enable autonomous use in everyday clinical practice? Has the gait pattern of patients who used additional robot-assisted gait self-training for several days been changed or improved compared to patients without this training? How does the use of a SAR-based self-training robot affect the motivation of the patients

Instrumentation and validation of a robotic cane for transportation and fall prevention in patients with affected mobility

Dissertação de mestrado integrado em Engenharia Física, (especialização em Dispositivos, Microssistemas e Nanotecnologias)O ato de andar é conhecido por ser a forma primitiva de locomoção do ser humano, sendo que este traz muitos benefícios que motivam um estilo de vida saudável e ativo. No entanto, há condições de saúde que dificultam a realização da marcha, o que por consequência pode resultar num agravamento da saúde, e adicionalmente, levar a um maior risco de quedas. Nesse sentido, o desenvolvimento de um sistema de deteção e prevenção de quedas, integrado num dispositivo auxiliar de marcha, seria essencial para reduzir estes eventos de quedas e melhorar a qualidade de vida das pessoas. Para ultrapassar estas necessidades e limitações, esta dissertação tem como objetivo validar e instrumentar uma bengala robótica, denominada Anti-fall Robotic Cane (ARCane), concebida para incorporar um sistema de deteção de quedas e um mecanismo de atuação que possibilite a prevenção de quedas, ao mesmo tempo que assiste a marcha. Para esse fim, foi realizada uma revisão do estado da arte em bengalas robóticas para adquirir um conhecimento amplo e aprofundado dos componentes, mecanismos e estratégias utilizadas, bem como os protocolos experimentais, principais resultados, limitações e desafios em dispositivos existentes. Numa primeira fase, foi estipulado o objetivo de: (i) adaptar a missão do produto; (ii) estudar as necessidades do consumidor; e (iii) atualizar as especificações alvo da ARCane, continuação do trabalho de equipa, para obter um produto com design e engenharia compatível com o mercado. Foi depois estabelecida a arquitetura de hardware e discutidos os componentes a ser instrumentados na ARCane. Em seguida foram realizados testes de interoperabilidade a fim de validar o funcionamento singular e coletivo dos componentes. Relativamente ao controlo de movimento, foi desenvolvido um sistema inovador, de baixo custo e intuitivo, capaz de detetar a intenção do movimento e de reconhecer as fases da marcha do utilizador. Esta implementação foi validada com seis voluntários saudáveis que realizaram testes de marcha com a ARCane para testar sua operabilidade num ambiente de contexto real. Obteve-se uma precisão de 97% e de 90% em relação à deteção da intenção de movimento e ao reconhecimento da fase da marcha do utilizador. Por fim, foi projetado um método de deteção de quedas e mecanismo de prevenção de quedas para futura implementação na ARCane. Foi ainda proposta uma melhoria do método de deteção de quedas, de modo a superar as limitações associadas, bem como a proposta de dispositivos de deteção a serem implementados na ARCane para obter um sistema completo de deteção de quedas.The act of walking is known to be the primitive form of the human being, and it brings many benefits that motivate a healthy and active lifestyle. However, there are health conditions that make walking difficult, which, consequently, can result in worse health and, in addition, lead to a greater risk of falls. Thus, the development of a fall detection and prevention system integrated with a walking aid would be essential to reduce these fall events and improve people quality of life. To overcome these needs and limitations, this dissertation aims to validate and instrument a cane-type robot, called Anti-fall Robotic Cane (ARCane), designed to incorporate a fall detection system and an actuation mechanism that allow the prevention of falls, while assisting the gait. Therefore, a State-of-the-Art review concerning robotic canes was carried out to acquire a broad and in-depth knowledge of the used components, mechanisms and strategies, as well as the experimental protocols, main results, limitations and challenges on existing devices. On a first stage, it was set an objective to (i) enhance the product's mission statement; (ii) study the consumer needs; and (iii) update the target specifications of the ARCane, extending teamwork, to obtain a product with a market-compatible design and engineering that meets the needs and desires of the ARCane users. It was then established the hardware architecture of the ARCane and discussed the electronic components that will instrument the control, sensory, actuator and power units, being afterwards subjected to interoperability tests to validate the singular and collective functioning of cane components altogether. Regarding the motion control of robotic canes, an innovative, cost-effective and intuitive motion control system was developed, providing user movement intention recognition, and identification of the user's gait phases. This implementation was validated with six healthy volunteers who carried out gait trials with the ARCane, in order to test its operability in a real context environment. An accuracy of 97% was achieved for user motion intention recognition and 90% for user gait phase recognition, using the proposed motion control system. Finally, it was idealized a fall detection method and fall prevention mechanism for a future implementation in the ARCane, based on methods applied to robotic canes in the literature. It was also proposed an improvement of the fall detection method in order to overcome its associated limitations, as well as detection devices to be implemented into the ARCane to achieve a complete fall detection system

A Survey of Applications and Human Motion Recognition with Microsoft Kinect

Microsoft Kinect, a low-cost motion sensing device, enables users to interact with computers or game consoles naturally through gestures and spoken commands without any other peripheral equipment. As such, it has commanded intense interests in research and development on the Kinect technology. In this paper, we present, a comprehensive survey on Kinect applications, and the latest research and development on motion recognition using data captured by the Kinect sensor. On the applications front, we review the applications of the Kinect technology in a variety of areas, including healthcare, education and performing arts, robotics, sign language recognition, retail services, workplace safety training, as well as 3D reconstructions. On the technology front, we provide an overview of the main features of both versions of the Kinect sensor together with the depth sensing technologies used, and review literatures on human motion recognition techniques used in Kinect applications. We provide a classification of motion recognition techniques to highlight the different approaches used in human motion recognition. Furthermore, we compile a list of publicly available Kinect datasets. These datasets are valuable resources for researchers to investigate better methods for human motion recognition and lower-level computer vision tasks such as segmentation, object detection and human pose estimation

A deep learning solution for real-time human motion decoding in smart walkers

Dissertação de mestrado integrado em Engenharia Biomédica (especialização em Eletrónica Médica)The treatment of gait impairments has increasingly relied on rehabilitation therapies which benefit from the use of smart walkers. These walkers still lack advanced and seamless Human-Robot Interaction, which intuitively understands the intentions of human motion, empowering the user’s recovery state and autonomy, while reducing the physician’s effort. This dissertation proposes the development of a deep learning solution to tackle the human motion decoding problematic in smart walkers, using only lower body vision information from a camera stream, mounted on the WALKit Smart Walker, a smart walker prototype for rehabilitation purposes. Different deep learning frameworks were designed for early human motion recognition and detec tion. A custom acquisition method, including a smart walker’s automatic driving algorithm and labelling procedure, was also designed to enable further training and evaluation of the proposed frameworks. Facing a 4-class (stop, walk, turn right/left) classification problem, a deep learning convolutional model with an attention mechanism achieved the best results: an offline f1-score of 99.61%, an online calibrated instantaneous precision higher than 97% and a human-centred focus slightly higher than 30%. Promising results were attained for early human motion detection, with enhancements in the focus of the proposed architectures. However, further improvements are still needed to achieve a more reliable solution for integration in a smart walker’s control strategy, based in the human motion intentions.O tratamento de distúrbios da marcha tem apostado cada vez mais em terapias de reabilitação que beneficiam do uso de andarilhos inteligentes. Estes ainda carecem de uma Interação Humano-Robô avançada e eficaz, capaz de entender, intuitivamente, as intenções do movimento humano, fortalecendo a recuperação autónoma do paciente e reduzindo o esforço médico. Esta dissertação propõe o desenvolvimento de uma solução de aprendizagem para o problema de descodificação de movimento humano em andarilhos inteligentes, usando apenas vídeos recolhidos pelo WALKit Smart Walker, um protótipo de andarilho inteligente usado para reabilitação. Foram desenvolvidos algoritmos de aprendizagem para o reconhecimento e detecção precoces de movimento humano. Um método de aquisição personalizado, incluindo um algoritmo de condução e labelização automatizados, foi projetado para permitir o conseguinte treino e avaliação dos algoritmos propostos. Perante a classificação de 4 ações (parar, andar, virar à direita/esquerda), um modelo convolucional com um mecanismo de atenção alcançou os melhores resultados: f1-score offline de 99,61%, precisão instantânea calibrada online de superior a 97 % e um foco centrado no ser humano ligeiramente superior a 30%. Com esta dissertação alcançaram-se resultados promissores para a detecção precoce de movimento humano, com aprimoramentos no foco dos algoritmos propostos. No entanto, ainda são necessárias melhorias adicionais para alcançar uma solução mais robusta para a integração na estratégia de controlo de um andarilho inteligente, com base nas intenções de movimento do utilizador

How a Diverse Research Ecosystem Has Generated New Rehabilitation Technologies: Review of NIDILRR’s Rehabilitation Engineering Research Centers

Over 50 million United States citizens (1 in 6 people in the US) have a developmental, acquired, or degenerative disability. The average US citizen can expect to live 20% of his or her life with a disability. Rehabilitation technologies play a major role in improving the quality of life for people with a disability, yet widespread and highly challenging needs remain. Within the US, a major effort aimed at the creation and evaluation of rehabilitation technology has been the Rehabilitation Engineering Research Centers (RERCs) sponsored by the National Institute on Disability, Independent Living, and Rehabilitation Research. As envisioned at their conception by a panel of the National Academy of Science in 1970, these centers were intended to take a “total approach to rehabilitation”, combining medicine, engineering, and related science, to improve the quality of life of individuals with a disability. Here, we review the scope, achievements, and ongoing projects of an unbiased sample of 19 currently active or recently terminated RERCs. Specifically, for each center, we briefly explain the needs it targets, summarize key historical advances, identify emerging innovations, and consider future directions. Our assessment from this review is that the RERC program indeed involves a multidisciplinary approach, with 36 professional fields involved, although 70% of research and development staff are in engineering fields, 23% in clinical fields, and only 7% in basic science fields; significantly, 11% of the professional staff have a disability related to their research. We observe that the RERC program has substantially diversified the scope of its work since the 1970’s, addressing more types of disabilities using more technologies, and, in particular, often now focusing on information technologies. RERC work also now often views users as integrated into an interdependent society through technologies that both people with and without disabilities co-use (such as the internet, wireless communication, and architecture). In addition, RERC research has evolved to view users as able at improving outcomes through learning, exercise, and plasticity (rather than being static), which can be optimally timed. We provide examples of rehabilitation technology innovation produced by the RERCs that illustrate this increasingly diversifying scope and evolving perspective. We conclude by discussing growth opportunities and possible future directions of the RERC program

Active Training and Assistance Device for an Individually Adaptable Strength and Coordination Training

Das Altern der Weltbevölkerung, insbesondere in der westlichen Welt, stellt die Menschheit vor eine große Herausforderung. Zu erwarten sind erhebliche Auswirkungen auf den Gesundheitssektor, der im Hinblick auf eine steigende Anzahl von Menschen mit altersbedingtem körperlichem und kognitivem Abbau und dem damit erhöhten Bedürfnis einer individuellen Versorgung vor einer großen Aufgabe steht. Insbesondere im letzten Jahrhundert wurden viele wissenschaftliche Anstrengungen unternommen, um Ursache und Entwicklung altersbedingter Erkrankungen, ihr Voranschreiten und mögliche Behandlungen, zu verstehen. Die derzeitigen Modelle zeigen, dass der entscheidende Faktor für die Entwicklung solcher Krankheiten der Mangel an sensorischen und motorischen Einflüssen ist, diese wiederum sind das Ergebnis verringerter Mobilität und immer weniger neuer Erfahrungen. Eine Vielzahl von Studien zeigt, dass erhöhte körperliche Aktivität einen positiven Effekt auf den Allgemeinzustand von älteren Erwachsenen mit leichten kognitiven Beeinträchtigungen und den Menschen in deren unmittelbarer Umgebung hat. Diese Arbeit zielt darauf ab, älteren Menschen die Möglichkeit zu bieten, eigenständig und sicher ein individuelles körperliches Training zu absolvieren. In den letzten zwei Jahrzehnten hat die Forschung im Bereich der robotischen Bewegungsassistenten, auch Smarte Rollatoren genannt, den Fokus auf die sensorische und kognitive Unterstützung für ältere und eingeschränkte Personen gesetzt. Durch zahlreiche Bemühungen entstand eine Vielzahl von Ansätzen zur Mensch-Rollator-Interaktion, alle mit dem Ziel, Bewegung und Navigation innerhalb der Umgebung zu unterstützen. Aber trotz allem sind Trainingsmöglichkeiten zur motorischen Aktivierung mittels Smarter Rollatoren noch nicht erforscht. Im Gegensatz zu manchen Smarten Rollatoren, die den Fokus auf Rehabilitationsmöglichkeiten für eine bereits fortgeschrittene Krankheit setzen, zielt diese Arbeit darauf ab, kognitive Beeinträchtigungen in einem frühen Stadium soweit wie möglich zu verlangsamen, damit die körperliche und mentale Fitness des Nutzers so lang wie möglich aufrechterhalten bleibt. Um die Idee eines solchen Trainings zu überprüfen, wurde ein Prototyp-Gerät namens RoboTrainer-Prototyp entworfen, eine mobile Roboter-Plattform, die mit einem zusätzlichen Kraft-Momente-Sensor und einem Fahrradlenker als Eingabe-Schnittstelle ausgestattet wurde. Das Training beinhaltet vordefinierte Trainingspfade mit Markierungen am Boden, entlang derer der Nutzer das Gerät navigieren soll. Der Prototyp benutzt eine Admittanzgleichung, um seine Geschwindigkeit anhand der Eingabe des Nutzers zu berechnen. Desweiteren leitet das Gerät gezielte Regelungsaktionen bzw. Verhaltensänderungen des Roboters ein, um das Training herausfordernd zu gestalten. Die Pilotstudie, die mit zehn älteren Erwachsenen mit beginnender Demenz durchgeführt wurde, zeigte eine signifikante Steigerung ihrer Interaktionsfähigkeit mit diesem Gerät. Sie bewies ebenfalls den Nutzen von Regelungsaktionen, um die Komplexität des Trainings ständig neu anzupassen. Obwohl diese Studie die Durchführbarkeit des Trainings zeigte, waren Grundfläche und mechanische Stabilität des RoboTrainer-Prototyps suboptimal. Deswegen fokussiert sich der zweite Teil dieser Arbeit darauf, ein neues Gerät zu entwerfen, um die Nachteile des Prototyps zu beheben. Neben einer erhöhten mechanischen Stabilität, ermöglicht der RoboTrainer v2 eine Anpassung seiner Grundfläche. Dieses spezifische Merkmal der Smarten Rollatoren dient vor allem dazu, die Unterstützungsfläche für den Benutzer anzupassen. Das ermöglicht einerseits ein agiles Training mit gesunden Personen und andererseits Rehabilitations-Szenarien bei Menschen, die körperliche Unterstützung benötigen. Der Regelungsansatz für den RoboTrainer v2 erweitert den Admittanzregler des Prototypen durch drei adaptive Strategien. Die erste ist die Anpassung der Sensitivität an die Eingabe des Nutzers, abhängig von der Stabilität des Nutzer-Rollater-Systems, welche Schwankungen verhindert, die dann passieren können, wenn die Hände des Nutzers versteifen. Die zweite Anpassung beinhaltet eine neuartige nicht-lineare, geschwindigkeits-basierende Änderung der Admittanz-Parameter, um die Wendigkeit des Rollators zu erhöhen. Die dritte Anpassung erfolgt vor dem eigentlichen Training in einem Parametrierungsprozess, wo nutzereigene Interaktionskräfte gemessen werden, um individuelle Reglerkonstanten fein abzustimmen und zu berechnen. Die Regelungsaktionen sind Verhaltensänderungen des Gerätes, die als Bausteine für unterstützende und herausfordernde Trainingseinheiten mit dem RoboTrainer dienen. Sie nutzen das virtuelle Kraft-Feld-Konzept, um die Bewegung des Gerätes in der Trainingsumgebung zu beeinflussen. Die Bewegung des RoboTrainers wird in der Gesamtumgebung durch globale oder, in bestimmten Teilbereichen, durch räumliche Aktionen beeinflusst. Die Regelungsaktionen erhalten die Absicht des Nutzers aufrecht, in dem sie eine unabhängige Admittanzdynamik implementieren, um deren Einfluss auf die Geschwindigkeit des RoboTrainers zu berechnen. Dies ermöglicht die entscheidende Trennung von Reglerzuständen, um während des Trainings passive und sichere Interaktionen mit dem Gerät zu erreichen. Die oben genannten Beiträge wurden getrennt ausgewertet und in zwei Studien mit jeweils 22 bzw. 13 jungen, gesunden Erwachsenen untersucht. Diese Studien ermöglichen einen umfassenden Einblick in die Zusammenhänge zwischen unterschiedlichen Funktionalitäten und deren Einfluss auf die Nutzer. Sie bestätigen den gesamten Ansatz, sowie die gemachten Vermutungen im Hinblick auf die Gestaltung einzelner Teile dieser Arbeit. Die Einzelergebnisse dieser Arbeit resultieren in einem neuartigen Forschungsgerät für physische Mensch-Roboter-Interaktionen während des Trainings mit Erwachsenen. Zukünftige Forschungen mit dem RoboTrainer ebnen den Weg für Smarte Rollatoren als Hilfe für die Gesellschaft im Hinblick auf den bevorstehenden demographischen Wandel

System architecture for the ASBGo* Smart Walker

Dissertação de mestrado em Engenharia Eletrónica Industrial e ComputadoresWeakness, mobility and balance problems are some of the obstacles that most certainly will go along with the last period of life, the old age. Also, those difficulties can strike young lives due to gait abnormalities resulted from degenerative diseases or even accidents. To patients with high motor deficit, traditional methods, such as wheelchairs, are usually prescribed, but the use of an assistive device that do not promotes the patient’s recovery will eventually lead him to a restrict daily life as well as a notable loss of motor skills. With previous questions in mind, the Adaptive System Behavior Group (ASBG) decided to develop a motorized smart walker capable of adapting to the needs of its users. The Adaptive System Behavior Group Project (ASBGo) counts already with four versions that have proved its worth in clinical environment and was renowned, for two consecutive times, as one of the best technological and innovating Portuguese research projects in the rehabilitation field. However, the electromechanical and software solutions of each prototype commonly impair the global development of the project, making each version obsolete, outdated or unusable. Now, it is time to go further and render these proof-of-concept devices in a mature version, excluding previous academic solutions and engineering a robust and trustworthy device that will establish this new rehabilitation concept. This master thesis, addressed to rehabilitation robotics, describes the design and implementation of a system architecture for the Adaptive System Behavior Group Project Star (ASBGo*) . The implementation of a unified modular system architecture embraces the development of software components, electronic hardware and electromechanical modifications required to its implementation. This new prototype is an upgrade of the all ASBGo previous versions, in which the sturdy and user-friendly solutions implemented provide robust tools for future development and usability. Firstly, the contextualization in the project was performed, including a brief study of robotic software platforms, the familiarity with the several ASBGo prototypes and the research of the best solutions to design a system architecture. Secondly, following a Top-Down strategy, the work plan was established bearing in mind the considerations to design and implement a global system architecture: definition of the main functionalities and behaviours of the prototype and, simultaneously, the strategies to be followed in the hardware, electromechanical and software development. Hereinafter, the development stage was conducted following an Hardware-Software co-design methodology. That strategy ensured that the design and implementation of electronic and electric circuits were in agreement with all system requirements guaranteeing trade-offs, robustness and safety. During all the development process, validations of the system were constantly performed and, in the end, intensive experimentations of the final device were executed in the laboratory with the intervention of colleagues of the ASBG group.Debilidade e dificuldades de mobilidade e equilíbrio são alguns dos problemas que muito certamente irão acompanhar o período final da vida, a velhice. Para além disso, esses problemas podem atingir jovens vidas devido a anomalias na marcha resultantes de doenças degenerativas ou até mesmo acidentes. Aos pacientes que apresentam um alto défice motor, métodos tradicionais, como cadeiras de rodas, são normalmente receitados. No entanto, o uso de dispositivos de assistência que não promovem a recuperação do paciente irão eventualmente levá-lo a uma vida restrita assim como a uma notável perda de capacidades motoras. Com tal ideias em mente, o Adaptive System Behavior Group (ASBG) decidiu desenvolver um andarilho inteligente motorizado capaz de se adaptar as necessidades dos seus utilizadores. O Adaptive System Behavior Group Project (ASBGo) já conta com quatro versões que provaram o seu valor em ambiente clínico e já foi reconhecido, por duas vezes consecutivas, como um dos projetos de investigação mais tecnológico e inovador na área de reabilitação. Contudo, as soluções eletromecânicas e de software de cada protótipo comprometem o desenvolvimento contínuo do projeto, tornando cada versão obsoleta, desatualizada e inutilizável. Agora, está na hora de ir mais longe e tornar estes dispositivos de prova de conceito numa versão mais madura, excluindo as soluções académicas anteriormente implementadas e concebendo um dispositivo robusto e fiável que irá afirmar este novo conceito de reabilitação. A presente dissertação de mestrado, no âmbito da robótica de reabilitação, descreve o design e implementação de uma arquitetura de sistema para o Adaptive System Behavior Group Project Star (ASBGo*). A implementação de uma arquitetura de sistema unificada e modular envolve o desenvolvimento de componentes de software, hardware eletrónico e modificações eletromecânicas necessárias à sua implementação. Este novo protótipo consiste numa melhoria avançada de todas as versões anteriores do ASBGo no qual as soluções vigorosas e acessíveis implementadas providenciam meios para desenvolvimento futuro e usabilidade. Inicialmente, foi realizada a contextualização no projeto que incluiu uma breve pesquisa de plataformas de software robótico, a familiarização com os diferentes protótipos ASBGo e o estudo das melhores soluções para a conceção da arquitetura do sistema. Seguindo uma estratégia Top-Down, o plano de trabalhos foi estabelecido tendo em conta considerações para o design e implementação de uma arquitetura de sistema unificada: definição das principais funcionalidades e comportamentos do protótipo e, concomitantemente, as estratégias a ser seguidas no desenvolvimento eletromecânico, de hardware e de software. Doravante, a fase de desenvolvimento foi acompanhada por uma metodologia de Hardware-Software co-design. Esta estratégia assegurou a concordância entre o design e a implementação de circuitos eletrónicos e elétricos e todos os requisitos do sistema garantindo desta forma, trade-offs, robustez e segurança. Durante todo o processo de desenvolvimento, validações do sistema foram constantemente realizadas, sendo que no final testes intensivos ao produto final foram executados em laboratório com a intervenção dos colegas do grupo ASBG

Humanoid Robots

For many years, the human being has been trying, in all ways, to recreate the complex mechanisms that form the human body. Such task is extremely complicated and the results are not totally satisfactory. However, with increasing technological advances based on theoretical and experimental researches, man gets, in a way, to copy or to imitate some systems of the human body. These researches not only intended to create humanoid robots, great part of them constituting autonomous systems, but also, in some way, to offer a higher knowledge of the systems that form the human body, objectifying possible applications in the technology of rehabilitation of human beings, gathering in a whole studies related not only to Robotics, but also to Biomechanics, Biomimmetics, Cybernetics, among other areas. This book presents a series of researches inspired by this ideal, carried through by various researchers worldwide, looking for to analyze and to discuss diverse subjects related to humanoid robots. The presented contributions explore aspects about robotic hands, learning, language, vision and locomotion