Deep Detection of People and their Mobility Aids for a Hospital Robot

Robots operating in populated environments encounter many different types of people, some of whom might have an advanced need for cautious interaction, because of physical impairments or their advanced age. Robots therefore need to recognize such advanced demands to provide appropriate assistance, guidance or other forms of support. In this paper, we propose a depth-based perception pipeline that estimates the position and velocity of people in the environment and categorizes them according to the mobility aids they use: pedestrian, person in wheelchair, person in a wheelchair with a person pushing them, person with crutches and person using a walker. We present a fast region proposal method that feeds a Region-based Convolutional Network (Fast R-CNN). With this, we speed up the object detection process by a factor of seven compared to a dense sliding window approach. We furthermore propose a probabilistic position, velocity and class estimator to smooth the CNN's detections and account for occlusions and misclassifications. In addition, we introduce a new hospital dataset with over 17,000 annotated RGB-D images. Extensive experiments confirm that our pipeline successfully keeps track of people and their mobility aids, even in challenging situations with multiple people from different categories and frequent occlusions. Videos of our experiments and the dataset are available at http://www2.informatik.uni-freiburg.de/~kollmitz/MobilityAidsComment: 7 pages, ECMR 2017, dataset and videos: http://www2.informatik.uni-freiburg.de/~kollmitz/MobilityAids

Personal Autonomy Rehabilitation in Home Environments by a Portable Assistive Robot

Increasingly disabled and elderly people with mobility problems want to live autonomously in their home environment. They are motivated to use robotic aids to perform tasks by themselves, avoiding permanent nurse or family assistant supervision. They must find means to rehabilitate their abilities to perform daily life activities (DLAs), such as eating, shaving, or drinking. These means may be provided by robotic aids that incorporate possibilities and methods to accomplish common tasks, aiding the user in recovery of partial or complete autonomy. Results are highly conditioned by the system's usability and potential. The developed portable assistive robot ASIBOT helps users perform most of these tasks in common living environments. Minimum adaptations are needed to provide the robot with mobility throughout the environment. The robot can autonomously climb from one surface to another, fixing itself to the best place to perform each task. When the robot is attached to its wheelchair, it can move along with it as a bundle. This paper presents the work performed with the ASIBOT in the area of rehabilitation robotics. First, a brief description of the ASIBOT system is given. A description of tests that have been performed with the robot and several impaired users is given. Insight into how these experiences have influenced our research efforts, especially, in home environments, is also included. A description of the test bed that has been developed to continue research on performing DLAs by the use of robotic aids, a kitchen environment, is given. Relevant conclusions are also included.This work has been supported by the CAM Project S2009/DPI-1559/ROBOCITY2030 I

Instrumentation and validation of a robotic cane for transportation and fall prevention in patients with affected mobility

Dissertação de mestrado integrado em Engenharia Física, (especialização em Dispositivos, Microssistemas e Nanotecnologias)O ato de andar é conhecido por ser a forma primitiva de locomoção do ser humano, sendo que este traz muitos benefícios que motivam um estilo de vida saudável e ativo. No entanto, há condições de saúde que dificultam a realização da marcha, o que por consequência pode resultar num agravamento da saúde, e adicionalmente, levar a um maior risco de quedas. Nesse sentido, o desenvolvimento de um sistema de deteção e prevenção de quedas, integrado num dispositivo auxiliar de marcha, seria essencial para reduzir estes eventos de quedas e melhorar a qualidade de vida das pessoas. Para ultrapassar estas necessidades e limitações, esta dissertação tem como objetivo validar e instrumentar uma bengala robótica, denominada Anti-fall Robotic Cane (ARCane), concebida para incorporar um sistema de deteção de quedas e um mecanismo de atuação que possibilite a prevenção de quedas, ao mesmo tempo que assiste a marcha. Para esse fim, foi realizada uma revisão do estado da arte em bengalas robóticas para adquirir um conhecimento amplo e aprofundado dos componentes, mecanismos e estratégias utilizadas, bem como os protocolos experimentais, principais resultados, limitações e desafios em dispositivos existentes. Numa primeira fase, foi estipulado o objetivo de: (i) adaptar a missão do produto; (ii) estudar as necessidades do consumidor; e (iii) atualizar as especificações alvo da ARCane, continuação do trabalho de equipa, para obter um produto com design e engenharia compatível com o mercado. Foi depois estabelecida a arquitetura de hardware e discutidos os componentes a ser instrumentados na ARCane. Em seguida foram realizados testes de interoperabilidade a fim de validar o funcionamento singular e coletivo dos componentes. Relativamente ao controlo de movimento, foi desenvolvido um sistema inovador, de baixo custo e intuitivo, capaz de detetar a intenção do movimento e de reconhecer as fases da marcha do utilizador. Esta implementação foi validada com seis voluntários saudáveis que realizaram testes de marcha com a ARCane para testar sua operabilidade num ambiente de contexto real. Obteve-se uma precisão de 97% e de 90% em relação à deteção da intenção de movimento e ao reconhecimento da fase da marcha do utilizador. Por fim, foi projetado um método de deteção de quedas e mecanismo de prevenção de quedas para futura implementação na ARCane. Foi ainda proposta uma melhoria do método de deteção de quedas, de modo a superar as limitações associadas, bem como a proposta de dispositivos de deteção a serem implementados na ARCane para obter um sistema completo de deteção de quedas.The act of walking is known to be the primitive form of the human being, and it brings many benefits that motivate a healthy and active lifestyle. However, there are health conditions that make walking difficult, which, consequently, can result in worse health and, in addition, lead to a greater risk of falls. Thus, the development of a fall detection and prevention system integrated with a walking aid would be essential to reduce these fall events and improve people quality of life. To overcome these needs and limitations, this dissertation aims to validate and instrument a cane-type robot, called Anti-fall Robotic Cane (ARCane), designed to incorporate a fall detection system and an actuation mechanism that allow the prevention of falls, while assisting the gait. Therefore, a State-of-the-Art review concerning robotic canes was carried out to acquire a broad and in-depth knowledge of the used components, mechanisms and strategies, as well as the experimental protocols, main results, limitations and challenges on existing devices. On a first stage, it was set an objective to (i) enhance the product's mission statement; (ii) study the consumer needs; and (iii) update the target specifications of the ARCane, extending teamwork, to obtain a product with a market-compatible design and engineering that meets the needs and desires of the ARCane users. It was then established the hardware architecture of the ARCane and discussed the electronic components that will instrument the control, sensory, actuator and power units, being afterwards subjected to interoperability tests to validate the singular and collective functioning of cane components altogether. Regarding the motion control of robotic canes, an innovative, cost-effective and intuitive motion control system was developed, providing user movement intention recognition, and identification of the user's gait phases. This implementation was validated with six healthy volunteers who carried out gait trials with the ARCane, in order to test its operability in a real context environment. An accuracy of 97% was achieved for user motion intention recognition and 90% for user gait phase recognition, using the proposed motion control system. Finally, it was idealized a fall detection method and fall prevention mechanism for a future implementation in the ARCane, based on methods applied to robotic canes in the literature. It was also proposed an improvement of the fall detection method in order to overcome its associated limitations, as well as detection devices to be implemented into the ARCane to achieve a complete fall detection system

Intelligent Hotel ROS-based Service Robot

With the advances of artificial intelligence (AI) technology, many studies and work have been carried out on how robots could replace human labor. In this paper, we present a ROS based intelligence hotel robot, which simplifies the check-in process. We use pioneer 3dx robot and considered different environment settings. The robot combined with Hokuyo Lidar and Kinect Xbox camera, can plan the routes accurately and reach rooms in different floors. In addition, we added an intelligent voice system which provides an assistant for the customers

Robot-assisted gait self-training: assessing the level achieved

This paper presents the technological status of robot-assisted gait self-training under real clinical environment conditions. A successful rehabilitation after surgery in hip endoprosthetics comprises self-training of the lessons taught by physiotherapists. While doing this, immediate feedback to the patient about deviations from the expected physiological gait pattern during training is important. Hence, the Socially Assistive Robot (SAR) developed for this type of training employs task-specific, user-centered navigation and autonomous, real-time gait feature classification techniques to enrich the self-training through companionship and timely corrective feedback. The evaluation of the system took place during user tests in a hospital from the point of view of technical benchmarking, considering the therapists’ and patients’ point of view with regard to training motivation and from the point of view of initial findings on medical efficacy as a prerequisite from an economic perspective. In this paper, the following research questions were primarily considered: Does the level of technology achieved enable autonomous use in everyday clinical practice? Has the gait pattern of patients who used additional robot-assisted gait self-training for several days been changed or improved compared to patients without this training? How does the use of a SAR-based self-training robot affect the motivation of the patients

People Tracking in Panoramic Video for Guiding Robots

A guiding robot aims to effectively bring people to and from specific places within environments that are possibly unknown to them. During this operation the robot should be able to detect and track the accompanied person, trying never to lose sight of her/him. A solution to minimize this event is to use an omnidirectional camera: its 360{\deg} Field of View (FoV) guarantees that any framed object cannot leave the FoV if not occluded or very far from the sensor. However, the acquired panoramic videos introduce new challenges in perception tasks such as people detection and tracking, including the large size of the images to be processed, the distortion effects introduced by the cylindrical projection and the periodic nature of panoramic images. In this paper, we propose a set of targeted methods that allow to effectively adapt to panoramic videos a standard people detection and tracking pipeline originally designed for perspective cameras. Our methods have been implemented and tested inside a deep learning-based people detection and tracking framework with a commercial 360{\deg} camera. Experiments performed on datasets specifically acquired for guiding robot applications and on a real service robot show the effectiveness of the proposed approach over other state-of-the-art systems. We release with this paper the acquired and annotated datasets and the open-source implementation of our method.Comment: Accepted to 17th International Conference on Intelligent Autonomous Systems (IAS-17

How a Diverse Research Ecosystem Has Generated New Rehabilitation Technologies: Review of NIDILRR’s Rehabilitation Engineering Research Centers

Over 50 million United States citizens (1 in 6 people in the US) have a developmental, acquired, or degenerative disability. The average US citizen can expect to live 20% of his or her life with a disability. Rehabilitation technologies play a major role in improving the quality of life for people with a disability, yet widespread and highly challenging needs remain. Within the US, a major effort aimed at the creation and evaluation of rehabilitation technology has been the Rehabilitation Engineering Research Centers (RERCs) sponsored by the National Institute on Disability, Independent Living, and Rehabilitation Research. As envisioned at their conception by a panel of the National Academy of Science in 1970, these centers were intended to take a “total approach to rehabilitation”, combining medicine, engineering, and related science, to improve the quality of life of individuals with a disability. Here, we review the scope, achievements, and ongoing projects of an unbiased sample of 19 currently active or recently terminated RERCs. Specifically, for each center, we briefly explain the needs it targets, summarize key historical advances, identify emerging innovations, and consider future directions. Our assessment from this review is that the RERC program indeed involves a multidisciplinary approach, with 36 professional fields involved, although 70% of research and development staff are in engineering fields, 23% in clinical fields, and only 7% in basic science fields; significantly, 11% of the professional staff have a disability related to their research. We observe that the RERC program has substantially diversified the scope of its work since the 1970’s, addressing more types of disabilities using more technologies, and, in particular, often now focusing on information technologies. RERC work also now often views users as integrated into an interdependent society through technologies that both people with and without disabilities co-use (such as the internet, wireless communication, and architecture). In addition, RERC research has evolved to view users as able at improving outcomes through learning, exercise, and plasticity (rather than being static), which can be optimally timed. We provide examples of rehabilitation technology innovation produced by the RERCs that illustrate this increasingly diversifying scope and evolving perspective. We conclude by discussing growth opportunities and possible future directions of the RERC program