Instrumentation and validation of a robotic cane for transportation and fall prevention in patients with affected mobility

Dissertação de mestrado integrado em Engenharia Física, (especialização em Dispositivos, Microssistemas e Nanotecnologias)O ato de andar é conhecido por ser a forma primitiva de locomoção do ser humano, sendo que este traz muitos benefícios que motivam um estilo de vida saudável e ativo. No entanto, há condições de saúde que dificultam a realização da marcha, o que por consequência pode resultar num agravamento da saúde, e adicionalmente, levar a um maior risco de quedas. Nesse sentido, o desenvolvimento de um sistema de deteção e prevenção de quedas, integrado num dispositivo auxiliar de marcha, seria essencial para reduzir estes eventos de quedas e melhorar a qualidade de vida das pessoas. Para ultrapassar estas necessidades e limitações, esta dissertação tem como objetivo validar e instrumentar uma bengala robótica, denominada Anti-fall Robotic Cane (ARCane), concebida para incorporar um sistema de deteção de quedas e um mecanismo de atuação que possibilite a prevenção de quedas, ao mesmo tempo que assiste a marcha. Para esse fim, foi realizada uma revisão do estado da arte em bengalas robóticas para adquirir um conhecimento amplo e aprofundado dos componentes, mecanismos e estratégias utilizadas, bem como os protocolos experimentais, principais resultados, limitações e desafios em dispositivos existentes. Numa primeira fase, foi estipulado o objetivo de: (i) adaptar a missão do produto; (ii) estudar as necessidades do consumidor; e (iii) atualizar as especificações alvo da ARCane, continuação do trabalho de equipa, para obter um produto com design e engenharia compatível com o mercado. Foi depois estabelecida a arquitetura de hardware e discutidos os componentes a ser instrumentados na ARCane. Em seguida foram realizados testes de interoperabilidade a fim de validar o funcionamento singular e coletivo dos componentes. Relativamente ao controlo de movimento, foi desenvolvido um sistema inovador, de baixo custo e intuitivo, capaz de detetar a intenção do movimento e de reconhecer as fases da marcha do utilizador. Esta implementação foi validada com seis voluntários saudáveis que realizaram testes de marcha com a ARCane para testar sua operabilidade num ambiente de contexto real. Obteve-se uma precisão de 97% e de 90% em relação à deteção da intenção de movimento e ao reconhecimento da fase da marcha do utilizador. Por fim, foi projetado um método de deteção de quedas e mecanismo de prevenção de quedas para futura implementação na ARCane. Foi ainda proposta uma melhoria do método de deteção de quedas, de modo a superar as limitações associadas, bem como a proposta de dispositivos de deteção a serem implementados na ARCane para obter um sistema completo de deteção de quedas.The act of walking is known to be the primitive form of the human being, and it brings many benefits that motivate a healthy and active lifestyle. However, there are health conditions that make walking difficult, which, consequently, can result in worse health and, in addition, lead to a greater risk of falls. Thus, the development of a fall detection and prevention system integrated with a walking aid would be essential to reduce these fall events and improve people quality of life. To overcome these needs and limitations, this dissertation aims to validate and instrument a cane-type robot, called Anti-fall Robotic Cane (ARCane), designed to incorporate a fall detection system and an actuation mechanism that allow the prevention of falls, while assisting the gait. Therefore, a State-of-the-Art review concerning robotic canes was carried out to acquire a broad and in-depth knowledge of the used components, mechanisms and strategies, as well as the experimental protocols, main results, limitations and challenges on existing devices. On a first stage, it was set an objective to (i) enhance the product's mission statement; (ii) study the consumer needs; and (iii) update the target specifications of the ARCane, extending teamwork, to obtain a product with a market-compatible design and engineering that meets the needs and desires of the ARCane users. It was then established the hardware architecture of the ARCane and discussed the electronic components that will instrument the control, sensory, actuator and power units, being afterwards subjected to interoperability tests to validate the singular and collective functioning of cane components altogether. Regarding the motion control of robotic canes, an innovative, cost-effective and intuitive motion control system was developed, providing user movement intention recognition, and identification of the user's gait phases. This implementation was validated with six healthy volunteers who carried out gait trials with the ARCane, in order to test its operability in a real context environment. An accuracy of 97% was achieved for user motion intention recognition and 90% for user gait phase recognition, using the proposed motion control system. Finally, it was idealized a fall detection method and fall prevention mechanism for a future implementation in the ARCane, based on methods applied to robotic canes in the literature. It was also proposed an improvement of the fall detection method in order to overcome its associated limitations, as well as detection devices to be implemented into the ARCane to achieve a complete fall detection system

People tracking and following with a smart wheelchair using an omnidirectional camera and a RGB-D Camera

The project implements a new service that enables a smart wheelchair user and another person to have a normal talk while freely strolling around the environment, without the need of any interaction towards the wheelchair, called Jiaolong

Coopération de réseaux de caméras ambiantes et de vision embarquée sur robot mobile pour la surveillance de lieux publics

Actuellement, il y a une demande croissante pour le déploiement de robots mobile dans des lieux publics. Pour alimenter cette demande, plusieurs chercheurs ont déployé des systèmes robotiques de prototypes dans des lieux publics comme les hôpitaux, les supermarchés, les musées, et les environnements de bureau. Une principale préoccupation qui ne doit pas être négligé, comme des robots sortent de leur milieu industriel isolé et commencent à interagir avec les humains dans un espace de travail partagé, est une interaction sécuritaire. Pour un robot mobile à avoir un comportement interactif sécuritaire et acceptable - il a besoin de connaître la présence, la localisation et les mouvements de population à mieux comprendre et anticiper leurs intentions et leurs actions. Cette thèse vise à apporter une contribution dans ce sens en mettant l'accent sur les modalités de perception pour détecter et suivre les personnes à proximité d'un robot mobile. Comme une première contribution, cette thèse présente un système automatisé de détection des personnes visuel optimisé qui prend explicitement la demande de calcul prévue sur le robot en considération. Différentes expériences comparatives sont menées pour mettre clairement en évidence les améliorations de ce détecteur apporte à la table, y compris ses effets sur la réactivité du robot lors de missions en ligne. Dans un deuxiè contribution, la thèse propose et valide un cadre de coopération pour fusionner des informations depuis des caméras ambiant affixé au mur et de capteurs montés sur le robot mobile afin de mieux suivre les personnes dans le voisinage. La même structure est également validée par des données de fusion à partir des différents capteurs sur le robot mobile au cours de l'absence de perception externe. Enfin, nous démontrons les améliorations apportées par les modalités perceptives développés en les déployant sur notre plate-forme robotique et illustrant la capacité du robot à percevoir les gens dans les lieux publics supposés et respecter leur espace personnel pendant la navigation.This thesis deals with detection and tracking of people in a surveilled public place. It proposes to include a mobile robot in classical surveillance systems that are based on environment fixed sensors. The mobile robot brings about two important benefits: (1) it acts as a mobile sensor with perception capabilities, and (2) it can be used as means of action for service provision. In this context, as a first contribution, it presents an optimized visual people detector based on Binary Integer Programming that explicitly takes the computational demand stipulated into consideration. A set of homogeneous and heterogeneous pool of features are investigated under this framework, thoroughly tested and compared with the state-of-the-art detectors. The experimental results clearly highlight the improvements the different detectors learned with this framework bring to the table including its effect on the robot's reactivity during on-line missions. As a second contribution, the thesis proposes and validates a cooperative framework to fuse information from wall mounted cameras and sensors on the mobile robot to better track people in the vicinity. Finally, we demonstrate the improvements brought by the developed perceptual modalities by deploying them on our robotic platform and illustrating the robot's ability to perceive people in supposed public areas and respect their personal space during navigation

Target tracking using laser range finder with occlusion

Mestrado em Engenharia MecânicaEste trabalho apresenta uma técnica para a detecção e seguimento de múltiplos alvos móveis usando um sensor de distâncias laser em situações de forte oclusão. O processo inicia-se com a aplicação de filtros temporais aos dados em bruto de modo a eliminar o ruído do sensor seguindo-se de uma segmentação em várias fases com o objectivo de contornar o problema da oclusão. Os segmentos obtidos representam objectos presentes no ambiente. Para cada segmento um ponto representativo da sua posição no mundo é calculado, este ponto é definido de modo a ser relativamente invariante à rotação e mudança de forma do objecto. Para fazer o seguimento de alvos uma lista de objectos a seguir é mantida, todos os objectos visíveis são associados a objectos desta lista usando técnicas de procura baseadas na previsão do movimento dos objectos. Uma zona de procura de forma elíptica é definida para cada objecto da lista sendo nesta zona que se dará a associação. A previsão do movimento é feita com base em dois modelos de movimento, um de velocidade constante e um de aceleração constante e com aplicação de filtros de Kalman. O algoritmo foi testado em diversas condições reais e mostrou-se robusto e eficaz no seguimento de pessoas mesmo em situações de extensa oclusão. ABSTRACT: In this work a technique for the detection and tracking of multiple moving targets in situations of strong occlusion using a laser rangefinder is presented. The process starts by the application of temporal filters to the raw data in order to remove noise followed by a multi phase segmentation with the goal of overcoming occlusions. The resulting segments represent objects in the environment. For each segment a representative point is defined; this point is calculated to better represent the object while keeping some invariance to rotation and shape changes. In order to perform the tracking, a list of objects to follow is maintained; all visible objects are associated with objects from this list using search techniques based on the predicted motion of objects. A search zone shaped as an ellipse is defined for each object; it is in this zone that the association is preformed. The motion prediction is based in two motion models, one with constant velocity and the other with constant acceleration and in the application of Kalman filters. The algorithm was tested in diverse real conditions and shown to be robust and effective in the tracking of people even in situations of long occlusions

Multi-robot Tethering Using Camera

An autonomous multi-robot or swarm robot able to perform various cooperative mission such as search and rescue, exploration of unknown or partially known area, transportation, surveillance, defence system, and also firefighting. However, multi-robot application often requires synchronised robotic configuration, reliable communication system and various sensors installed on each robot. This approach has resulted system complexity and very high cost of development

Computational intelligence approaches to robotics, automation, and control [Volume guest editors]

No abstract available

Mobile Robot Localization Based on Kalman Filter

Robot localization is one of the most important subjects in the Robotics science. It is an interesting and complicated topic. There are many algorithms to solve the problem of localization. Each localization system has its own set of features, and based on them, a solution will be chosen. In my thesis, I want to present a solution to find the best estimate for a robot position in certain space for which a map is available. The thesis started with an elementary introduction to the probability and the Gaussian theories. Simple and advanced practical examples are presented to illustrate each concept related to localization. Extended Kalman Filter is chosen to be the main algorithm to find the best estimate of the robot position. It was presented through two chapters with many examples. All these examples were simulated in Matlab in this thesis in order to give the readers and future students a clear and complete introduction to Kalman Filter. Fortunately, I applied this algorithm on a robot that I have built its base from scratch. MCECS-Bot was a project started in Winter 2012 and it was assigned to me from my adviser, Dr. Marek Perkowski. This robot consists of the base with four Mecanum wheels, the waist based on four linear actuators, an arm, neck and head. The base is equipped with many sensors, which are bumper switches, encoders, sonars, LRF and Kinect. Additional devices can provide extra information as backup sensors, which are a tablet and a camera. The ultimate goal of this thesis is to have the MCECS-Bot as an open source system accessed by many future classes, capstone projects and graduate thesis students for education purposes. A well-known MRPT software system was used to present the results of the Extended Kalman Filter (EKF). These results are simply the robot positions estimated by EKF. They are demonstrated on the base floor of the FAB building of PSU. In parallel, simulated results to all different solutions derived in this thesis are presented using Matlab. A future students will have a ready platform and a good start to continue developing this system

Calibração de sensores do ATLASCAR2 por otimização global

In autonomous vehicles, it is often necessary to install a large number of sensors on board. Thus, the extrinsic calibration of these multi-sensory systems is a problem of high relevance for the development algorithms of autonomous driving or of assistance to the driving. This work proposes a tool to automatically calibrate simultaneously multiple cameras. In the process, aruco markers are used, which allows establishing a graph from which the geometric transformations between the various cameras and a global reference are extracted. Initially, markers are detected in the images using an OpenCV tool. Subsequently, the graph is established where the nodes are cameras or markers and the edges are the transformations between them. Then an initial estimate of the extrinsic parameters of all cameras is calculated based on the detections of the markers and the paths obtained from the graph. In the end, an optimization of the parameters is done, where the reprojection error is minimized. In order to demonstrate the process, several datasets were created in order to validate the obtained results.Em veículos autónomos é frequente a necessidade de instalar um grande número de sensores a bordo. Assim, a calibração extrínseca destes sistemas multi-sensoriais é um problema de grande relevância para o desenvolvimento de algoritmos de condução autónoma ou de apoio á condução. Este trabalho propõe um mecanismo capaz de fazer uma calibração automática em simultaâneo de várias câmaras. No processo são usados marcadores aruco, o que permite estabelecer um grafo de onde se extraem as transformações geométricas entre as várias câmaras e um referencial global. Inicialmente, os marcadores são detetados nas imagens usando uma ferramenta do OpenCV. Posteriormente é construido o grafo em que os nós são câmaras ou marcadores, e as ligações entre nós são transformações geométricas em pares câmara aruco. Em seguida é calculada uma estimativa inicial dos parâmetros extrínsecos de todas as câmaras, baseada nas deteções dos marcadores e nos caminhos obtidos do grafo. No fim, é feita uma otimização dos parâmetros, onde é minimizado o erro de reprojeção. Para demonstrar o processo foram criados vários "datasets", de modo a validar os resultados obtidos.Mestrado em Engenharia Mecânic

Mobile Robots Navigation

Mobile robots navigation includes different interrelated activities: (i) perception, as obtaining and interpreting sensory information; (ii) exploration, as the strategy that guides the robot to select the next direction to go; (iii) mapping, involving the construction of a spatial representation by using the sensory information perceived; (iv) localization, as the strategy to estimate the robot position within the spatial map; (v) path planning, as the strategy to find a path towards a goal location being optimal or not; and (vi) path execution, where motor actions are determined and adapted to environmental changes. The book addresses those activities by integrating results from the research work of several authors all over the world. Research cases are documented in 32 chapters organized within 7 categories next described