Assistente de navegação com apontador laser para conduzir cadeiras de rodas robotizadas

Orientador: Eric RohmerDissertação (mestrado) - Universidade Estadual de Campinas, Faculdade de Engenharia Elétrica e de ComputaçãoResumo: As soluções de robótica assistida ajudam as pessoas a recuperar sua mobilidade e autonomia perdidas em suas vidas diárias. Este documento apresenta um assistente de navegação de baixo custo projetado para pessoas tetraplégicas para dirigir uma cadeira de rodas robotizada usando a combinação da orientação da cabeça e expressões faciais (sorriso e sobrancelhas para cima) para enviar comandos para a cadeira. O assistente fornece dois modos de navegação: manual e semi-autônomo. Na navegação manual, uma webcam normal com o algoritmo OpenFace detecta a orientação da cabeça do usuário e expressões faciais (sorriso, sobrancelhas para cima) para compor comandos e atuar diretamente nos movimentos da cadeira de rodas (parar, ir à frente, virar à direita, virar à esquerda). No modo semi-autônomo, o usuário controla um laser pan-tilt com a cabeça para apontar o destino desejado no solo e valida com o comando sobrancelhas para cima que faz com que a cadeira de rodas robotizada realize uma rotação seguida de um deslocamento linear para o alvo escolhido. Embora o assistente precise de melhorias, os resultados mostraram que essa solução pode ser uma tecnologia promissora para pessoas paralisadas do pescoço para controlar uma cadeira de rodas robotizadaAbstract: Assistive robotics solutions help people to recover their lost mobility and autonomy in their daily life. This document presents a low-cost navigation assistant designed for people paralyzed from down the neck to drive a robotized wheelchair using the combination of the head's posture and facial expressions (smile and eyebrows up) to send commands to the chair. The assistant provides two navigation modes: manual and semi-autonomous. In the manual navigation, a regular webcam with the OpenFace algorithm detects the user's head orientation and facial expressions (smile, eyebrows up) to compose commands and actuate directly on the wheelchair movements (stop, go front, turn right, turn left). In the semi-autonomous, the user controls a pan-tilt laser with his/her head to point the desired destination on the ground and validates with eyebrows up command which makes the robotized wheelchair performs a rotation followed by a linear displacement to the chosen target. Although the assistant need improvements, results have shown that this solution may be a promising technology for people paralyzed from down the neck to control a robotized wheelchairMestradoEngenharia de ComputaçãoMestre em Engenharia ElétricaCAPE

Development of a new robust hybrid automata algorithm based on surface electromyography (SEMG) signal for instrumented wheelchair control

Instrumented wheelchair operates based on surface electromyography (sEMG) is one of alternative to assist impairment person for mobility. SEMG is chosen due to good in accuracy and easier preparation to place the electrodes. Motor neuron transmit electrical potential to muscle fibre to perform isometric, concentric or eccentric contraction. These electrical changes that is called Motor Unit Action Potential (MUAP) can be acquired and amplified by electrodes located on targeted muscles changes can be recorded and analysed using sEMG devices. But, sEMG device cost up to USD 2,100 for a sEMG data acquisition device that available on market is one of the drawback to be used by impairment person that most of them has financial problem due to unable to work like before. In addition, it is a closed source system that cannot be modified to improve the accuracy and adding more features. Open source system such as Arduino has limitation of specifications that makes able to apply nonpattern recognition control methods which is simpler and easier compared to pattern recognition. However, classification accuracy is lower than pattern recognition and it cannot be applied to higher number participants from different background and gender. This research aims are to develop an open-source Arduino based sEMG data acquisition device by formulating hybrid automata algorithm to differentiate MUAP activity during wheelchair propulsion. Addition of hybrid automata algorithm to run pattern and non-pattern recognition based control methods is an advantage to increase accuracy in differentiating forward stroke or hand return activity. Electrodes are placed on Biceps (BIC), Triceps (TRI), Extensor (EXT), Flexor (FIX) and MUAP activity recorded for 30 healthy persons. Then, experiment result was validated with simulation result using OpenSim biomedical modelling software. Mean, standard deviation (SD), confidence interval (CI) and maximum point different (MPD) of MUAP were calculated and to be used as thresholds for non-pattern recognition control method in method selection experiment. Meanwhile, pattern recognition is using Probability Density Function (PDF) to determine MUAP according to type of activities. Total of ten control methods determined from population and individual data were tested against another 10 healthy persons to evaluate the algorithm performance. Assessment of each control method done by misclassification matrix looking at True Positive (TP) and False Negative (FN) of power assist system activation period. Developed sEMG data acquisition device that is operated by Arduino MEGA 2560 and Myoware muscle sensors with sampling rate of above 400Hz successfully recorded MUAP from four arm muscles. Furthermore, 2.5 ms of average data latency for device to record, analyse, validate and creating commands to activate the power assist system. Data obtained from the device shows that most active muscle during wheelchair propulsion is TRI, followed by BIC and matched to OpenSim simulation result. In method selection experiment, 96.28% of average accuracy was achieved and different control methods were selected by misclassification matrix for each of persons. This method would be a control method to activate power assist system and selected based on conditions set in the algorithm. These findings indicated that open source Arduino board is capable of running real time pattern, non-pattern recognition based control methods by producing classification accuracy up to 99.48% even though it is known as just a microcontroller that has limitation to run complex classifiers. At the same time, a device that cost less than USD200 has 400Hz of sampling rate is as good as closed source device that is come with expensive price tag to own it. Based on algorithm evaluation, it shows that one control method couldn’t fit to all persons as per proven in method selection experiment. Different person has different control method that suit them the most. Lastly, BIC and TRI can be reference muscles to activate assistive device in instrumented wheelchair that is using propulsion as indication

Mobile robot teleoperation through eye-gaze (telegaze)

In most teleoperation applications the human operator is required to monitor the status of the robot, as well as, issue controlling commands for the whole duration of the operation. Using a vision based feedback system, monitoring the robot requires the operator to look at a continuous stream of images displayed on an interaction screen. The eyes of the operator therefore, are fully engaged in monitoring and the hands in controlling. Since the eyes of the operator are engaged in monitoring anyway, inputs from their gaze can be used to aid in controlling. This frees the hands of the operator, either partially or fully, from controlling which can then be used to perform any other necessary tasks. However, the challenge here lies in distinguishing between the inputs that can be used for controlling and the inputs that can be used for monitoring. In mobile robot teleoperation, controlling is mainly composed of issuing locomotion commands to drive the robot. Monitoring on the other hand, is looking where the robot goes and looking for any obstacles in the route. Interestingly, there exist a strong correlation between human's gazing behaviours and their moving intentions. This correlation has been exploited in this thesis to investigate novel means for mobile robot teleoperation through eye-gaze, which has been named TeleGaze for short

Autonomous Navigation of Mobile Robots: Marker-based Localization System and On-line Path

Traditional wheelchairs are controlled mainly by joystick, which is not suitable solution with major disabilities. Current thesis aiming to create a human-machine interface and create a software, which performs indoor autonomous navigation of the commercial wheelchair RoboEye, developed at the Measurements Instrumentations Robotic Laboratory at the University of Trento in collaboration with Robosense and Xtrensa,. RoboEye is an intelligent wheelchair that aims to support people by providing independence and autonomy of movement, affected by serious mobility problems from impairing pathologies (for example ALS – amyotrophic lateral sclerosis). This thesis is divided into two main parts – human machine interface creation plus integration of existing services into developed solution, and performing possible solution how given wheelchair can navigate manually utilizing eye-tracking technologies, TOF cameras, odometric localization and Aruco markers. Developed interface supports manual, semi-autonomous and autonomous navigation. In addition to that following user experience specific for eye-tracking devices and people with major disabilities. Application delevoped on Unity 3D software using C# script following state-machine approach with multiple scenes and components. In the current master thesis, suggested solution satisfies user’s need to navigate hands-free, as less tiring as possible. Moreover, user can choose the destination point from defined in advance points of interests and reach it with no further input needed. User interface is intuitive and clear for experienced and inexperienced users. The user can choose UI’s icons image, scale and font size. Software performs in a state machine module, which is tested among users using test cases. Path planning routine is solved using Dijkstra approach and proved to be efficient

The Relationship between Anthropometric Variables and Features of Electromyography Signal for Human-Computer Interface

http://doi.org/10.4018/978-1-4666-6090-8 ISBN 13 : 9781466660908 EISBN13: 9781466660915International audienceMuscle-computer interfaces (MCIs) based on surface electromyography (EMG) pattern recognition have been developed based on two consecutive components: feature extraction and classification algorithms. Many features and classifiers are proposed and evaluated, which yield the high classification accuracy and the high number of discriminated motions under a single-session experimental condition. However, there are many limitations to use MCIs in the real-world contexts, such as the robustness over time, noise, or low-level EMG activities. Although the selection of the suitable robust features can solve such problems, EMG pattern recognition has to design and train for a particular individual user to reach high accuracy. Due to different body compositions across users, a feasibility to use anthropometric variables to calibrate EMG recognition system automatically/semi-automatically is proposed. This chapter presents the relationships between robust features extracted from actions associated with surface EMG signals and twelve related anthropometric variables. The strong and significant associations presented in this chapter could benefit a further design of the MCIs based on EMG pattern recognition

Mechatronic Systems

Mechatronics, the synergistic blend of mechanics, electronics, and computer science, has evolved over the past twenty five years, leading to a novel stage of engineering design. By integrating the best design practices with the most advanced technologies, mechatronics aims at realizing high-quality products, guaranteeing at the same time a substantial reduction of time and costs of manufacturing. Mechatronic systems are manifold and range from machine components, motion generators, and power producing machines to more complex devices, such as robotic systems and transportation vehicles. With its twenty chapters, which collect contributions from many researchers worldwide, this book provides an excellent survey of recent work in the field of mechatronics with applications in various fields, like robotics, medical and assistive technology, human-machine interaction, unmanned vehicles, manufacturing, and education. We would like to thank all the authors who have invested a great deal of time to write such interesting chapters, which we are sure will be valuable to the readers. Chapters 1 to 6 deal with applications of mechatronics for the development of robotic systems. Medical and assistive technologies and human-machine interaction systems are the topic of chapters 7 to 13.Chapters 14 and 15 concern mechatronic systems for autonomous vehicles. Chapters 16-19 deal with mechatronics in manufacturing contexts. Chapter 20 concludes the book, describing a method for the installation of mechatronics education in schools

A Driving Behaviour Model of Electrical Wheelchair Users

In spite of the presence of powered wheelchairs, some of the users still experience steering challenges and manoeuvring difficulties that limit their capacity of navigating effectively. For such users, steering support and assistive systems may be very necessary. To appreciate the assistance, there is need that the assistive control is adaptable to the user’s steering behaviour. This paper contributes to wheelchair steering improvement by modelling the steering behaviour of powered wheelchair users, for integration into the control system. More precisely, the modelling is based on the improved Directed Potential Field (DPF) method for trajectory planning. The method has facilitated the formulation of a simple behaviour model that is also linear in parameters. To obtain the steering data for parameter identification, seven individuals participated in driving the wheelchair in different virtual worlds on the augmented platform. The obtained data facilitated the estimation of user parameters, using the ordinary least square method, with satisfactory regression analysis results

Hardware for recognition of human activities: a review of smart home and AAL related technologies

Activity recognition (AR) from an applied perspective of ambient assisted living (AAL) and smart homes (SH) has become a subject of great interest. Promising a better quality of life, AR applied in contexts such as health, security, and energy consumption can lead to solutions capable of reaching even the people most in need. This study was strongly motivated because levels of development, deployment, and technology of AR solutions transferred to society and industry are based on software development, but also depend on the hardware devices used. The current paper identifies contributions to hardware uses for activity recognition through a scientific literature review in the Web of Science (WoS) database. This work found four dominant groups of technologies used for AR in SH and AAL—smartphones, wearables, video, and electronic components—and two emerging technologies: Wi-Fi and assistive robots. Many of these technologies overlap across many research works. Through bibliometric networks analysis, the present review identified some gaps and new potential combinations of technologies for advances in this emerging worldwide field and their uses. The review also relates the use of these six technologies in health conditions, health care, emotion recognition, occupancy, mobility, posture recognition, localization, fall detection, and generic activity recognition applications. The above can serve as a road map that allows readers to execute approachable projects and deploy applications in different socioeconomic contexts, and the possibility to establish networks with the community involved in this topic. This analysis shows that the research field in activity recognition accepts that specific goals cannot be achieved using one single hardware technology, but can be using joint solutions, this paper shows how such technology works in this regard

Shared Control for Wheelchair Interfaces

Independent mobility is fundamental to the quality of life of people with impairment. Most people with severe mobility impairments, whether congenital, e.g., from cerebral palsy, or acquired, e.g., from spinal cord injury, are prescribed a wheelchair. A small yet significant number of people are unable to use a typical powered wheelchair controlled with a joystick. Instead, some of these people require alternative interfaces such as a head- array or Sip/Puff switch to drive their powered wheelchairs. However, these alternative interfaces do not work for everyone and often cause frustration, fatigue and collisions. This thesis develops a novel technique to help improve the usability of some of these alternative interfaces, in particular, the head-array and Sip/Puff switch. Control is shared between a powered wheelchair user, using an alternative interface and a pow- ered wheelchair fitted with sensors. This shared control then produces a resulting motion that is close to what the user desires to do but a motion that is also safe. A path planning algorithm on the wheelchair is implemented using techniques in mo- bile robotics. Afterwards, the output of the path planning algorithm and the user’s com- mand are both modelled as random variables. These random variables are then blended in a joint probability distribution where the final velocity to the wheelchair is the one that maximises the joint probability distribution. The performance of the probabilistic approach to blending the user’s inputs with the output of a path planner, is benchmarked against the most common form of shared control called linear blending. The benchmarking consists of several experiments with end users both in a simulated world and in the real-world. The thesis concludes that probabilistic shared control provides safer motion compared with the traditional shared control for difficult tasks and hard-to-use interfaces