Towards Natural Human Control and Navigation of Autonomous Wheelchairs

Approximately 2.2 million people in the United States depend on a wheelchair to assist with their mobility. Often times, the wheelchair user can maneuver around using a conventional joystick. Visually impaired or wheelchair patients with restricted hand mobility, such as stroke, arthritis, limb injury, Parkinson’s, cerebral palsy or multiple sclerosis, prevent them from using traditional joystick controls. The resulting mobility limitations force these patients to rely on caretakers to perform everyday tasks. This minimizes the independence of the wheelchair user. Modern day speech recognition systems can be used to enhance user experiences when using electronic devices. By expanding the motorized wheelchair control interface to include the detection of user speech commands, the independence is given back to the mobility impaired. A speech recognition interface was developed for a smart wheelchair. By integrating navigation commands with a map of the wheelchair’s surroundings, the wheelchair interface is more natural and intuitive to use. Complex speech patterns are interpreted for users to command the smart wheelchair to navigate to specified locations within the map. Pocketsphinx, a speech toolkit, is used to interpret the vocal commands. A language model and dictionary were generated based on a set of possible commands and locations supplied to the speech recognition interface. The commands fall under the categories of speed, directional, or destination commands. Speed commands modify the relative speed of the wheelchair. Directional commands modify the relative direction of the wheelchair. Destination commands require a known location on a map to navigate to. The completion of the speech input processer and the connection between wheelchair components via the Robot Operating System make map navigation possible

The Remote Controllable Electric Wheelchair System combined Human and Machine Intelligence for Caregivers and Care Receivers

Thesis (Master of Science in Informatics)--University of Tsukuba, no. 41280, 2019.3.2

Advanced Map Matching Technologies and Techniques for Pedestrian/Wheelchair Navigation

Due to the constantly increasing technical advantages of mobile devices (such as smartphones), pedestrian/wheelchair navigation recently has achieved a high level of interest as one of smartphones’ potential mobile applications. While vehicle navigation systems have already reached a certain level of maturity, pedestrian/wheelchair navigation services are still in their infancy. By comparing vehicle navigation systems, a set of map matching requirements and challenges unique in pedestrian/wheelchair navigation is identified. To provide navigation assistance to pedestrians and wheelchair users, there is a need for the design and development of new map matching techniques. The main goal of this research is to investigate and develop advanced map matching technologies and techniques particular for pedestrian/wheelchair navigation services. As the first step in map matching, an adaptive candidate segment selection algorithm is developed to efficiently find candidate segments. Furthermore, to narrow down the search for the correct segment, advanced mathematical models are applied. GPS-based chain-code map matching, Hidden Markov Model (HMM) map matching, and fuzzy-logic map matching algorithms are developed to estimate real-time location of users in pedestrian/wheelchair navigation systems/services. Nevertheless, GPS signal is not always available in areas with high-rise buildings and even when there is a signal, the accuracy may not be high enough for localization of pedestrians and wheelchair users on sidewalks. To overcome these shortcomings of GPS, multi-sensor integrated map matching algorithms are investigated and developed in this research. These algorithms include a movement pattern recognition algorithm, using accelerometer and compass data, and a vision-based positioning algorithm to fill in signal gaps in GPS positioning. Experiments are conducted to evaluate the developed algorithms using real field test data (GPS coordinates and other sensors data). The experimental results show that the developed algorithms and the integrated sensors, i.e., a monocular visual odometry, a GPS, an accelerometer, and a compass, can provide high-quality and uninterrupted localization services in pedestrian/wheelchair navigation systems/services. The map matching techniques developed in this work can be applied to various pedestrian/wheelchair navigation applications, such as tracking senior citizens and children, or tourist service systems, and can be further utilized in building walking robots and automatic wheelchair navigation systems

Implementation of target tracking in Smart Wheelchair Component System

Independent mobility is critical to individuals of any age. While the needs of many individuals with disabilities can be satisfied with power wheelchairs, some members of the disabled community find it difficult or impossible to operate a standard power wheelchair. This population includes, but is not limited to, individuals with low vision, visual field neglect, spasticity, tremors, or cognitive deficits. To meet the needs of this population, our group is involved in developing cost effective modularly designed Smart Wheelchairs. Our objective is to develop an assistive navigation system which will seamlessly integrate into the lifestyle of individual with disabilities and provide safe and independent mobility and navigation without imposing an excessive physical or cognitive load. The Smart Wheelchair Component System (SWCS) can be added to a variety of commercial power wheelchairs with minimal modification to provide navigation assistance. Previous versions of the SWCS used acoustic and infrared rangefinders to identify and avoid obstacles, but these sensors do not lend themselves to many desirable higher-level behaviors. To achieve these higher level behaviors we integrated a Continuously Adapted Mean Shift (CAMSHIFT) target tracking algorithm into the SWCS, along with the Minimal Vector Field Histogram (MVFH) obstacle avoidance algorithm. The target tracking algorithm provides the basis for two distinct operating modes: (1) a "follow-the-leader" mode, and (2) a "move to stationary target" mode.The ability to track a stationary or moving target will make smart wheelchairs more useful as a mobility aid, and is also expected to be useful for wheeled mobility training and evaluation. In addition to wheelchair users, the caregivers, clinicians, and transporters who provide assistance to wheelchair users will also realize beneficial effects of providing safe and independent mobility to wheelchair users which will reduce the level of assistance needed by wheelchair users

SmartBFA: A passive crowdsourcing system for point-to-point barrier-free access

National Research Foundation (NRF) Singapore under its Industry Alignment Fund (Pre-positioning) Funding Initiative; Tote Board’s Enabling Lives Initiative (TB-ELI) Gran

Principle and Robotic Applications of Conical Scanning Method

We have proposed a surrounding environmental recognition method using conical scanning distance measurement for mobile robots. The conical scanning method has a high robustness against ambient light noise and its calculation cost is low. This report describes the principle of this method and the two implementations. The first application is the quadrupedal wheeled robot. The robot can recognize a stair in sight, approach to it and climb. A second application is the indoor-outdoor wheeled mobile robot. The robot can recognize its position and obstacles on floor from information of a laser range finder and a 3D ToF camera. The validity of this method was confirmed by these robotic applications

Mechatronic Systems

Mechatronics, the synergistic blend of mechanics, electronics, and computer science, has evolved over the past twenty five years, leading to a novel stage of engineering design. By integrating the best design practices with the most advanced technologies, mechatronics aims at realizing high-quality products, guaranteeing at the same time a substantial reduction of time and costs of manufacturing. Mechatronic systems are manifold and range from machine components, motion generators, and power producing machines to more complex devices, such as robotic systems and transportation vehicles. With its twenty chapters, which collect contributions from many researchers worldwide, this book provides an excellent survey of recent work in the field of mechatronics with applications in various fields, like robotics, medical and assistive technology, human-machine interaction, unmanned vehicles, manufacturing, and education. We would like to thank all the authors who have invested a great deal of time to write such interesting chapters, which we are sure will be valuable to the readers. Chapters 1 to 6 deal with applications of mechatronics for the development of robotic systems. Medical and assistive technologies and human-machine interaction systems are the topic of chapters 7 to 13.Chapters 14 and 15 concern mechatronic systems for autonomous vehicles. Chapters 16-19 deal with mechatronics in manufacturing contexts. Chapter 20 concludes the book, describing a method for the installation of mechatronics education in schools

Smart bus stop: people counting in a multi-view camera environment

As paragens de autocarros nos dias de hoje tem de estar cada vez mais ao serviço dos utentes, esta dissertação explora as ideias fundamentais sobre o que deve ser uma paragem de autocarro inteligente, reunindo num texto os conceitos mais utilizados e as mais recentes tecnologias sobre este tópico. Os fundamentos do que é uma paragem de autocarro inteligente são explorados, bem como a arquitetura de todo o sistema, não só a paragem propriamente dita. Ao analisar a bibliografia já existentes compreende-se que a paragem de autocarro não é uma entidade totalmente independente, pois esta está dependente de informação vinda de variadíssimas fontes. Assim sendo, a paragem de autocarro inteligente será um subsistema de um sistema muito mais complexo, composto pela própria paragem, pelo autocarro e por uma central. Em que a comunicação flui entre estes de forma a manter toda a informação do sistema atualizada em tempo real. O autocarro recolherá informação, como quantos passageiros tem abordo e a sua localização geográfica por exemplo. A central receberá toda a informação de todos os autocarros existentes assim como de todas as paragens de autocarro existentes. Por sua vez a paragem de autocarro, recolherá dados também, tais como quantas pessoas estão na paragem, temperatura, humidade, emissões de dióxido de carbono, ruido, entre outros. A paragem de autocarro deverá contar com um conjunto de interfaces de comunicação, tais como Bluetooth e/ou NFC, hi-fi e RFID ou Beacons, para que possam ser feitas comunicações com os utilizadores, com os autocarros e com a central. Deverá ter também ecrãs interativos que poderão ser acedidos usando gestos e/ou toque e/ou voz para que se possam efetuar as ações pretendidas. A informação não será apenas transmitida nos ecrãs interativos, será transmitida também através de som. A informação contida na paragem pode ser de todo o tipo, desde as rotas, horários, posição atual do próximo autocarro, assim como o número do mesmo, publicidade animada, etc. A paragem conta também com outras funcionalidades como conectores onde se possam carregar dispositivos móveis, aquecimento, iluminação controlada face à afluência de utilizadores e horário, um sistema de armazenamento de energia pois deverá contar com fontes de energia renováveis para que possa ser o mais autossustentável possível, e obviamente câmeras de vigilância para segurança dos utilizadores. Sendo o principal objetivo deste trabalho, o desenvolvimento de um algoritmo capaz de contar quantas pessoas se encontram na paragem de autocarro, através do processamento das imagens vindas de várias câmaras, o foco principal é explorar as tecnologias de visão computacional e como estas podem ser utilizadas dentro do conceito da paragem de autocarro inteligente. Uma vez que o mundo da visão computacional evoluiu muito nos últimos anos e as suas aplicações são quase ilimitadas, dai a sua implementação nas mais diversas áreas, como reconstrução de cenários, deteção de eventos, monitorização de vídeo, reconhecimento de objetos, estimativa de movimento, restauração de imagem, etc. Ao combinar os diferentes algoritmos das diferentes aplicações, podem ser criadas ferramentas mais poderosas. Assim sendo o algoritmo desenvolvido utiliza redes neuronais convulsionais para detetar todas as pessoas de uma imagem, devolvendo uma região de interesse. Essa região de interesse é processada em busca de caras e caso estas existam essa informação é guardada no perfil da pessoa. Isto é possível através da utilização de reconhecimento facial, que utiliza um algoritmo de Deep Learning (DL). Essa região de interesse também é convertida para uma escala de cinzentos e posteriormente para uma matriz, essa matriz será também guardada no perfil do utilizador. Está informação é necessária para que se possa treinar um modelo que utiliza algoritmos de aprendizagem de máquina (Support Vector Machine - SVM). Os algoritmos de DL e SVM são necessários para que se possa fazer a identificação dos utilizadores a cada imagem e para que se possa cruzar os vários perfis vindos das várias origens, para que possa eliminar os perfis repetidos. Com isto a mesma pessoa é contada as vezes que apareça nas imagens, em função do número de câmeras existentes na paragem. Assim sendo é preciso eliminar essas repetições de forma a ter um número de pessoas correto. Num ambiente controlado o algoritmo proposto tem uma taxa de sucesso elevada, praticamente sem falhas, mas quando testado no ambiente para o qual foi desenhado já não é bem assim, ou porque numa paragem de autocarro as pessoas estão em contante movimento ou porque ficam na frente umas das outras e não é possível visualizá-las a todas. Mesmo com muitas câmeras colocadas no local, acabam sempre por haver pontos mortos, devido à estrutura da paragem ou até mesmo devido ao meio, por exemplo árvores ou um carro mal-estacionado, etc.Bus stops nowadays have to be increasingly at the user’s service, this thesis explores the fundamentals ideas of what a Smart Bus Stop should be and bring all together into one concept using today’s technologies. Although the fundamentals of a Smart Bus Stop (SBS) are explored, the primary focus here is to explore computer vision technology and how they can be used inside the Smart Bus Stop concept. The world of computer vision has evolved a lot in recent years and its applications are almost limitless, so they have been incorporated into many different areas like scene reconstruction, event detection, video tracking, object recognition, motion estimation, image restoration, etc. When combining the different algorithms of the different applications more powerful tools can be created. This work uses a Convolutional Neural Network (CNN) based algorithm to detect people in a multi video feeds. It also counts the number of persons in the SBS, using facial recognition, using with Deep Learning algorithm, and Support Vector Machine algorithm. It is important to stress, these last two are used to keep track of the user and also to remove the repeated profiles gathered in the different video sources, since the SBS is in a multi-camera environment. Combining these technologies was possible to count how many people were in the SBS. In laboratory the propose algorithm presents an extremely high success rate, when applied to real bus stops que success rate decreases due to blind spots for instance