Real-time 2D–3D door detection and state classification on a low-power device

In this paper, we propose three methods for door state classifcation with the goal to improve robot navigation in indoor spaces. These methods were also developed to be used in other areas and applications since they are not limited to door detection as other related works are. Our methods work ofine, in low-powered computers as the Jetson Nano, in real-time with the ability to diferentiate between open, closed and semi-open doors. We use the 3D object classifcation, PointNet, real-time semantic segmentation algorithms such as, FastFCN, FC-HarDNet, SegNet and BiSeNet, the object detection algorithm, DetectNet and 2D object classifcation networks, AlexNet and GoogleNet. We built a 3D and RGB door dataset with images from several indoor environments using a 3D Realsense camera D435. This dataset is freely available online. All methods are analysed taking into account their accuracy and the speed of the algorithm in a low powered computer. We conclude that it is possible to have a door classifcation algorithm running in real-time on a low-power device.info:eu-repo/semantics/publishedVersio

An Image Understanding System for Detecting Indoor Features

The capability of identifying physical structures of an unknown environment is very important for vision based robot navigation and scene understanding. Among physical structures in indoor environments, corridor lines and doors are important visual landmarks for robot navigation since they show the topological structure in an indoor environment and establish connections among the different places or regions in the indoor environment. Furthermore, they provide clues for understanding the image. In this thesis, I present two algorithms to detect the vanishing point, corridor lines, and doors respectively using a single digital video camera. In both algorithms, we utilize a hypothesis generation and verification method to detect corridor and door structures using low level linear features. The proposed method consists of low, intermediate, and high level processing stages which correspond to the extraction of low level features, the formation of hypotheses, and verification of the hypotheses via seeking evidence actively. In particular, we extend this single-pass framework by employing a feedback strategy for more robust hypothesis generation and verification. We demonstrate the robustness of the proposed methods on a large number of real video images in a variety of corridor environments, with image acquisitions under different illumination and reflection conditions, with different moving speeds, and with different viewpoints of the camera. Experimental results performed on the corridor line detection algorithm validate that the method can detect corridor line locations in the presence of many spurious line features about one second. Experimental results carried on the door detection algorithm show that the system can detect visually important doors in an image with a very high accuracy rate when a robot navigates along a corridor environment

Navigation semi-autonome d'un fauteuil roulant motorisé en environnements restreints

RÉSUMÉ Ce travail introduit un système de navigation semi-autonome pour fauteuil roulant motorisé (FRM) visant à assister les personnes à mobilité réduite dans leurs déplacements quotidiens. Différentes fonctionnalités d’assistance à la navigation sont offertes, telles que l’assistance d’évitement de collision et de déblocage lors d’impasses, puis l’exécution de manoeuvres automatiques : de mouvements rectilignes, de suivi de murs, de suivi de personnes, de traversée de passages étroits et de stationnement. Élaboré avec l’appui de professionnels en réadaptation et d’usagers de FRM, le système de navigation proposé se distingue par sa facilité d’appropriation attribuable à des fonctionnalités simples à maîtriser, par sa navigation intuitive, confortable et sécuritaire, puis par ses aptitudes à opérer en environnements restreints, variés et inconnus. Le système est validé à l’aide du WST (Wheelchair Skills Test) en collaboration avec des professionnels en réadaptation du centre Lucie-Bruneau de Montréal. Huit usagers sains et neuf usagers handicapés ont exécuté le test. Les 8.8 kilomètres parcourus sur vingt-cinq heures d’expérimentation ont démontré l’efficacité et la fiabilité de la solution proposée. Les commentaires positifs des usagers confirment l’intérêt des différentes facettes du système. Les expériences ont, en outre, démontré une aptitude exceptionnelle à la réalisation automatique de manoeuvres en environnements restreints, comme lors de passage de portes standards (moins de 90 cm de large) et de stationnement. Ces performances sont rendues possibles en redéfinissant comment une plate-forme mobile doit se mouvoir dans l’espace, réagir aux obstacles, gérer l’incertitude des mesures, compenser l’erreur d’exécution, établir des stratégies valides dans la plupart des contextes liés aux fonctionnalités et, enfin, organiser l’ensemble des fonctionnalités sous une architecture de contrôle facilitant leur développement. Ce travail contribue ainsi à la robotique mobile par l’apport d’approches de suivi de séquence de points (et de trajectoires), d’évitement stratégique de collisions et d’impasses, de stationnement parallèle, de traversée de passages étroits, d’autolocalisation en environnement partiellement inconnu et par une architecture de contrôle sous laquelle opère l’ensemble des fonctionnalités. Ces méthodes se distinguent par leur capacité à s’adapter à des contextes variés et restreints, puis par leur simplicité d’utilisation. Chaque approche est étudiée individuellement, puis conjointement dans l’application de fauteuil roulant semi-autonome.----------ABSTRACT This work presents a semi-autonomous navigation system for powered wheelchairs which provides assistance to people with mobility impairments during their activities of daily living. The system has various functionalities such as collision assistance, anti-deadlock assistance, automatic maneuvers including moving on a distance, following a wall, following a person, passing through a narrow passage and parking in restrained areas. Built in regard to the advices of clinicians and wheelchair users, the proposed navigation system is especially easy to run due to a set of easy to- understand functions providing an intuitive, comfortable and secure navigation. It is capable to operate in various restrained and unknown environments. The system has been validated using the Wheelchair Skills Test (WST) by clinicians from the Lucie-Bruneau rehabilitation center in Montreal. Eight healty users and nine impaired users have done the test. The 8.8 kilometers covered in 25 hours of experimentation have shown the efficiency and the reliability of the proposed solution. The positive user comments point out the relevance of the different system features. Specifically, the experiments have demonstrated the system’s exceptional skills in carrying out automatic maneuvers in restrained environments, such as passing standard doors (width less than 90 cm) and parking. The special system’s efficiency to perform in various restrained environments is made possible by redefining how a mobile platform should move, react to obstacles, deal with measurement uncertainties, compensate for execution errors, set up strategies that are applicable in most operation contexts and organize the overall functionalities into an efficient control architecture. The work resulting from this investigation contributes to the mobile robotic field by proposing novel approaches for the problems of waypoint-following (and path-following), collision assistance, deadlock avoidance, parallel parking, narrow passages and self-localization, as well as for a special control architecture that underlies all functionalities. All these approaches differ from previous ones in their adaptability to varied and restrained contexts, and for their simplicity of use. The approaches are first studied separately, next jointly in the semi-autonomous navigation system