Modeling the environment with egocentric vision systems

Cada vez más sistemas autónomos, ya sean robots o sistemas de asistencia, están presentes en nuestro día a día. Este tipo de sistemas interactúan y se relacionan con su entorno y para ello necesitan un modelo de dicho entorno. En función de las tareas que deben realizar, la información o el detalle necesario del modelo varía. Desde detallados modelos 3D para sistemas de navegación autónomos, a modelos semánticos que incluyen información importante para el usuario como el tipo de área o qué objetos están presentes. La creación de estos modelos se realiza a través de las lecturas de los distintos sensores disponibles en el sistema. Actualmente, gracias a su pequeño tamaño, bajo precio y la gran información que son capaces de capturar, las cámaras son sensores incluidos en todos los sistemas autónomos. El objetivo de esta tesis es el desarrollar y estudiar nuevos métodos para la creación de modelos del entorno a distintos niveles semánticos y con distintos niveles de precisión. Dos puntos importantes caracterizan el trabajo desarrollado en esta tesis: - El uso de cámaras con punto de vista egocéntrico o en primera persona ya sea en un robot o en un sistema portado por el usuario (wearable). En este tipo de sistemas, las cámaras son solidarias al sistema móvil sobre el que van montadas. En los últimos años han aparecido muchos sistemas de visión wearables, utilizados para multitud de aplicaciones, desde ocio hasta asistencia de personas. - El uso de sistemas de visión omnidireccional, que se distinguen por su gran campo de visión, incluyendo mucha más información en cada imagen que las cámara convencionales. Sin embargo plantean nuevas dificultades debido a distorsiones y modelos de proyección más complejos. Esta tesis estudia distintos tipos de modelos del entorno: - Modelos métricos: el objetivo de estos modelos es crear representaciones detalladas del entorno en las que localizar con precisión el sistema autónomo. Ésta tesis se centra en la adaptación de estos modelos al uso de visión omnidireccional, lo que permite capturar más información en cada imagen y mejorar los resultados en la localización. - Modelos topológicos: estos modelos estructuran el entorno en nodos conectados por arcos. Esta representación tiene menos precisión que la métrica, sin embargo, presenta un nivel de abstracción mayor y puede modelar el entorno con más riqueza. %, por ejemplo incluyendo el tipo de área de cada nodo, la localización de objetos importantes o el tipo de conexión entre los distintos nodos. Esta tesis se centra en la creación de modelos topológicos con información adicional sobre el tipo de área de cada nodo y conexión (pasillo, habitación, puertas, escaleras...). - Modelos semánticos: este trabajo también contribuye en la creación de nuevos modelos semánticos, más enfocados a la creación de modelos para aplicaciones en las que el sistema interactúa o asiste a una persona. Este tipo de modelos representan el entorno a través de conceptos cercanos a los usados por las personas. En particular, esta tesis desarrolla técnicas para obtener y propagar información semántica del entorno en secuencias de imágen

Mobile Robots Navigation

Mobile robots navigation includes different interrelated activities: (i) perception, as obtaining and interpreting sensory information; (ii) exploration, as the strategy that guides the robot to select the next direction to go; (iii) mapping, involving the construction of a spatial representation by using the sensory information perceived; (iv) localization, as the strategy to estimate the robot position within the spatial map; (v) path planning, as the strategy to find a path towards a goal location being optimal or not; and (vi) path execution, where motor actions are determined and adapted to environmental changes. The book addresses those activities by integrating results from the research work of several authors all over the world. Research cases are documented in 32 chapters organized within 7 categories next described

Detección y modelado de escaleras con sensor RGB-D para asistencia personal

La habilidad de avanzar y moverse de manera efectiva por el entorno resulta natural para la mayoría de la gente, pero no resulta fácil de realizar bajo algunas circunstancias, como es el caso de las personas con problemas visuales o cuando nos movemos en entornos especialmente complejos o desconocidos. Lo que pretendemos conseguir a largo plazo es crear un sistema portable de asistencia aumentada para ayudar a quienes se enfrentan a esas circunstancias. Para ello nos podemos ayudar de cámaras, que se integran en el asistente. En este trabajo nos hemos centrado en el módulo de detección, dejando para otros trabajos el resto de módulos, como podría ser la interfaz entre la detección y el usuario. Un sistema de guiado de personas debe mantener al sujeto que lo utiliza apartado de peligros, pero también debería ser capaz de reconocer ciertas características del entorno para interactuar con ellas. En este trabajo resolvemos la detección de uno de los recursos más comunes que una persona puede tener que utilizar a lo largo de su vida diaria: las escaleras. Encontrar escaleras es doblemente beneficioso, puesto que no sólo permite evitar posibles caídas sino que ayuda a indicar al usuario la posibilidad de alcanzar otro piso en el edificio. Para conseguir esto hemos hecho uso de un sensor RGB-D, que irá situado en el pecho del sujeto, y que permite captar de manera simultánea y sincronizada información de color y profundidad de la escena. El algoritmo usa de manera ventajosa la captación de profundidad para encontrar el suelo y así orientar la escena de la manera que aparece ante el usuario. Posteriormente hay un proceso de segmentación y clasificación de la escena de la que obtenemos aquellos segmentos que se corresponden con "suelo", "paredes", "planos horizontales" y una clase residual, de la que todos los miembros son considerados "obstáculos". A continuación, el algoritmo de detección de escaleras determina si los planos horizontales son escalones que forman una escalera y los ordena jerárquicamente. En el caso de que se haya encontrado una escalera, el algoritmo de modelado nos proporciona toda la información de utilidad para el usuario: cómo esta posicionada con respecto a él, cuántos escalones se ven y cuáles son sus medidas aproximadas. En definitiva, lo que se presenta en este trabajo es un nuevo algoritmo de ayuda a la navegación humana en entornos de interior cuya mayor contribución es un algoritmo de detección y modelado de escaleras que determina toda la información de mayor relevancia para el sujeto. Se han realizado experimentos con grabaciones de vídeo en distintos entornos, consiguiendo buenos resultados tanto en precisión como en tiempo de respuesta. Además se ha realizado una comparación de nuestros resultados con los extraídos de otras publicaciones, demostrando que no sólo se consigue una eciencia que iguala al estado de la materia sino que también se aportan una serie de mejoras. Especialmente, nuestro algoritmo es el primero capaz de obtener las dimensiones de las escaleras incluso con obstáculos obstruyendo parcialmente la vista, como puede ser gente subiendo o bajando. Como resultado de este trabajo se ha elaborado una publicación aceptada en el Second Workshop on Assitive Computer Vision and Robotics del ECCV, cuya presentación tiene lugar el 12 de Septiembre de 2014 en Zúrich, Suiza

Fuzzy Logic Approach for Mobile Robot in Intelligent Space

This project introduces the fuzzy logic approach for mobile robot in intelligent space. There are three major algorithms involved. They are known as object classification, object tracking and obstacle avoidance. The inputs are received from cameras which are mounted at a ceiling. The main idea of the object classification is to classify object into three categories depending upon their colors; the categories are mobile robot, destinations and obstacle position. These categories are represented by X symbol with different colors. This system is to teach and train the mobile robot proceeding to destination without hitting the obstacle. The mobile robot is autonomous; that means, it could be pursuing to the target position automatically without user guided. In this project, fuzzy logic is use to guide the mobile robot direction until it reaches the target position. This system is generates in real-time and suitable for indoor environment applications. One of the unique advantages of this project is that it only uses, there only used a camera and image processing generated by the algorithms itself without additional sensor such as sonar or IR sensor

CHARMIE: a collaborative healthcare and home service and assistant robot for elderly care

The global population is ageing at an unprecedented rate. With changes in life expectancy across the world, three major issues arise: an increasing proportion of senior citizens; cognitive and physical problems progressively affecting the elderly; and a growing number of single-person households. The available data proves the ever-increasing necessity for efficient elderly care solutions such as healthcare service and assistive robots. Additionally, such robotic solutions provide safe healthcare assistance in public health emergencies such as the SARS-CoV-2 virus (COVID-19). CHARMIE is an anthropomorphic collaborative healthcare and domestic assistant robot capable of performing generic service tasks in non-standardised healthcare and domestic environment settings. The combination of its hardware and software solutions demonstrates map building and self-localisation, safe navigation through dynamic obstacle detection and avoidance, different human-robot interaction systems, speech and hearing, pose/gesture estimation and household object manipulation. Moreover, CHARMIE performs end-to-end chores in nursing homes, domestic houses, and healthcare facilities. Some examples of these chores are to help users transport items, fall detection, tidying up rooms, user following, and set up a table. The robot can perform a wide range of chores, either independently or collaboratively. CHARMIE provides a generic robotic solution such that older people can live longer, more independent, and healthier lives.This work has been supported by FCT—Fundação para a Ciência e Tecnologia within the R&D Units Project Scope: UIDB/00319/2020. The author T.R. received funding through a doctoral scholarship from the Portuguese Foundation for Science and Technology (Fundação para a Ciência e a Tecnologia) [grant number SFRH/BD/06944/2020], with funds from the Portuguese Ministry of Science, Technology and Higher Education and the European Social Fund through the Programa Operacional do Capital Humano (POCH). The author F.G. received funding through a doctoral scholarship from the Portuguese Foundation for Science and Technology (Fundação para a Ciência e a Tecnologia) [grant number SFRH/BD/145993/2019], with funds from the Portuguese Ministry of Science, Technology and Higher Education and the European Social Fund through the Programa Operacional do Capital Humano (POCH)

Locomotion system for ground mobile robots in uneven and unstructured environments

One of the technology domains with the greatest growth rates nowadays is service robots. The extensive use of ground mobile robots in environments that are unstructured or structured for humans is a promising challenge for the coming years, even though Automated Guided Vehicles (AGV) moving on flat and compact grounds are already commercially available and widely utilized to move components and products inside indoor industrial buildings. Agriculture, planetary exploration, military operations, demining, intervention in case of terrorist attacks, surveillance, and reconnaissance in hazardous conditions are important application domains. Due to the fact that it integrates the disciplines of locomotion, vision, cognition, and navigation, the design of a ground mobile robot is extremely interdisciplinary. In terms of mechanics, ground mobile robots, with the exception of those designed for particular surroundings and surfaces (such as slithering or sticky robots), can move on wheels (W), legs (L), tracks (T), or hybrids of these concepts (LW, LT, WT, LWT). In terms of maximum speed, obstacle crossing ability, step/stair climbing ability, slope climbing ability, walking capability on soft terrain, walking capability on uneven terrain, energy efficiency, mechanical complexity, control complexity, and technology readiness, a systematic comparison of these locomotion systems is provided in [1]. Based on the above-mentioned classification, in this thesis, we first introduce a small-scale hybrid locomotion robot for surveillance and inspection, WheTLHLoc, with two tracks, two revolving legs, two active wheels, and two passive omni wheels. The robot can move in several different ways, including using wheels on the flat, compact ground,[1] tracks on soft, yielding terrain, and a combination of tracks, legs, and wheels to navigate obstacles. In particular, static stability and non-slipping characteristics are considered while analyzing the process of climbing steps and stairs. The experimental test on the first prototype has proven the planned climbing maneuver’s efficacy and the WheTLHLoc robot's operational flexibility. Later we present another development of WheTLHLoc and introduce WheTLHLoc 2.0 with newly designed legs, enabling the robot to deal with bigger obstacles. Subsequently, a single-track bio-inspired ground mobile robot's conceptual and embodiment designs are presented. This robot is called SnakeTrack. It is designed for surveillance and inspection activities in unstructured environments with constrained areas. The vertebral column has two end modules and a variable number of vertebrae linked by compliant joints, and the surrounding track is its essential component. Four motors drive the robot: two control the track motion and two regulate the lateral flexion of the vertebral column for steering. The compliant joints enable limited passive torsion and retroflection of the vertebral column, which the robot can use to adapt to uneven terrain and increase traction. Eventually, the new version of SnakeTrack, called 'Porcospino', is introduced with the aim of allowing the robot to move in a wider variety of terrains. The novelty of this thesis lies in the development and presentation of three novel designs of small-scale mobile robots for surveillance and inspection in unstructured environments, and they employ hybrid locomotion systems that allow them to traverse a variety of terrains, including soft, yielding terrain and high obstacles. This thesis contributes to the field of mobile robotics by introducing new design concepts for hybrid locomotion systems that enable robots to navigate challenging environments. The robots presented in this thesis employ modular designs that allow their lengths to be adapted to suit specific tasks, and they are capable of restoring their correct position after falling over, making them highly adaptable and versatile. Furthermore, this thesis presents a detailed analysis of the robots' capabilities, including their step-climbing and motion planning abilities. In this thesis we also discuss possible refinements for the robots' designs to improve their performance and reliability. Overall, this thesis's contributions lie in the design and development of innovative mobile robots that address the challenges of surveillance and inspection in unstructured environments, and the analysis and evaluation of these robots' capabilities. The research presented in this thesis provides a foundation for further work in this field, and it may be of interest to researchers and practitioners in the areas of robotics, automation, and inspection. As a general note, the first robot, WheTLHLoc, is a hybrid locomotion robot capable of combining tracked locomotion on soft terrains, wheeled locomotion on flat and compact grounds, and high obstacle crossing capability. The second robot, SnakeTrack, is a small-size mono-track robot with a modular structure composed of a vertebral column and a single peripherical track revolving around it. The third robot, Porcospino, is an evolution of SnakeTrack and includes flexible spines on the track modules for improved traction on uneven but firm terrains, and refinements of the shape of the track guidance system. This thesis provides detailed descriptions of the design and prototyping of these robots and presents analytical and experimental results to verify their capabilities

Real-time trajectory generation for dynamic systems with nonholonomic constraints using Player/Stage and NTG.

This thesis will present various methods of trajectory generation for various types of mobile robots. Then it will progress to evaluating Robot Operating Systems (ROS’s) that can be used to control and simulate mobile robots, and it will explain why Player/Stage was chosen as the ROS for this thesis. It will then discuss Nonlinear Trajectory Generation as the main method for producing a path for mobile robots with dynamic and kinematic constraints. Finally, it will combine Player, Stage, and NTG into a system that produces a trajectory in real-time for a mobile robot and simulates a differential drive robot being driven from the initial state to the goal state in the presence of obstacles. Experiments will include the following: Blobfinding for physical and simulated camera systems, position control of physical and simulated differential drive robots, wall following using simulated range sensors, trajectory generation for omnidirectional and differential drive robots, and a combination of blobfinding, position control, and trajectory generation. Each experiment was a success, to varying degrees. The culmination of the thesis will present a real-time trajectory generation and position control method for a differential drive robot in the presence of obstacles

Climbing and Walking Robots

Nowadays robotics is one of the most dynamic fields of scientific researches. The shift of robotics researches from manufacturing to services applications is clear. During the last decades interest in studying climbing and walking robots has been increased. This increasing interest has been in many areas that most important ones of them are: mechanics, electronics, medical engineering, cybernetics, controls, and computers. Today’s climbing and walking robots are a combination of manipulative, perceptive, communicative, and cognitive abilities and they are capable of performing many tasks in industrial and non- industrial environments. Surveillance, planetary exploration, emergence rescue operations, reconnaissance, petrochemical applications, construction, entertainment, personal services, intervention in severe environments, transportation, medical and etc are some applications from a very diverse application fields of climbing and walking robots. By great progress in this area of robotics it is anticipated that next generation climbing and walking robots will enhance lives and will change the way the human works, thinks and makes decisions. This book presents the state of the art achievments, recent developments, applications and future challenges of climbing and walking robots. These are presented in 24 chapters by authors throughtot the world The book serves as a reference especially for the researchers who are interested in mobile robots. It also is useful for industrial engineers and graduate students in advanced study