156 research outputs found
Multi-Scale Hierarchical Conditional Random Field for Railway Electrification Scene Classification Using Mobile Laser Scanning Data
With the recent rapid development of high-speed railway in many countries, precise inspection for railway electrification systems has become more significant to ensure safe railway operation. However, this time-consuming manual inspection is not satisfactory for the high-demanding inspection task, thus a safe, fast and automatic inspection method is required. With LiDAR (Light Detection and Ranging) data becoming more available, the accurate railway electrification scene understanding using LiDAR data becomes feasible towards automatic 3D precise inspection.
This thesis presents a supervised learning method to classify railway electrification objects from Mobile Laser Scanning (MLS) data. First, a multi-range Conditional Random Field (CRF), which characterizes not only labeling homogeneity at a short range, but also the layout compatibility between different objects at a middle range in the probabilistic graphical model is implemented and tested. Then, this multi-range CRF model will be extended and improved into a hierarchical CRF model to consider multi-scale layout compatibility at full range. The proposed method is evaluated on a dataset collected in Korea with complex railway electrification systems environment. The experiment shows the effectiveness of proposed model
Person Detection for an Orthogonally Placed Monocular Camera
Counting of passengers entering and exiting means of transport is one of the basic functionalities of passenger flow monitoring systems. Exact numbers of passengers are important in areas such as public transport surveillance, passenger flow prediction, transport planning, and transport vehicle load monitoring. To allow mass utilization of passenger flow monitoring systems, their cost must be low. As the overall price is mainly given by prices of the used sensor and processing unit, we propose the utilization of a visible spectrum camera and data processing algorithms of low time complexity to ensure a low price of the final product. To guarantee the anonymity of passengers, we suggest orthogonal scanning of a scene. As the precision of the counting is relevantly influenced by the precision of passenger recognition, we focus on the development of an appropriate recognition method. We present two opposite approaches which can be used for the passenger recognition in means of transport with and without entrance steps, or with split level flooring. The first approach is the utilization of an appropriate convolutional neural network (ConvNet), which is currently the prevailing approach in computer vision. The second approach is the utilization of histograms of oriented gradients (HOG) features in combination with a support vector machine classifier. This approach is a representative of classical methods. We study both approaches in terms of practical applications, where real-time processing of data is one of the basic assumptions. Specifically, we examine classification performance and time complexity of the approaches for various topologies and settings, respectively. For this purpose, we form and make publicly available a large-scale, class-balanced dataset of labelled RGB images. We demonstrate that, compared to ConvNets, the HOG-based passenger recognition is more suitable for practical applications. For an appropriate setting, it defeats the ConvNets in terms of time complexity while keeping excellent classification performance. To allow verification of theoretical findings, we construct an engineering prototype of the system
3D Reconstruction of Indoor Corridor Models Using Single Imagery and Video Sequences
In recent years, 3D indoor modeling has gained more attention due to its role in decision-making process of maintaining the status and managing the security of building indoor spaces. In this thesis, the problem of continuous indoor corridor space modeling has been tackled through two approaches. The first approach develops a modeling method based on middle-level perceptual organization. The second approach develops a visual Simultaneous Localisation and Mapping (SLAM) system with model-based loop closure.
In the first approach, the image space was searched for a corridor layout that can be converted into a geometrically accurate 3D model. Manhattan rule assumption was adopted, and indoor corridor layout hypotheses were generated through a random rule-based intersection of image physical line segments and virtual rays of orthogonal vanishing points. Volumetric reasoning, correspondences to physical edges, orientation map and geometric context of an image are all considered for scoring layout hypotheses. This approach provides physically plausible solutions while facing objects or occlusions in a corridor scene.
In the second approach, Layout SLAM is introduced. Layout SLAM performs camera localization while maps layout corners and normal point features in 3D space. Here, a new feature matching cost function was proposed considering both local and global context information. In addition, a rotation compensation variable makes Layout SLAM robust against cameras orientation errors accumulations. Moreover, layout model matching of keyframes insures accurate loop closures that prevent miss-association of newly visited landmarks to previously visited scene parts.
The comparison of generated single image-based 3D models to ground truth models showed that average ratio differences in widths, heights and lengths were 1.8%, 3.7% and 19.2% respectively. Moreover, Layout SLAM performed with the maximum absolute trajectory error of 2.4m in position and 8.2 degree in orientation for approximately 318m path on RAWSEEDS data set. Loop closing was strongly performed for Layout SLAM and provided 3D indoor corridor layouts with less than 1.05m displacement errors in length and less than 20cm in width and height for approximately 315m path on York University data set. The proposed methods can successfully generate 3D indoor corridor models compared to their major counterpart
Recent Advances in Indoor Localization Systems and Technologies
Despite the enormous technical progress seen in the past few years, the maturity of indoor localization technologies has not yet reached the level of GNSS solutions. The 23 selected papers in this book present the recent advances and new developments in indoor localization systems and technologies, propose novel or improved methods with increased performance, provide insight into various aspects of quality control, and also introduce some unorthodox positioning methods
Human-Centric Machine Vision
Recently, the algorithms for the processing of the visual information have greatly evolved, providing efficient and effective solutions to cope with the variability and the complexity of real-world environments. These achievements yield to the development of Machine Vision systems that overcome the typical industrial applications, where the environments are controlled and the tasks are very specific, towards the use of innovative solutions to face with everyday needs of people. The Human-Centric Machine Vision can help to solve the problems raised by the needs of our society, e.g. security and safety, health care, medical imaging, and human machine interface. In such applications it is necessary to handle changing, unpredictable and complex situations, and to take care of the presence of humans
Scene understanding for autonomous robots operating in indoor environments
Mención Internacional en el título de doctorThe idea of having robots among us is not new. Great efforts are continually made to
replicate human intelligence, with the vision of having robots performing different activities,
including hazardous, repetitive, and tedious tasks. Research has demonstrated that robots are
good at many tasks that are hard for us, mainly in terms of precision, efficiency, and speed.
However, there are some tasks that humans do without much effort that are challenging for
robots. Especially robots in domestic environments are far from satisfactorily fulfilling some
tasks, mainly because these environments are unstructured, cluttered, and with a variety of
environmental conditions to control.
This thesis addresses the problem of scene understanding in the context of autonomous
robots operating in everyday human environments. Furthermore, this thesis is developed
under the HEROITEA research project that aims to develop a robot system to help
elderly people in domestic environments as an assistant. Our main objective is to develop
different methods that allow robots to acquire more information from the environment to
progressively build knowledge that allows them to improve the performance on high-level
robotic tasks. In this way, scene understanding is a broad research topic, and it is considered
a complex task due to the multiple sub-tasks that are involved. In that context, in this thesis,
we focus on three sub-tasks: object detection, scene recognition, and semantic segmentation
of the environment.
Firstly, we implement methods to recognize objects considering real indoor environments.
We applied machine learning techniques incorporating uncertainties and more modern
techniques based on deep learning. Besides, apart from detecting objects, it is essential to
comprehend the scene where they can occur. For this reason, we propose an approach
for scene recognition that considers the influence of the detected objects in the prediction
process. We demonstrate that the exiting objects and their relationships can improve the
inference about the scene class. We also consider that a scene recognition model can
benefit from the advantages of other models. We propose a multi-classifier model for scene
recognition based on weighted voting schemes. The experiments carried out in real-world
indoor environments demonstrate that the adequate combination of independent classifiers
allows obtaining a more robust and precise model for scene recognition.
Moreover, to increase the understanding of a robot about its surroundings, we propose
a new division of the environment based on regions to build a useful representation of
the environment. Object and scene information is integrated into a probabilistic fashion
generating a semantic map of the environment containing meaningful regions within each
room. The proposed system has been assessed on simulated and real-world domestic
scenarios, demonstrating its ability to generate consistent environment representations.
Lastly, full knowledge of the environment can enhance more complex robotic tasks; that is
why in this thesis, we try to study how a complete knowledge of the environment influences
the robot’s performance in high-level tasks. To do so, we select an essential task, which
is searching for objects. This mundane task can be considered a precondition to perform
many complex robotic tasks such as fetching and carrying, manipulation, user requirements,
among others. The execution of these activities by service robots needs full knowledge of
the environment to perform each task efficiently. In this thesis, we propose two searching
strategies that consider prior information, semantic representation of the environment, and
the relationships between known objects and the type of scene. All our developments are
evaluated in simulated and real-world environments, integrated with other systems, and
operating in real platforms, demonstrating their feasibility to implement in real scenarios, and
in some cases outperforming other approaches. We also demonstrate how our representation
of the environment can boost the performance of more complex robotic tasks compared to
more standard environmental representations.La idea de tener robots entre nosotros no es nueva. Continuamente se realizan grandes
esfuerzos para replicar la inteligencia humana, con la visión de tener robots que realicen
diferentes actividades, incluidas tareas peligrosas, repetitivas y tediosas. La investigación ha
demostrado que los robots son buenos en muchas tareas que resultan difíciles para nosotros,
principalmente en términos de precisión, eficiencia y velocidad. Sin embargo, existen tareas
que los humanos realizamos sin mucho esfuerzo y que son un desafío para los robots.
Especialmente, los robots en entornos domésticos están lejos de cumplir satisfactoriamente
algunas tareas, principalmente porque estos entornos no son estructurados, pueden estar
desordenados y cuentan con una gran variedad de condiciones ambientales que controlar.
Esta tesis aborda el problema de la comprensión de la escena en el contexto de robots
autónomos que operan en entornos humanos cotidianos. Asimismo, esta tesis se desarrolla
en el marco del proyecto de investigación HEROITEA que tiene como objetivo desarrollar
un sistema robótico que funcione como asistente para ayudar a personas mayores en entornos
domésticos. Nuestro principal objetivo es desarrollar diferentes métodos que permitan a
los robots adquirir más información del entorno a fin de construir progresivamente un
conocimiento que les permita mejorar su desempeño en tareas robóticas más complejas.
En este sentido, la comprensión de escenas es un tema de investigación amplio, y se
considera una tarea compleja debido a las múltiples subtareas involucradas. En esta tesis
nos enfocamos específicamente en tres subtareas: detección de objetos, reconocimiento de
escenas y etiquetado semántico del entorno.
Por un lado, implementamos métodos para el reconocimiento de objectos considerando
entornos interiores reales. Aplicamos técnicas de aprendizaje automático incorporando
incertidumbres y técnicas más modernas basadas en aprendizaje profundo. Además, aparte
de detectar objetos, es fundamental comprender la escena donde estos se encuentran. Por esta
razón, proponemos un modelo para el reconocimiento de escenas que considera la influencia
de los objetos detectados en el proceso de predicción. Demostramos que los objetos existentes
y sus relaciones pueden mejorar el proceso de inferencia de la categoría de la escena. También
consideramos que un modelo de reconocimiento de escenas puede beneficiarse de las ventajas
de otros modelos. Por ello, proponemos un multiclasificador para el reconocimiento de escenas basado en esquemas de votación ponderados. Los experimentos llevados a cabo
en entornos interiores reales demuestran que la combinación adecuada de clasificadores
independientes permite obtener un modelo más robusto y preciso para el reconocimiento
de escenas.
Adicionalmente, para aumentar la comprensión de un robot acerca de su entorno,
proponemos una nueva división del entorno basada en regiones a fin de construir una
representación útil del entorno. La información de objetos y de la escena se integra de forma
probabilística generando un mapa semántico que contiene regiones significativas dentro de
cada habitación. El sistema propuesto ha sido evaluado en entornos domésticos simulados y
reales, demostrando su capacidad para generar representaciones consistentes del entorno.
Por otro lado, el conocimiento integral del entorno puede mejorar tareas robóticas más
complejas; es por ello que en esta tesis analizamos cómo el conocimiento completo del
entorno influye en el desempeño del robot en tareas de alto nivel. Para ello, seleccionamos una
tarea fundamental, que es la búsqueda de objetos. Esta tarea mundana puede considerarse
una condición previa para realizar diversas tareas robóticas complejas, como transportar
objetos, tareas de manipulación, atender requerimientos del usuario, entre otras. La
ejecución de estas actividades por parte de robots de servicio requiere un conocimiento
profundo del entorno para realizar cada tarea de manera eficiente. En esta tesis proponemos
dos estrategias de búsqueda de objetos que consideran información previa, la representación
semántica del entorno, las relaciones entre los objetos conocidos y el tipo de escena. Todos
nuestros desarrollos son evaluados en entornos simulados y reales, integrados con otros
sistemas y operando en plataformas reales, demostrando su viabilidad de ser implementados
en escenarios reales y, en algunos casos, superando a otros enfoques. También demostramos
cómo nuestra representación del entorno puede mejorar el desempeño de tareas robóticas
más complejas en comparación con representaciones del entorno más tradicionales.Programa de Doctorado en Ingeniería Eléctrica, Electrónica y Automática por la Universidad Carlos III de MadridPresidente: Carlos Balaguer Bernaldo de Quirós.- Secretario: Fernando Matía Espada.- Vocal: Klaus Strob
Automating Inspection of Tunnels With Photogrammetry and Deep Learning
Asset Management of large underground transportation infrastructure requires frequent and detailed inspections to assess its overall structural conditions and to focus available funds where required. At the time of writing, the common approach to perform visual inspections is heavily manual, therefore slow, expensive, and highly subjective.
This research evaluates the applicability of an automated pipeline to perform visual inspections of underground infrastructure for asset management purposes. It also analyses the benefits of using lightweight and low-cost hardware versus high-end technology. The aim is to increase the automation in performing such task to overcome the main drawbacks of the traditional regime. It replaces subjectivity, approximation and limited repeatability of the manual inspection with objectivity and consistent accuracy. Moreover, it reduces the overall end-to-end time required for the inspection and the associated costs. This might translate to more frequent inspections per given budget, resulting in increased service life of the infrastructure. Shorter inspections have social benefits as well. In fact, local communities can rely on a safe transportation with minimum levels of disservice. At last, but not least, it drastically improves health and safety conditions for the inspection engineers who need to spend less time in this hazardous environment.
The proposed pipeline combines photogrammetric techniques for photo-realistic 3D reconstructions alongside with machine learning-based defect detection algorithms. This approach allows to detect and map visible defects on the tunnel’s lining in local coordinate system and provides the asset manager with a clear overview of the critical areas over all infrastructure.
The outcomes of the research show that the accuracy of the proposed pipeline largely outperforms human results, both in three-dimensional mapping and defect detection performance, pushing the benefit-cost ratio strongly in favour of the automated approach. Such outcomes will impact the way construction industry approaches visual inspections and shift towards automated strategies
Automated Semantic Content Extraction from Images
In this study, an automatic semantic segmentation and object recognition methodology is implemented which bridges the semantic gap between low level features of image content and high level conceptual meaning. Semantically understanding an image is essential in modeling autonomous robots, targeting customers in marketing or reverse engineering of building information modeling in the construction industry. To achieve an understanding of a room from a single image we proposed a new object recognition framework which has four major components: segmentation, scene detection, conceptual cueing and object recognition. The new segmentation methodology developed in this research extends Felzenswalb\u27s cost function to include new surface index and depth features as well as color, texture and normal features to overcome issues of occlusion and shadowing commonly found in images. Adding depth allows capturing new features for object recognition stage to achieve high accuracy compared to the current state of the art. The goal was to develop an approach to capture and label perceptually important regions which often reflect global representation and understanding of the image. We developed a system by using contextual and common sense information for improving object recognition and scene detection, and fused the information from scene and objects to reduce the level of uncertainty. This study in addition to improving segmentation, scene detection and object recognition, can be used in applications that require physical parsing of the image into objects, surfaces and their relations. The applications include robotics, social networking, intelligence and anti-terrorism efforts, criminal investigations and security, marketing, and building information modeling in the construction industry. In this dissertation a structural framework (ontology) is developed that generates text descriptions based on understanding of objects, structures and the attributes of an image
Fusion of Data from Heterogeneous Sensors with Distributed Fields of View and Situation Evaluation for Advanced Driver Assistance Systems
In order to develop a driver assistance system for pedestrian protection, pedestrians in the environment of a truck are detected by radars and a camera and are tracked across distributed fields of view using a Joint Integrated Probabilistic Data Association filter. A robust approach for prediction of the system vehicles trajectory is presented. It serves the computation of a probabilistic collision risk based on reachable sets where different sources of uncertainty are taken into account
Intelligent Transportation Related Complex Systems and Sensors
Building around innovative services related to different modes of transport and traffic management, intelligent transport systems (ITS) are being widely adopted worldwide to improve the efficiency and safety of the transportation system. They enable users to be better informed and make safer, more coordinated, and smarter decisions on the use of transport networks. Current ITSs are complex systems, made up of several components/sub-systems characterized by time-dependent interactions among themselves. Some examples of these transportation-related complex systems include: road traffic sensors, autonomous/automated cars, smart cities, smart sensors, virtual sensors, traffic control systems, smart roads, logistics systems, smart mobility systems, and many others that are emerging from niche areas. The efficient operation of these complex systems requires: i) efficient solutions to the issues of sensors/actuators used to capture and control the physical parameters of these systems, as well as the quality of data collected from these systems; ii) tackling complexities using simulations and analytical modelling techniques; and iii) applying optimization techniques to improve the performance of these systems. It includes twenty-four papers, which cover scientific concepts, frameworks, architectures and various other ideas on analytics, trends and applications of transportation-related data
- …