A Survey on Audio-Video based Defect Detection through Deep Learning in Railway Maintenance

Within Artificial Intelligence, Deep Learning (DL) represents a paradigm that has been showing unprecedented performance in image and audio processing by supporting or even replacing humans in defect and anomaly detection. The Railway sector is expected to benefit from DL applications, especially in predictive maintenance applications, where smart audio and video sensors can be leveraged yet kept distinct from safety-critical functions. Such separation is crucial, as it allows for improving system dependability with no impact on its safety certification. This is further supported by the development of DL in other transportation domains, such as automotive and avionics, opening for knowledge transfer opportunities and highlighting the potential of such a paradigm in railways. In order to summarize the recent state-of-the-art while inquiring about future opportunities, this paper reviews DL approaches for the analysis of data generated by acoustic and visual sensors in railway maintenance applications that have been published until August 31st, 2021. In this paper, the current state of the research is investigated and evaluated using a structured and systematic method, in order to highlight promising approaches and successful applications, as well as to identify available datasets, current limitations, open issues, challenges, and recommendations about future research directions

Lane detection in autonomous vehicles : A systematic review

One of the essential systems in autonomous vehicles for ensuring a secure circumstance for drivers and passengers is the Advanced Driver Assistance System (ADAS). Adaptive Cruise Control, Automatic Braking/Steer Away, Lane-Keeping System, Blind Spot Assist, Lane Departure Warning System, and Lane Detection are examples of ADAS. Lane detection displays information specific to the geometrical features of lane line structures to the vehicle's intelligent system to show the position of lane markings. This article reviews the methods employed for lane detection in an autonomous vehicle. A systematic literature review (SLR) has been carried out to analyze the most delicate approach to detecting the road lane for the benefit of the automation industry. One hundred and two publications from well-known databases were chosen for this review. The trend was discovered after thoroughly examining the selected articles on the method implemented for detecting the road lane from 2018 until 2021. The selected literature used various methods, with the input dataset being one of two types: self-collected or acquired from an online public dataset. In the meantime, the methodologies include geometric modeling and traditional methods, while AI includes deep learning and machine learning. The use of deep learning has been increasingly researched throughout the last four years. Some studies used stand-Alone deep learning implementations for lane detection problems. Meanwhile, some research focuses on merging deep learning with other machine learning techniques and classical methodologies. Recent advancements imply that attention mechanism has become a popular combined strategy with deep learning methods. The use of deep algorithms in conjunction with other techniques showed promising outcomes. This research aims to provide a complete overview of the literature on lane detection methods, highlighting which approaches are currently being researched and the performance of existing state-of-The-Art techniques. Also, the paper covered the equipment used to collect the dataset for the training process and the dataset used for network training, validation, and testing. This review yields a valuable foundation on lane detection techniques, challenges, and opportunities and supports new research works in this automation field. For further study, it is suggested to put more effort into accuracy improvement, increased speed performance, and more challenging work on various extreme conditions in detecting the road lane

Applications in Electronics Pervading Industry, Environment and Society

This book features the manuscripts accepted for the Special Issue “Applications in Electronics Pervading Industry, Environment and Society—Sensing Systems and Pervasive Intelligence” of the MDPI journal Sensors. Most of the papers come from a selection of the best papers of the 2019 edition of the “Applications in Electronics Pervading Industry, Environment and Society” (APPLEPIES) Conference, which was held in November 2019. All these papers have been significantly enhanced with novel experimental results. The papers give an overview of the trends in research and development activities concerning the pervasive application of electronics in industry, the environment, and society. The focus of these papers is on cyber physical systems (CPS), with research proposals for new sensor acquisition and ADC (analog to digital converter) methods, high-speed communication systems, cybersecurity, big data management, and data processing including emerging machine learning techniques. Physical implementation aspects are discussed as well as the trade-off found between functional performance and hardware/system costs

Detecting human engagement propensity in human-robot interaction

Elaborazione di immagini ricavate dal flusso di una semplice videocamera RGB di un robot al fine di stimare la propensione all'interazione di una persona in situazioni di interazione uomo-robot. Per calcolare la stima finale, tecniche basate su deep learning sono usate per estrarre alcune informazioni ausiliarie come: stima della posa di una persona, quale tipo di posa, orientamento del corpo, orientamento della testa, come appaiono le mani.Processing of images retrieved from a simple robot RGB camera stream in order to estimate the engagement propensity of a person in human-robot interaction scenarios. To compute the final estimation, deep learning based technique are used to extract some auxiliary information as: estimation of the pose of a person, which type of pose, body orientation, head orientation, how hands appear

Self-supervised Learning of Monocular Depth from Video

Image-based depth estimation as a fundamental problem in computer vision allows for understanding the scene geometry using only cameras. This thesis addresses the specific problem of monocular depth estimation via self-supervised learning from RGB-only videos. Although existing work has shown partial excellent results in benchmark datasets, there remain several vital challenges that limit the use of these algorithms in general scenarios. To summarize, my identified challenges and contributions include: (i) Previous methods predict inconsistent depths over a video, which limits their uses in visual localization and mapping. To this end, I propose a geometry consistency loss that penalizes the multi-view depth misalignment in training, which enables scale-consistent depth estimation at inference time; (ii) Previous methods often diverge or show low-accuracy results when training on handheld camera captured videos. To address the challenge, I analyze the effect of camera motion on depth network gradients, and I propose an auto-rectify network to remove the relative rotation in training image pairs for robust learning; (iii) Previous methods fail to learn reasonable depths from highly dynamic scenes due to the non-rigidity. In this scenario, I propose a novel method, which constrains dynamic regions using an external well-trained depth estimation network and supervises static regions via multi-view losses. Comprehensive quantitative results and rich qualitative results are provided to demonstrate the advantages of the proposed methods over existing alternatives. The codes and pre-trained models have been released at https://github.com/JiawangBianThesis (Ph.D.) -- University of Adelaide, School of Computer Science, 202

Leveraging Structures of the Data in Deep Learning

The performance of deep learning frameworks could be significantly improved through considering the particular underlying structures for each dataset. In this thesis, I summarize our three work about boosting the performance of deep learning models through leveraging structures of the data. In the first work, we theoretically justify that, for convolutional neural networks (CNNs), neighborhoods of a pixel should be redefined as its most correlated spatial locations, in order to achieve a lower generalization error. Based on the correlation pattern, we propose a data-driven approach to design multiple layers of different customized filter shapes by repeatedly solving lasso problems. In the second work, we address the problem of scale-invariance in deep learning. We propose ScaleNet to predict object scales. Through recursively applying ScaleNet and rescaling, pretrained deep networks can identify objects with scales significantly different from the training set. In the last work, we perform an extensive study on PointConv based frameworks to tackle the problems of scale \& rotation invariances in point cloud convolution. PointConv is a novel convolution operation that can be directly applied on point clouds, and achieves parity with 2D CNNs in terms of formulation and performance. It takes coordinates of points as inputs to generate corresponding weights for convolution. We identify two effective strategies -- first, for point clouds converted from regular 2D raster images, we replace the multi-layer perceptrons (MLPs) based weight function with much simpler cubic polynomials, and achieve more robustness and better performance than traditional 2D CNNs on MNIST dataset. Next, for 3D point clouds, we introduce a novel viewpoint-invariant (VI) descriptor utilizing geometric properties between a center point and its local neighbors, as the additional input to the weight function. Integrated with the VI descriptor, we not only significantly improve the robustness of PointConv but also achieve comparable or better performance in comparison to the state-of-the-art point-based approaches on both SemanticKITTI and ScanNet

GIS and Remote Sensing for Renewable Energy Assessment and Maps

This book aims at providing the state-of-the-art on all of the aforementioned tools in different energy applications and at different scales, i.e., urban, regional, national, and even continental for renewable scenarios planning and policy making