Promoting Connectivity of Network-Like Structures by Enforcing Region Separation

We propose a novel, connectivity-oriented loss function for training deep convolutional networks to reconstruct network-like structures, like roads and irrigation canals, from aerial images. The main idea behind our loss is to express the connectivity of roads, or canals, in terms of disconnections that they create between background regions of the image. In simple terms, a gap in the predicted road causes two background regions, that lie on the opposite sides of a ground truth road, to touch in prediction. Our loss function is designed to prevent such unwanted connections between background regions, and therefore close the gaps in predicted roads. It also prevents predicting false positive roads and canals by penalizing unwarranted disconnections of background regions. In order to capture even short, dead-ending road segments, we evaluate the loss in small image crops. We show, in experiments on two standard road benchmarks and a new data set of irrigation canals, that convnets trained with our loss function recover road connectivity so well, that it suffices to skeletonize their output to produce state of the art maps. A distinct advantage of our approach is that the loss can be plugged in to any existing training setup without further modifications

Multi-stage Suture Detection for Robot Assisted Anastomosis based on Deep Learning

In robotic surgery, task automation and learning from demonstration combined with human supervision is an emerging trend for many new surgical robot platforms. One such task is automated anastomosis, which requires bimanual needle handling and suture detection. Due to the complexity of the surgical environment and varying patient anatomies, reliable suture detection is difficult, which is further complicated by occlusion and thread topologies. In this paper, we propose a multi-stage framework for suture thread detection based on deep learning. Fully convolutional neural networks are used to obtain the initial detection and the overlapping status of suture thread, which are later fused with the original image to learn a gradient road map of the thread. Based on the gradient road map, multiple segments of the thread are extracted and linked to form the whole thread using a curvilinear structure detector. Experiments on two different types of sutures demonstrate the accuracy of the proposed framework.Comment: Submitted to ICRA 201

Deep Learning for Safe Autonomous Driving: Current Challenges and Future Directions

[EN] Advances in information and signal processing technologies have a significant impact on autonomous driving (AD), improving driving safety while minimizing the efforts of human drivers with the help of advanced artificial intelligence (AI) techniques. Recently, deep learning (DL) approaches have solved several real-world problems of complex nature. However, their strengths in terms of control processes for AD have not been deeply investigated and highlighted yet. This survey highlights the power of DL architectures in terms of reliability and efficient real-time performance and overviews state-of-the-art strategies for safe AD, with their major achievements and limitations. Furthermore, it covers major embodiments of DL along the AD pipeline including measurement, analysis, and execution, with a focus on road, lane, vehicle, pedestrian, drowsiness detection, collision avoidance, and traffic sign detection through sensing and vision-based DL methods. In addition, we discuss on the performance of several reviewed methods by using different evaluation metrics, with critics on their pros and cons. Finally, this survey highlights the current issues of safe DL-based AD with a prospect of recommendations for future research, rounding up a reference material for newcomers and researchers willing to join this vibrant area of Intelligent Transportation Systems.This work was supported by Institute of Information & Communications Technology Planning & Evaluation (IITP) Grant funded by the Korea Government (MSIT) (2019-0-00136, Development of AI-Convergence Technologies for Smart City Industry Productivity Innovation); The work of Javier Del Ser was supported by the Basque Government through the EMAITEK and ELKARTEK Programs, as well as by the Department of Education of this institution (Consolidated Research Group MATHMODE, IT1294-19); VHCA received support from the Brazilian National Council for Research and Development (CNPq, Grant #304315/2017-6 and #430274/2018-1).Muhammad, K.; Ullah, A.; Lloret, J.; Del Ser, J.; De Albuquerque, VHC. (2021). Deep Learning for Safe Autonomous Driving: Current Challenges and Future Directions. IEEE Transactions on Intelligent Transportation Systems. 22(7):4316-4336. https://doi.org/10.1109/TITS.2020.30322274316433622

Fine-Grained Extraction of Road Networks via Joint Learning of Connectivity and Segmentation

Road network extraction from satellite images is widely applicated in intelligent traffic management and autonomous driving fields. The high-resolution remote sensing images contain complex road areas and distracted background, which make it a challenge for road extraction. In this study, we present a stacked multitask network for end-to-end segmenting roads while preserving connectivity correctness. In the network, a global-aware module is introduced to enhance pixel-level road feature representation and eliminate background distraction from overhead images; a road-direction-related connectivity task is added to ensure that the network preserves the graph-level relationships of the road segments. We also develop a stacked multihead structure to jointly learn and effectively utilize the mutual information between connectivity learning and segmentation learning. We evaluate the performance of the proposed network on three public remote sensing datasets. The experimental results demonstrate that the network outperforms the state-of-the-art methods in terms of road segmentation accuracy and connectivity maintenance

Utilizing Hybrid Trajectory Prediction Models to Recognize Highly Interactive Traffic Scenarios

Autonomous vehicles hold great promise in improving the future of transportation. The driving models used in these vehicles are based on neural networks, which can be difficult to validate. However, ensuring the safety of these models is crucial. Traditional field tests can be costly, time-consuming, and dangerous. To address these issues, scenario-based closed-loop simulations can simulate many hours of vehicle operation in a shorter amount of time and allow for specific investigation of important situations. Nonetheless, the detection of relevant traffic scenarios that also offer substantial testing benefits remains a significant challenge. To address this need, in this paper we build an imitation learning based trajectory prediction for traffic participants. We combine an image-based (CNN) approach to represent spatial environmental factors and a graph-based (GNN) approach to specifically represent relations between traffic participants. In our understanding, traffic scenes that are highly interactive due to the network's significant utilization of the social component are more pertinent for a validation process. Therefore, we propose to use the activity of such sub networks as a measure of interactivity of a traffic scene. We evaluate our model using a motion dataset and discuss the value of the relationship information with respect to different traffic situations

Detecção de estradas rurais em imagens Planet usando rede convolutional U-Net

Orientador: Prof. Dr. Jorge Antônio Silva CentenoCoorientador: Dr. Mario Ernesto Jijón PalmaDissertação (mestrado) - Universidade Federal do Paraná, Setor de Ciências da Terra, Programa de Pós-Graduação em Ciências Geodésicas. Defesa : Curitiba, 25/08/2023Inclui referênciasResumo: O Brasil, um dos países mais extensos do mundo, possui uma significativa parcela de vias de rodagem situada em ambiente rural, sem a devida manutenção, o que dificulta a extensão de serviços à população rural. Muitas dessas estradas desempenham um papel fundamental na gestão territorial, uma vez que são responsáveis pelo escoamento da produção agrícola do interior do país e pela conectividade das comunidades rurais. A manutenção destas estradas e sua exploração para a extensão de serviços básicos, como energia e água, é somente possível com uma adequada atualização cartográfica da rede viária. Mais recentemente, o uso de métodos de aprendizado profundo para a análise de imagens orbitais tem crescido significativamente. Dentro desta nova realidade, esta pesquisa propõe uma abordagem baseada em técnicas de sensoriamento remoto aliadas a ferramentas de inteligência artificial, com o intuito de contribuir para solucionar o problema do mapeamento de estradas em áreas rurais. Para isto, se propõe o uso das redes convolucionais. Utilizando a arquitetura U-Net, foi possível identificar um potencial promissor na detecção de estradas rurais em imagens da constelação Planet. A taxa de detecção alcançada foi notável, atingindo uma acurácia de 92%. Contudo, é importante ressaltar a necessidade de aprimoramentos, visto que outras métricas de avaliação, como a precisão (76,66%) e o f1-score (69,48%), indicam margem para otimização dos parâmetros utilizados. No estudo também é feita uma análise comparativa entre o uso dos interpretadores na nuvem, do Google Colab (em ambiente virtual) e Pyzo (em ambiente local, utilizando o computador desktop/workstation fornecido pela UFPR). Verificou-se que o Colab apresenta vantagens em termos de custo e acesso a recursos de processamento. Entretanto, é relevante destacar que o uso do Colab também traz consigo algumas limitações, as quais requerem uma abordagem cuidadosa ao ajustar a complexidade do modelo e o tamanho do conjunto de dados.Abstract: Brazil, one of the largest countries in the world, has a significant number of roads in rural areas that are not properly maintained, making it difficult to extend services to the rural population. Many of these roads play a fundamental role in land management, as they are responsible for transporting agricultural produce from the interior of the country and for connecting rural communities. Maintaining these roads and exploiting them to extend basic services, such as energy and water, is only possible with a proper cartographic update of the road network. More recently, the use of deep learning methods to analyse orbital images has grown significantly. Within this new reality, this research proposes an approach based on remote sensing techniques combined with artificial intelligence tools, with the aim of helping to solve the problem of mapping roads in rural areas. To this end, the use of convolutional networks is proposed. Using the U-Net architecture, it was possible to identify promising potential for detecting rural roads in images from the Planet constellation. The detection rate achieved was remarkable, reaching an accuracy of 92 per cent. However, it is important to highlight the need for improvement, since other evaluation metrics, such as accuracy (76.66%) and f1-score (69.48%), indicate room for optimization of the parameters used. The study also makes a comparative analysis between the use of interpreters in the cloud, Google Colab (in a virtual environment) and Pyzo (in a local environment, using the desktop/workstation computer provided by UFPR). Colab was found to have advantages in terms of cost and access to processing resources. However, it is important to emphasize that the use of Colab also brings with it some limitations, which require a careful approach when adjusting the complexity of the model and the size of the data set

Interactive Attention Learning on Detection of Lane and Lane Marking on the Road by Monocular Camera Image

Vision-based identification of lane area and lane marking on the road is an indispensable function for intelligent driving vehicles, especially for localization, mapping and planning tasks. However, due to the increasing complexity of traffic scenes, such as occlusion and discontinuity, detecting lanes and lane markings from an image captured by a monocular camera becomes persistently challenging. The lanes and lane markings have a strong position correlation and are constrained by a spatial geometry prior to the driving scene. Most existing studies only explore a single task, i.e., either lane marking or lane detection, and do not consider the inherent connection or exploit the modeling of this kind of relationship between both elements to improve the detection performance of both tasks. In this paper, we establish a novel multi-task encoder–decoder framework for the simultaneous detection of lanes and lane markings. This approach deploys a dual-branch architecture to extract image information from different scales. By revealing the spatial constraints between lanes and lane markings, we propose an interactive attention learning for their feature information, which involves a Deformable Feature Fusion module for feature encoding, a Cross-Context module as information decoder, a Cross-IoU loss and a Focal-style loss weighting for robust training. Without bells and whistles, our method achieves state-of-the-art results on tasks of lane marking detection (with 32.53% on IoU, 81.61% on accuracy) and lane segmentation (with 91.72% on mIoU) of the BDD100K dataset, which showcases an improvement of 6.33% on IoU, 11.11% on accuracy in lane marking detection and 0.22% on mIoU in lane detection compared to the previous methods