Why my photos look sideways or upside down? Detecting Canonical Orientation of Images using Convolutional Neural Networks

Image orientation detection requires high-level scene understanding. Humans use object recognition and contextual scene information to correctly orient images. In literature, the problem of image orientation detection is mostly confronted by using low-level vision features, while some approaches incorporate few easily detectable semantic cues to gain minor improvements. The vast amount of semantic content in images makes orientation detection challenging, and therefore there is a large semantic gap between existing methods and human behavior. Also, existing methods in literature report highly discrepant detection rates, which is mainly due to large differences in datasets and limited variety of test images used for evaluation. In this work, for the first time, we leverage the power of deep learning and adapt pre-trained convolutional neural networks using largest training dataset to-date for the image orientation detection task. An extensive evaluation of our model on different public datasets shows that it remarkably generalizes to correctly orient a large set of unconstrained images; it also significantly outperforms the state-of-the-art and achieves accuracy very close to that of humans

Why my photos look sideways or upside down? Detecting Canonical Orientation of Images using Convolutional Neural Networks

Image orientation detection requires high-level scene understanding. Humans use object recognition and contextual scene information to correctly orient images. In literature, the problem of image orientation detection is mostly confronted by using low-level vision features, while some approaches incorporate few easily detectable semantic cues to gain minor improvements. The vast amount of semantic content in images makes orientation detection challenging, and therefore there is a large semantic gap between existing methods and human behavior. Also, existing methods in literature report highly discrepant detection rates, which is mainly due to large differences in datasets and limited variety of test images used for evaluation. In this work, for the first time, we leverage the power of deep learning and adapt pre-trained convolutional neural networks using largest training dataset to-date for the image orientation detection task. An extensive evaluation of our model on different public datasets shows that it remarkably generalizes to correctly orient a large set of unconstrained images; it also significantly outperforms the state-of-the-art and achieves accuracy very close to that of humans

Application of Deep Learning for Imaging-Based Stream Gaging

In the field of water resources management, one vital instrument utilized is the stream gage. Stream gages monitor and record flow and water height within some water body. The United States Geological Survey maintains a network of stream gages at many locations across the country. Many of these sites are also equipped with webcams monitoring the state of the water body at the moment of measurement. Previous studies have outlined methods to approximate stream gage data remotely with limitations such as the requirement of detailed depth information for each site. This study seeks to create a process for training a deep neural network that will utilize the webcam and stream gage data to generate a water height prediction based on the visual state of the water body. The goal of this study is to outline a training process and model that can be utilized on a variety of different sites while only requiring that location’s existing webcam and stream gage data. This paper outlines the experiments on stream gages located at the Clear Creek in Iowa, Auglaize River in Ohio, and Milwaukee River in Wisconsin. The process outlined utilizes transfer learning and well-known image classification models as a basis for a generalized river height regression model. Across the training, validation, and deployment experiments, the developed process shows great success in creating an accurate model for various sites of different conditions and river clarity. The results of this study show confidence in future studies utilizing remote stream gaging with image regression

Explainable machine learning for precise fatigue crack tip detection

Data-driven models based on deep learning have led to tremendous breakthroughs in classical computer vision tasks and have recently made their way into natural sciences. However, the absence of domain knowledge in their inherent design significantly hinders the understanding and acceptance of these models. Nevertheless, explainability is crucial to justify the use of deep learning tools in safety-relevant applications such as aircraft component design, service and inspection. In this work, we train convolutional neural networks for crack tip detection in fatigue crack growth experiments using full-field displacement data obtained by digital image correlation. For this, we introduce the novel architecture ParallelNets—a network which combines segmentation and regression of the crack tip coordinates—and compare it with a classical U-Net-based architecture. Aiming for explainability, we use the Grad-CAM interpretability method to visualize the neural attention of several models. Attention heatmaps show that ParallelNets is able to focus on physically relevant areas like the crack tip field, which explains its superior performance in terms of accuracy, robustness, and stability

Concepts and tools to improve the thermal energy performance of buildings and urban districts - diagnosis, assessment, improvement strategies and cost-benefit analyses

Retrofitting existing buildings to optimize their thermal energy performance is a key factor in achieving climate neutrality by 2045 in Germany. Analyzing buildings in their current condition is the first step toward preparing effective and efficient energy retrofit measures. A high-quality building analysis helps to evaluate whether a building or its components are suitable for retrofitting or replacement. Subsequently, appropriate combinations of retrofit measures that create financial and environmental synergies can be determined. This dissertation is a cumulative work based on nine papers on the thermal analysis of existing buildings. The focus of this work and related papers is on thermography with drones for building audits, intelligent processing of thermographic images to detect and assess thermal weaknesses, and building modeling approaches to evaluate thermal retrofit options. While individual buildings are usually the focus of retrofit planning, this dissertation also examines the role of buildings in the urban context, particularly on a district level. Multiple adjacent buildings offer numerous possibilities for further improving retrofits, such as the economies of scale for planning services and material procurement, neighborhood dynamics, and exchange of experiences between familiar building owners. This work reveals the opportunities and obstacles for panorama drone thermography for building audits. It shows that drones can contribute to a quick and structured data collection, particularly for large building stocks, and thus complement current approaches for district-scale analysis. However, the significant distance between the drone camera and building, which is necessary for automated flight routes, and varying recording angles limit the quantitative interpretability of thermographic images. Therefore, innovative approaches were developed to process image datasets generated using drones. A newly designed AI-based approach can automate the detection of thermal bridges on rooftops. Using generalizations about certain building classes as demonstrated by buildings from the 1950s and 1960s, a novel interpretation method for drone images is suggested. It enables decision-making regarding the need to retrofit thermal bridges of recorded buildings. A novel optimization model for German single-family houses was developed and applied in a case study to investigate the financial and ecological benefits of different thermal retrofit measures. The results showed that the retrofitting of building façades can significantly save energy. However, they also revealed that replacing the heating systems turns out to be more cost-effective for carbon dioxide savings. Small datasets, limited availability of technical equipment, and the need for simplified assumptions for building characteristics without any information were the main challenges of the approaches in this dissertation

EG-ICE 2021 Workshop on Intelligent Computing in Engineering

The 28th EG-ICE International Workshop 2021 brings together international experts working at the interface between advanced computing and modern engineering challenges. Many engineering tasks require open-world resolutions to support multi-actor collaboration, coping with approximate models, providing effective engineer-computer interaction, search in multi-dimensional solution spaces, accommodating uncertainty, including specialist domain knowledge, performing sensor-data interpretation and dealing with incomplete knowledge. While results from computer science provide much initial support for resolution, adaptation is unavoidable and most importantly, feedback from addressing engineering challenges drives fundamental computer-science research. Competence and knowledge transfer goes both ways

EG-ICE 2021 Workshop on Intelligent Computing in Engineering

The 28th EG-ICE International Workshop 2021 brings together international experts working at the interface between advanced computing and modern engineering challenges. Many engineering tasks require open-world resolutions to support multi-actor collaboration, coping with approximate models, providing effective engineer-computer interaction, search in multi-dimensional solution spaces, accommodating uncertainty, including specialist domain knowledge, performing sensor-data interpretation and dealing with incomplete knowledge. While results from computer science provide much initial support for resolution, adaptation is unavoidable and most importantly, feedback from addressing engineering challenges drives fundamental computer-science research. Competence and knowledge transfer goes both ways