Towards Airborne Thermography via Low-Cost Thermopile Infrared Sensors

International audienceThis paper presents a novel tool capable of collecting thermal signatures inside a building by using low-cost IR temperature sensors mounted on-board an aerial platform. The proposed system aims to facilitate the detection of heat loss inside buildings, which is a key aspect for improving energy efficiency in large commercial or industrial buildings. Current detection systems usually require manual labor as well as the use of expensive instrumentation. The proposed system on the other hand, relies on the use of a small unmanned aerial vehicle carrying low-cost thermopile IR sensors. Moreover, the system delivers a fast temperature sensing scheme and it provides coverage to inaccessible areas, thus overcoming the limitations of current mobile platforms which use ground robots. Different experiments were carried out in order to assess the behavior of the sensors as well as to validate the full system. Moreover, the hypothesis that thermopile IR sensors can be used to track temperature signature on-the-fly is validated experimentally with the use of the proposed system over different targets

Lighting in the third dimension : laser scanning as an architectural survey and representation method

This paper proposes tridimensional (3D) laser scanning to architects and lighting designers as a lighting enquiry and visualization method for existing built environments. The method constitutes a complement to existing lighting methods by responding to limitations of photometric measurements, computer simulation and HDR imagery in surveying and visualizing light in actual buildings. The research explores advantages and limitations of 3D laser scanning in a case study addressing a vast, geometrically complex and fragmented naturally and artificially lit space. Lighting patterns and geometry of the case study are captured with a 3D laser scanner through a series of four scans. A single 3D model of the entire space is produced from the aligned and fused scans. Lighting distribution patterns are showcased in relation to the materiality, geometry and position of windows, walls, lighting fixtures and day lighting sources. Results and presented through images similar to architectural presentation drawings. More specifically, the lighting distribution patterns are illustrated in a floor plan, a reflected ceiling plan, an axonometry and a cross-section. The point cloud model of the case study is also generated into a video format representing the entire building as well as different viewpoints. The study shows that the proposed method provides powerful visualization results due to the unlimited number of images that can be generated from a point cloud and facilitates understanding of existing lighting conditions in spaces

Herramienta software para la calibración extrínseca de cámaras infrarrojas y RGBD aplicada a inspección termográfica

Context:  Thermographic inspections are currently used to assess energy efficiency in electrical equipment and civil structures or to detect failures in cooling systems and electrical or electronic devices. However, thermal images lack texture details, which does not allow for a precise identification of the geometry of the scene or the objects in it. Method: In this work, the development of the software tool called DepTherm is described. This tool allows performing intrinsic and extrinsic calibration between infrared and RGBD cameras in order to fuse thermal, RGB, and RGBD images, as well as to record thermal and depth data. Additional features include user management, a visualization GUI for all three types of images, database storage, and report generation. Results: In addition to the integration tests performed to validate the functionality of DepTherm, two quantitative tests were conducted in order to evaluate its accuracy. A maximum re-projection error of 1,47±0,64 pixels was found, and the maximum mean error in registering an 11 cm side cube was 4,15 mm. Conclusions: The features of the DepTherm software tool are focused on facilitating thermographic inspections by capturing 3D scene models with thermal data.Contexto: Las inspecciones termográficas se utilizan en la actualidad para evaluar la eficiencia energética de equipos eléctricos y estructuras civiles o para detectar fallas en sistemas de enfriamiento y dispositivos eléctricos o electrónicos. Sin embargo, las imágenes térmicas carecen de detalles de textura, lo cual no permite identificar con precisión la geometría de la escena ni los objetos en ella. Método: En este trabajo se describe el desarrollo de la herramienta de software DepTherm, la cual permite realizar calibraciones intrínsecas y extrínsecas entre cámaras infrarrojas y RGBD para fusionar imágenes térmicas, RGB y RGBD, así como para registrar datos térmicos y de profundidad. Funcionalidades adicionales incluyen el manejo de usuarios, una GUI para visualización de los tres tipos de imágenes, el almacenamiento en una base de datos y la generación de reportes. Resultados: Además de las pruebas de integración para validar la funcionalidad de DepTherm, se realizaron dos pruebas cuantitativas para evaluar su precisión. Se encontró un error máximo de reproyección de 1,47±0,64 pixeles, mientras que el registro de un cubo con 11 cm de lado tuvo un error promedio máximo de 4,147 mm. Conclusiones: Las funcionalidades de la herramienta software DepTherm están enfocadas en facilitar las inspecciones termográficas capturando modelos 3D de las escenas con información térmica

Point clouds and thermal data fusion for automated gbXML-based building geometry model generation

Existing residential and small commercial buildings now represent the greatest opportunity to improve building energy efficiency. Building energy simulation analysis is becoming increasingly important because the analysis results can assist the decision makers to make decisions on improving building energy efficiency and reducing environmental impacts. However, manually measuring as-is conditions of building envelops including geometry and thermal value is still a labor-intensive, costly, and slow process. Thus, the primary objective of this research was to automatically collect and extract the as-is geometry and thermal data of the building envelope components and create a gbXML-based building geometry model. In the proposed methodology, a rapid and low-cost data collection hardware system was designed by integrating 3D laser scanners and an infrared (IR) camera. Secondly, several algorithms were created to automatically recognize various components of building envelope as objects from collected raw data. The extracted 3D semantic geometric model was then automatically saved as an industry standard file format for data interoperability. The feasibility of the proposed method was validated through three case studies. The contributions of this research include 1) a customized low-cost hybrid data collection system development to fuse various data into a thermal point cloud; 2) an automatic method of extracting building envelope components and its geometry data to generate gbXML-based building geometry model. The broader impacts of this research are that it could offer a new way to collect as is building data without impeding occupants’ daily life, and provide an easier way for laypeople to understand the energy performance of their buildings via 3D thermal point cloud visualization.Ph.D

Laboratorij za autonomne sustave i mobilnu robotiku

Laboratorij za autonomne sustave i mobilnu robotiku (LAMOR) istraživački je laboratorij koji djeluje u okviru Zavoda za automatiku i računalno inženjerstvo Fakulteta elektrotehnike i računarstva Sveučilišta u Zagrebu. LAMOR je jedan od vodećih laboratorija u Republici Hrvatskoj u području robotike, a svoje je istraživačko djelovanje usmjerio na temeljna istraživanja algoritama upravljanja, estimacije i umjetne inteligencije s primjenom u razvoju sustava autonomije mobilnih robota i vozila u nepoznatim i dinamičkim okruženjima te sustava djelotvorne i sigurne interakcije autonomnih mobilnih robota i ljudi. U radu je opisana uspostava laboratorija i prikazan je njegov 20-godišnji razvoj, a potom su predstavljene osnovne informacije o njegovoj istraživačkoj djelatnosti, najznačajnijim znanstvenim postignućima, najvažnijim istraživačkim projektima, međunarodnoj suradnji te doprinosu razvoju znanosti i gospodarstva u Republici Hrvatskoj

Laboratorij za autonomne sustave i mobilnu robotiku

Laboratorij za autonomne sustave i mobilnu robotiku (LAMOR) istraživački je laboratorij koji djeluje u okviru Zavoda za automatiku i računalno inženjerstvo Fakulteta elektrotehnike i računarstva Sveučilišta u Zagrebu. LAMOR je jedan od vodećih laboratorija u Republici Hrvatskoj u području robotike, a svoje je istraživačko djelovanje usmjerio na temeljna istraživanja algoritama upravljanja, estimacije i umjetne inteligencije s primjenom u razvoju sustava autonomije mobilnih robota i vozila u nepoznatim i dinamičkim okruženjima te sustava djelotvorne i sigurne interakcije autonomnih mobilnih robota i ljudi. U radu je opisana uspostava laboratorija i prikazan je njegov 20-godišnji razvoj, a potom su predstavljene osnovne informacije o njegovoj istraživačkoj djelatnosti, najznačajnijim znanstvenim postignućima, najvažnijim istraživačkim projektima, međunarodnoj suradnji te doprinosu razvoju znanosti i gospodarstva u Republici Hrvatskoj

RGB-D And Thermal Sensor Fusion: A Systematic Literature Review

In the last decade, the computer vision field has seen significant progress in multimodal data fusion and learning, where multiple sensors, including depth, infrared, and visual, are used to capture the environment across diverse spectral ranges. Despite these advancements, there has been no systematic and comprehensive evaluation of fusing RGB-D and thermal modalities to date. While autonomous driving using LiDAR, radar, RGB, and other sensors has garnered substantial research interest, along with the fusion of RGB and depth modalities, the integration of thermal cameras and, specifically, the fusion of RGB-D and thermal data, has received comparatively less attention. This might be partly due to the limited number of publicly available datasets for such applications. This paper provides a comprehensive review of both, state-of-the-art and traditional methods used in fusing RGB-D and thermal camera data for various applications, such as site inspection, human tracking, fault detection, and others. The reviewed literature has been categorised into technical areas, such as 3D reconstruction, segmentation, object detection, available datasets, and other related topics. Following a brief introduction and an overview of the methodology, the study delves into calibration and registration techniques, then examines thermal visualisation and 3D reconstruction, before discussing the application of classic feature-based techniques as well as modern deep learning approaches. The paper concludes with a discourse on current limitations and potential future research directions. It is hoped that this survey will serve as a valuable reference for researchers looking to familiarise themselves with the latest advancements and contribute to the RGB-DT research field.Comment: 33 pages, 20 figure

3D Thermal Modeling of Built Environments Using Visual and Infrared Sensing

Infrared thermography (IR) is a modern, non-destructive evaluation technology for monitoring and assessing built environments. It mainly relies on measuring surface temperature to identify any potential defects or damages. Currently, IR has been introduced widely in applications such as facility condition assessment and energy performance analysis of existing buildings. However, most of the current practices in IR rely only on 2D thermal images which are time-consuming and labor-intensive. On the other hand, the rapid improvement of high-defined IR cameras has become a powerful tool in infrared sensing. Accordingly, this has facilitated its implementation in 3D thermal modeling techniques to replace the current 2D approach in thermal inspection and building energy efficiency. Yet, further studies need to be performed to overcome 3D thermal modeling limitations such as the high cost, slow process, and the need of highly trained professionals. The main objectives of this research are to (a) test the potentiality of using 2D visible and thermal images which were collected separately through digital and infrared cameras respectively, for the 3D thermal modeling of built environments, and (b) investigate the efficiency of the proposed methodology by comparing it to a developed experimental design in terms of evaluating density, time, and cost. In specific, the visible images were used in modeling 3D point clouds by applying the structure from motion (sfm) approach. In parallel, the overlapping thermal images were stitched to form a thermal panoramic image that covers a large surface area with an accurate temperature representation. The stitched thermal images were then mapped to the reconstructed 3D point cloud in order to generate both thermal and metric measurements of built environments. Correspondingly, the output was compared to another 3D thermal point clouds which were developed by a laser scanner and an infrared camera. The comparison was conducted by means of evaluating density, time, and cost. Finally, the comparison results of three different built environments in the city of Montreal, Canada; demonstrate that 3D thermal modeling using separate 2D thermal and visible images was able to generate a dense geometric and thermal information of built environments. Also, this approach is affordable in terms of cost and time

Detection of water leaks and heating in electrical elements in tunnels using 3D-point clouds with thermographic information

3D-modellering har eksistert lenge og i den siste tiden har det blitt eksperimentert med en blanding av laserskanning og termografi. Kombinasjonen av disse kan gi 3D-modeller med god nøyaktighet inklusive informasjon om varme objekter, lekkasje i form av vann eller isolasjon og eventuelle defekte elektriske komponenter. Denne masteroppgaven tar for seg hva slags informasjon som kan hentes ut av en slik punktsky. Datainnsamlingen er gjort med et mobilt kartleggingssystem med en termisk enhet, som består av 4 termiske kameraer med 360˚ dekning vertikalt på kjøreretningen. Oppgaven går ut på å detektere varmgang i elektroniske elementer og vannlekkasjer i tunnel. Det ble gjort et forsøk på å kalibrere kameraene da innsamlingen av data ikke ga en uniform punktsky og samme overflate/objekt som har blitt fanget opp av to ulike kameraer ikke hadde samme farge i punktskyen, altså ulik målt temperatur. Dermed ble det gjort et forsøk på å finne årsaken til forskjellen i målt temperatur i hvert kamera og å kalibrere kameraene for å kunne lage en informativ og god punktsky. Mulighetene for en automatisering av deteksjonsprosessen for å effektivisere arbeidet ble også drøftet, samt det ble testet ut en egen metode for dette. Denne masteroppgaven kom frem til at det er mulig å detektere både vannlekkasjer i tunnel der vannet har trengt inn i tunnelen og varmgang i elektriske elementer basert på en punktsky med termisk informasjon. Det må tas hensyn til hva som skal detekteres i form av hvilken egenskap på punktskyen det skal filtreres på og satt temperaturspenn på punktskyen. En uniform punktsky ble laget etter en offset-korreksjon, og at en kalibrering kanskje ikke er nødvendig da en feilkilde kan skyldes den forvirrende punktskyen. En automatisering av deteksjonsprosessen kan gjøres for å oppdage varmgang, men det må jobbes mer med andre alternativer for å detektere lekkasje enn det som er gjort i denne oppgaven grunnet begrensinger i brukt programvare.3D-modeling has been around for a long time and lately it has been experimented with a blend of laser scanning and thermography. The combination of these can provide 3D-models with good accuracy including information about hot objects, leakage in the form of water or insulation and any defective electrical components. This master thesis takes care of what information can be obtained from such a point cloud. The data collection is done with a mobile mapping system with a thermal device, with 4 thermal cameras that provide 360 ° vertical coverage to the driving direction. The task is to detect heating in electronic elements and water leaks in tunnels. Due to the fact that there are 4 cameras, an attempt has also been made to calibrate these as the collection of data did not give a uniform point cloud and the same surface / object that was captured by two different cameras did not have the same color in the cloud, i.e. different measured temperature. Thus, an attempt was made to find the cause of the difference in measured temperature and to calibrate the cameras to create an informative and good spot cloud. The possibilities for automation of the detection process to streamline the work were also discussed and an own developed method was tested for this. This master thesis revealed that it is possible to detect both water leaks in tunnels where the water has penetrated the tunnel and heating in electrical elements from a point cloud with thermal information. Consideration must be given to what is to be detected in terms of which property on the point cloud it is to be filtered and set temperature range on the point cloud. A uniform point cloud was made after an offset correction and that a calibration may not be necessary as a source of error may be due to the confusing point cloud. Automation of the detection process can be done to detect heating, but it is necessary to work with other alternatives to detect leakage than what has been done in this task due to limitations in used software.M-GEO