This paper introduces the ray tracing method and algorithm for calculation of thermal radiant flux onto the focal plane array and generation of thermal images. Ray tracing is a powerful method for rendering of 3D objects and scenes on to the image plane of the camera. Besides, the method is used in the area of computer graphics but rarely is introduced in the domain of thermal imaging simulation. The author also has simulated thermal images of ground targets and compared them with real experiment test. Finally, the paper collates two simulation methods for generation of thermal images: the ray tracing method and the pseudo colour method

Vu, Duc Hieu

English

Journal of Science and Technique

    Tạp chí Khoa học và Kỹ thuật – ISSN 1859-0209    125 USING RAY TRACING ALGORITHM FOR RENDERING  OF THERMAL IMAGE OF 3D OBJECTS Duc Hieu Vu* Le Quy Don Technical University Abstract This paper introduces the ray tracing method and algorithm for calculation of thermal radiant flux onto the focal plane array and generation of thermal images. Ray tracing is a powerful method for rendering of 3D objects and scenes on to the image plane of the camera. Besides, the method is used in the area of computer graphics but rarely is introduced in the domain of thermal imaging simulation. The author also has simulated thermal images of ground targets and compared them with real experiment test . Finally, the paper collates two simulation methods for generation of thermal images: the ray tracing method and the pseudo colour method.  Keywords: Imaging system simulation; thermal imaging system; ray tracing method.  1. Introduction The thermal system simulation is the effective way to research, teach and develop an application in the area of thermal imaging. The benefit of simulation is that the system designer and the end user have the same understanding of the designed imaging system performances. Some applications such as NVTherm [1], TOD [2], and TRM3 [3] allow the user to evaluate the performance of thermal imager as MTF, MRTD, NETD and the probability of range discrimination. These parameters are powerful for comparing imaging systems, but they are inefficient for evaluation of the thermal camouflage. Besides, there is commercial software, as SIMTERM [4], having the ability to simulate thermal images of 2D objects. In addition, some authors [4]-[6] have used the pseudo colour method to simulate thermal imaging based on utilization of image processing. The pseudo colour method is not able to describe thermal image processing in the real thermal system correctly. Therefore, it is necessary to develop realistic rendering method as ray tracing to generate 3D real object models onto the 2D thermal image. In this paper, the basic background of thermal camera principle, ray tracing algorithm and using of that for simulation of thermal camera imaging are described.                                                   * Email: vuduchieu.v2@gmail.com                                        https://doi.org/10.56651/lqdtu.jst.v14.n01.503     Journal of Science and Technique – ISSN 1859-0209    1262. Ray tracing algorithm 2.1. Background Planck [7]-[9] showed that the spectral radiant emittance (or spectral radiant exitance) Me,λ of a black body depends on the wavelength  and the absolute surface temperature T of the object as:  5 -2 -12,1( ) 1 [exp( ) 1] (Wm μm )ecM cT       (1) It is assumed in the model that all irradiating objects are Lambertian surfaces and following Lambert’s cosine law. That means the radiant intensity from an ideal perfectly diffuse source is proportional to the cosine of the angle between the normal of the surface and the viewing angle. For the flat surface, the spectral radiance is calculated from spectral radiant Me,λ (1) by below equation:   ,,eeML (1) The output voltage of a detector is proportional to [7, 8]:           212    ,   , 4e t s e b DAS sd a odL T A L T A AV G RF        (2) where dV  is output voltage of a detector; G  is system gain;  dRis sensor sensitivity;    ,e tL T is target spectral radiance;    ,e bL T is background spectral radiance;  T is temperature of source (K); sA is target area; DASA is detector area; F is optical f-number; o() is spectral optical transmittance; a() is spectral atmospheric transmittance. 2.2. Characteristics of thermal imaging system The two main characteristics of thermal imaging system (TIS), geometrical and thermal resolution, are discussed in this section. Geometrical resolution of TIS is adequately characterized by its modulation transfer function (MTF). Blurring effects due to the filtering action of TIS components are incorporated into the system via the cascaded MTF/PTF approach. The system transfer function is the product of the component transfer functions [10, 11].              , , , , , , ,sys atm opt det elec disp eyeH H H H H H H               (3) where atmH  is the atmosphere MTF components; optH is the optical MTF components; elecH is the electronic MTF component; dispH is the display MTF component; eyeH is the human eyes MTF component; ,   are the spatial frequencies;     Tạp chí Khoa học và Kỹ thuật – ISSN 1859-0209    127 The temperature resolution is characterized by noise equivalent temperature difference (NETD). NETD provides the information about the noise that affects a thermal imager and which limits the smallest temperature difference detectable by TIS. With focal plane array thermal imagers, a parameter named fill factor is used. Thus, the NETD is calculated by the following function [7, 12]:    212 *,*d 0 *2NETD =  R factorea ppFF fM DA D dT D   (4) where F is the f-number; dA  is the area of each detector’s element, (cm2); 0 is transmission of the optical system; a is the transmission in the atmosphere; D*() is the spectral specific detectivity, (cm.Hz0.5/W); D*p is the peak spectral specific detectivity, (cm.Hz0.5/W); Me,λ is the spectral existence, defined by Plank function;  FR is the frame rate, (Hz);  factorf is the fill factor ratio and (1,2) is the spectral range of the system. The effect of both MTF and NETD is calculated in this thermal imaging program, presented in this paper. 2.3. Ray tracing algorithm In computer graphic, ray tracing is a technique for projecting a three-dimensional (3D) object on a two-dimensional (2D) display by tracing a path of light through pixels in an image plane. Ray tracing is capable of simulating a wide variety of optical effects, such as reflection, refraction, scattering, and dispersion phenomena (such as chromatic aberration). Radiant flux incident onto camera sensors of includes three components:    e o s bQ Q Q Q     (5) where eQ is the radiant flux leaving from object (emitted, reflected and transmitted); oQ  is the self-emitted radiant flux of the object; sQ  is the reflected radiant flux of surrounding component, reflected on the surface of object; bQ is the transmitted radiant flux of background, behind object;   is the emissivity of object;  is the reflectivity of object;   is the transmissivity of object,   1     . For an opaque surface, the transmitted component vanishes and only two components remain:  e o sQ Q Q   (6)     Journal of Science and Technique – ISSN 1859-0209    128As a principle, the thermal camera has the same structure components as the visual camera. While rendering 3D objects onto the 2D thermal image, the biggest algorithm problem is how to calculate irradiance of the object’s facet. We have assumed that the object is opaque and the reflected radiant flux can be ignored. In ray tracing method, the thermal images are computed pixel by pixel. Each ray from each pixel on the image plane through optical centre “hit” the surface of the target. Then, the location of hit point, facet identification number, and object identification number are obtained. If the ray does not hit the object, it is called the infinite ray. The pixel, point source of the infinite ray, represents thermal background radiance only.   Fig. 1. Principle of the ray tracing method  The algorithm is introduced in Fig. 2 to clarify the matter.                       Begin Input parameters Background Radiant flux  Raytracing Ouput signal of the pixel  Facet + background radiant flux  Display  Facet radiant flux  end ADAS>As Matrix indices  i=1:1:Totalpixel Parameters of facet (area As, ADAS ,Le , normal vector) Hitpoint(i)=[] ID facet of target   Fig. 2. Ray tracing and rendering algorithm Background Surrounding  Object    Transmitted radiation Reflected radiation Emitted radiation Focal plane arrays   Optical centre      Tạp chí Khoa học và Kỹ thuật – ISSN 1859-0209    129 2.4. Assumptions The following assumptions are used in the model: - All objects are opaque objects, and the reflective component are ignored - The effect of atmosphere is ignored - Only temporal noise of TIS is calculated These assumptions can be acceptable for simulation of a staring thermal camera under the good weather condition. Besides, the distance between target and camera is not too far.  3. Simulation and experimental results  In this section, thermal images are simulated by using thermal camera simulation, the input parameters of which are shown in Tab. 1. Tab. 1. Characteristics of thermal cameras Camera name  FLIR A40M FLIR 7650E Horizontal field of view, HFOV  24o 2.75o Vertical field of view, VFOV 18o 2.2o NETD 0.08oC at 300 K 0.02oC at 300 K Detector type  Uncooled micro bolometer InSb Pixel pitch 25 µm 15 µm Pixel array 320 x 240  640 x 512 Frame rate  60 Hz 100 Hz Cut on wave length  7.5 µm 3.5 µm Cut off wave length  13 µm 5 µm Average optical transmission 0.95 0.95 F number (F) 1 2.5 Focus length (f) 19 mm 200 mm  The two 3D models were generated for the purpose of the analysis: tank model and the building model. The models were created based on real structures in the Vyškov military training centre. 3.1. Tank model  The tank model is separated into three main parts: - Caterpillars (temperature: 30oC, emissivity: 0.9) (Fig. 3a);     Journal of Science and Technique – ISSN 1859-0209    130- Engine cover (temperature: 35oC, emissivity: 0.9) (Fig. 3b); - Residual portion (temperature: 25oC, emissivity: 0.9). Fig. 3. Two parts of tank model The tank input model for both cameras is depicted in Fig. 4. The camera coordinate is (100,200,0) [m,m,m]. The appropriate surface radiance of initial tank model is demonstrated in Fig. 5. a) Camera FLIR A40M b) Camera FLIR 7650E Fig. 4. Initial temperature of tank model  a) Camera FLIR A40M b) Camera FLIR 7650E  Fig. 5. Radiance of Tank model  a) Caterpillars  b) Engine cover     Tạp chí Khoa học và Kỹ thuật – ISSN 1859-0209    131  Fig. 6. Spectral radiance in band (7.5-13) m   Fig. 7. Spectral radiance in band (3.5-5) m Tab. 2. Band radiance of model components (W.sr-1.m-2) Part Caterpillars Engine cover Residual portion Camera FLIR A40M 48.7295 52.6901 44.9577 Camera FLIR 7650E  1.8090 2.1503 1.5135 These Figs. 6-7 and Tab. 2 clearly demonstrated that LWIR radiance of the grey body with outer surface’s temperature around 300 K is much bigger than its MWIR radiance. Additionally, in the dark night conditions, when reflective radiation from the     Journal of Science and Technique – ISSN 1859-0209    132environment such as clouds, moon or stars, etc. can be ignored, thermal band radiance obtained on the sensor is only from the self-emitted target. In this case, without regard to the atmospheric attenuation whereas the target’s temperature is smaller than 100oC, LWIR has an advantage over MWIR. However, MWIR takes advantage of day condition, when the reflective energy from surrounding (the sun, cloud) prevail or observes target such as aircraft, missile, vehicle. Moreover, depending on the weather condition, location and survey time, LWIR and MWIR have different atmospheric attenuation. Therefore, each band has the advantage of its application.  Tank thermal image simulation of camera FLIR A40M was illustrated in Fig. 8 below:   a) Tank at location [100 200 0] b) Tank at location [10 20 0] Fig. 8. Tank’s thermal image using thermal camera FLIR A40M Based on the information of Tab. 1, the vertical DAS of FLIR A40M is 1.3158 mrad. The height of the tank model is 2.4 m. In the Fig. 8.a, the distance between camera and target is 223.6 m. At this distance, the image of the tank must have 8 pixels vertically. To verify this issue, the Fig. 8.a was zoomed in, checked and depicted in Fig. 9 below:  Fig. 9. Zooming in of tank’s thermal image 8 pixels     Tạp chí Khoa học và Kỹ thuật – ISSN 1859-0209    133 Fig. 9 showed that the result was correct. In addition, because of short focal length and wide FOV, commercial thermal camera FLIR A40M is not suitable for the military application. 3.2. Building model  The real building, including observation tower, operation building, bunker and yard, is located in the Vyškov military training centre. This building system was captured by camera FLIR A40M on August 8th, 2016. The distance between the camera and the bunker is 75 m. All facets have emissivity equal 0.90. The environment has 15oC temperature, and its emissivity is 0.1.  The thermal images were shown in the figure below:  Fig. 10. Real thermal image of buildings captured by thermal camera FLIR A40M The geometrical model was created and imported into thermal simulation program (Fig. 11):   Fig. 11. Geometrical model of buildings Input parameters were set following table: Tab. 3. Input paremeter  Camera name  FLIR A40M Camera location (m,m,m) [-5, -75, 2.5] Target location (m,m,m) [0 0 0]     Journal of Science and Technique – ISSN 1859-0209    134Rendering by using thermal simulation program, the virtual thermal images of the scene was created and compared with the real thermal image:   Fig. 12. Virtual thermal image and real thermal image  On the left above side in Fig. 12, the virtual thermal image of radiant flux of building system firstly was generated. Then, the virtual ideal thermal image was calculated by interpolation table of temperature and incident radiant flux. The final virtual thermal image result was obtained by adding noise (NETD). On the right below side, the real thermal image of building system was captured by camera FLIR A40M. The simulation result appropriates with the real result.  The rendering ray tracing method in this paper has a difference with some authors [5, 6]. They used “pseudo colour” solution for generating thermal images. Firstly, they calculated the radiant flux of all surfaces. Then, they display results as “pseudo colour” of all faces. This image can be transformed into a sensor intensity matrix with the hardcopy function [5] (capture screen or print function). Then, they calculated the attenuation of the signal by reduction colour sensitivity of facets. These assumptions were acceptable if the emissivity of surfaces, optical transmission, and atmospheric transmission are constants. In this paper, the power on the every pixel was directly calculated. Due to this algorithm, the program is able to calculate realistic thermal images without above assumptions.     Tạp chí Khoa học và Kỹ thuật – ISSN 1859-0209    135 4. Conclusion Ray tracing is a popular method for rendering visual images of 3D objects. It is a powerful algorithm in the area of computer graphic. However, using ray tracing in the modelling thermal imaging system is still rare. This paper presents a simulating design for calculating radiant flux on every pixel of thermal camera’s FPAs. This result is used for generating thermal images of thermal imaging systems. This paper aims to use ray tracing method for simulating thermal image, beside other methods [5, 6]. Authors developed algorithm firstly for calculating band radiance from object’s emittance themselves. It means that algorithm is suitable for modelling thermal camera in MWIR or LWIR band. In the paper, the geometry model of battle tank T54 and buildings model were generated firstly. Then, their thermal images were rendered by using ray tracing algorithm. The simulated and real results are relatively appropriate. To simulate transmitted and reflected thermal radiation, it is necessary to develop and upgrade this algorithm.  References 1. NVESD U.S Army Night Vision and Electronic Sensors Directorate (2001). Night Vision Thermal Imaging Systems Performance Model. 2. P. Bijl and J. M. Valeton (1999). Guidelines for accurate TOD measurement, 14-25. 3. W. Wittenstein, W. Fick, and U. A. Raidt (1996). Range performance of two staring imagers: Presentation of the field trial and data analysis, 132-143. 4. K. Chrzanowski and M. Krupski (2004). Computer simulator for training operators of thermal cameras. in Enhanced and Synthetic Vision 2004, SPIE-5424, 187. 5. J. P. Tremblay and C. R. Viau (2009). A MATLAB/Simulink methodology for simulating dynamic imaging IR missile scenarios for use in countermeasure development and evaluation. in Technologies for Optical Countermeasures VI, 7483, 74830K. 6. S. Baqar (2007). Low-Cost PC-Based High-Fidelity Infrared Signature Modelling and Simulation. Cranfield University. 7. J. M. Lloyd (1975). Thermal Imaging Systems. Boston, MA: Springer US. 8. G. C. Holst (2000). Electro-Optical Imaging System Performance, 2nd ed. JCD publishing. 9. M. Mollmann, K.; Vollmer (2014). Infrared Thermal Imaging: Fundamentals, Research and Applications, 1. 10. R. H. VOLLMERHAUSEN and R. G. Driggers (2000). Analysis of Sampled Imaging Systems. SPIE Optical Engineering Press.     Journal of Science and Technique – ISSN 1859-0209    13611. G. D. Boreman (2001). Modulation Transfer Function in Optical and Electro-optical Systems. SPIE Press, Bellingham, WA. 12. D. Knežević, A. Redjimi, K. Mišković, D. Vasiljević, Z. Nikolić, and J. Babić (Jun. 2016). Minimum resolvable temperature difference model, simulation, measurement and analysis. Opt. Quantum Electron., 48(6), 332. SỬ DỤNG PHƯƠNG PHÁP “RAY TRACING”  ĐỂ DIỄN ẢNH NHIỆT CỦA ĐỐI TƯỢNG 3D Tóm tắt: Bài báo mô tả phương pháp dò tia (ray tracing) và sử dụng thuật toán của phương pháp trong việc tính toán dòng bức xạ nhiệt tác động lên ma trận cảm biến của thiết bị ảnh nhiệt để mô phỏng ảnh nhiệt của mục tiêu. “Ray tracing” là phương pháp rất mạnh trong việc diễn hình của mục tiêu ba chiều và phông nền lên trên mặt phẳng ma trận cảm biến của camera. Mặc dù phương pháp này được sử dụng rộng rãi trong lĩnh vực đồ họa máy tính nhưng lại rất ít được sử dụng và giới thiệu trong lĩnh vực mô phỏng ảnh nhiệt. Tác giả đã tiến hành mô phỏng ảnh nhiệt của mục tiêu mặt đất và so sánh kết quả đó với kết quả thực nghiệm. Cuối cùng, bài báo so sánh hai phương pháp mô phỏng ảnh nhiệt: phương pháp dò tia (ray tracing) và phương pháp ảnh giả màu.  Từ khóa: Mô phỏng hệ thống ảnh nhiệt; hệ thống ảnh nhiệt; phương pháp ray tracing. Received: 12/11/2018; Revised: 02/01/2019; Accepted for publication: 27/02/2019  

USING RAY TRACING ALGORITHM FOR RENDERING OF THERMAL IMAGE OF 3D OBJECTS

http://jst.lqdtu.edu.vn/index.php/jst/article/download/503/354

USING RAY TRACING ALGORITHM FOR RENDERING OF THERMAL IMAGE OF 3D OBJECTS

Abstract

Similar works

Full text

Available Versions

Journal of Science and Technique