Ubiquitous volume rendering in the web platform

176 p.The main thesis hypothesis is that ubiquitous volume rendering can be achieved using WebGL. The thesis enumerates the challenges that should be met to achieve that goal. The results allow web content developers the integration of interactive volume rendering within standard HTML5 web pages. Content developers only need to declare the X3D nodes that provide the rendering characteristics they desire. In contrast to the systems that provide specific GPU programs, the presented architecture creates automatically the GPU code required by the WebGL graphics pipeline. This code is generated directly from the X3D nodes declared in the virtual scene. Therefore, content developers do not need to know about the GPU.The thesis extends previous research on web compatible volume data structures for WebGL, ray-casting hybrid surface and volumetric rendering, progressive volume rendering and some specific problems related to the visualization of medical datasets. Finally, the thesis contributes to the X3D standard with some proposals to extend and improve the volume rendering component. The proposals are in an advance stage towards their acceptance by the Web3D Consortium

Ubiquitous volume rendering in the web platform

176 p.The main thesis hypothesis is that ubiquitous volume rendering can be achieved using WebGL. The thesis enumerates the challenges that should be met to achieve that goal. The results allow web content developers the integration of interactive volume rendering within standard HTML5 web pages. Content developers only need to declare the X3D nodes that provide the rendering characteristics they desire. In contrast to the systems that provide specific GPU programs, the presented architecture creates automatically the GPU code required by the WebGL graphics pipeline. This code is generated directly from the X3D nodes declared in the virtual scene. Therefore, content developers do not need to know about the GPU.The thesis extends previous research on web compatible volume data structures for WebGL, ray-casting hybrid surface and volumetric rendering, progressive volume rendering and some specific problems related to the visualization of medical datasets. Finally, the thesis contributes to the X3D standard with some proposals to extend and improve the volume rendering component. The proposals are in an advance stage towards their acceptance by the Web3D Consortium

3D LiDAR Point Cloud Processing Algorithms

In the race for autonomous vehicles and advanced driver assistance systems (ADAS), the automotive industry has energetically pursued research in the area of sensor suites to achieve such technological feats. Commonly used autonomous and ADAS sensor suites include multiples of cameras, radio detection and ranging (RADAR), light detection and ranging (LiDAR), and ultrasonic sensors. Great interest has been generated in the use of LiDAR sensors and the value added in an automotive application. LiDAR sensors can be used to detect and track vehicles, pedestrians, cyclists, and surrounding objects. A LiDAR sensor operates by emitting light amplification by stimulated emission of radiation (LASER) beams and receiving the reflected LASER beam to acquire relevant distance information. LiDAR reflections are organized in a three-dimensional environment known as a point cloud. A major challenge in modern autonomous automotive research is to be able to process the dimensional environmental data in real time. The LiDAR sensor used in this research is the Velodyne HDL 32E, which provides nearly 700,000 data points per second. The large amount of data produced by a LiDAR sensor must be processed in a highly efficient way to be effective. This thesis provides an algorithm to process the LiDAR data from the sensors user datagram protocol (UDP) packet to output geometric shapes that can be further analyzed in a sensor suite or utilized for Bayesian tracking of objects. The algorithm can be divided into three stages: Stage One - UDP packet extraction; Stage Two - data clustering; and Stage Three - shape extraction. Stage One organizes the LiDAR data from a negative to a positive vertical angle during packet extraction so that subsequent steps can fully exploit the programming efficiencies. Stage Two utilizes an adaptive breakpoint detector (ABD) for clustering objects based on a Euclidean distance threshold in the point cloud. Stage Three classifies each cluster into a shape that is either a point, line, L-shape, or a polygon using principal component analysis and shape fitting algorithms that have been modified to take advantage of the pre-organized data from Stage One. The proposed algorithm was written in the C language and the runtime was tested on a two Windows equipped machines where the algorithm completed the processing, on average, sparing 30% of the time between UDP data packets sent from the HDL32E. In comparison to related research, this algorithm performed over seven hundred and thirty-seven times faster

AI-For-Mobility—A New Research Platform for AI-Based Control Methods

AI-For-Mobility (AFM) is the new research platform to investigate and implement novel control methods based on Artificial Intelligence (AI) within the Department of Vehicle System Dynamics at the German Aerospace Center (DLR). A production hybrid vehicle serves as a base platform. Since AI-based methods are data-driven, the vehicle is equipped with manifold sensors to provide the required data. They measure the vehicle’s state holistically and perceive the surrounding environment, while high performance on-board CPUs and GPUs handle the sensor data. A full by-wire control system enables the vehicle to be used for applications in the field of automated driving. Despite all modifications, it is approved for public road use and meets the driving dynamics properties of a standard road vehicle. This makes it an attractive research and test platform, both for automotive applications and technology demonstrations in other scientific fields (e.g., robotics, aviation, etc.). This paper presents the vehicle’s design and architecture in a detailed manner and shows a promising application potential of AFM in the context of AI-based control methods

Benchmarking of Embedded Object Detection in Optical and RADAR Scenes

A portable, real-time vital sign estimation protoype is developed using neural network- based localization, multi-object tracking, and embedded processing optimizations. The system estimates heart and respiration rates of multiple subjects using directional of arrival techniques on RADAR data. This system is useful in many civilian and military applications including search and rescue. The primary contribution from this work is the implementation and benchmarking of neural networks for real time detection and localization on various systems including the testing of eight neural networks on a discrete GPU and Jetson Xavier devices. Mean average precision (mAP) and inference speed benchmarks were performed. We have shown fast and accurate detection and tracking using synthetic and real RADAR data. Another major contribution is the quantification of the relationship between neural network mAP performance and data augmentations. As an example, we focused on image and video compression methods, such as JPEG, WebP, H264, and H265. The results show WebP at a quantization level of 50 and H265 at a constant rate factor of 30 provide the best balance between compression and acceptable mAP. Other minor contributions are achieved in enhancing the functionality of the real-time prototype system. This includes the implementation and benchmarking of neural network op- timizations, such as quantization and pruning. Furthermore, an appearance-based synthetic RADAR and real RADAR datasets are developed. The latter contains simultaneous optical and RADAR data capture and cross-modal labels. Finally, multi-object tracking methods are benchmarked and a support vector machine is utilized for cross-modal association. In summary, the implementation, benchmarking, and optimization of methods for detection and tracking helped create a real-time vital sign system on a low-profile embedded device. Additionally, this work established a relationship between compression methods and different neural networks for optimal file compression and network performance. Finally, methods for RADAR and optical data collection and cross-modal association are implemented

Raster Time Series: Learning and Processing

As the amount of remote sensing data is increasing at a high rate, due to great improvements in sensor technology, efficient processing capabilities are of utmost importance. Remote sensing data from satellites is crucial in many scientific domains, like biodiversity and climate research. Because weather and climate are of particular interest for almost all living organisms on earth, the efficient classification of clouds is one of the most important problems. Geostationary satellites such as Meteosat Second Generation (MSG) offer the only possibility to generate long-term cloud data sets with high spatial and temporal resolution. This work, therefore, addresses research problems on efficient and parallel processing of MSG data to enable new applications and insights. First, we address the lack of a suitable processing chain to generate a long-term Fog and Low Stratus (FLS) time series. We present an efficient MSG data processing chain that processes multiple tasks simultaneously, and raster data in parallel using the Open Computing Language (OpenCL). The processing chain delivers a uniform FLS classification that combines day and night approaches in a single method. As a result, it is possible to calculate a year of FLS rasters quite easy. The second topic presents the application of Convolutional Neural Networks (CNN) for cloud classification. Conventional approaches to cloud detection often only classify single pixels and ignore the fact that clouds are highly dynamic and spatially continuous entities. Therefore, we propose a new method based on deep learning. Using a CNN image segmentation architecture, the presented Cloud Segmentation CNN (CS-CNN) classifies all pixels of a scene simultaneously. We show that CS-CNN is capable of processing multispectral satellite data to identify continuous phenomena such as highly dynamic clouds. The proposed approach provides excellent results on MSG satellite data in terms of quality, robustness, and runtime, in comparison to Random Forest (RF), another widely used machine learning method. Finally, we present the processing of raster time series with a system for Visualization, Transformation, and Analysis (VAT) of spatio-temporal data. It enables data-driven research with explorative workflows and uses time as an integral dimension. The combination of various raster and vector data time series enables new applications and insights. We present an application that combines weather information and aircraft trajectories to identify patterns in bad weather situations

Intelligent Computational Transportation

Transportation is commonplace around our world. Numerous researchers dedicate great efforts to vast transportation research topics. The purpose of this dissertation is to investigate and address a couple of transportation problems with respect to geographic discretization, pavement surface automatic examination, and traffic ow simulation, using advanced computational technologies. Many applications require a discretized 2D geographic map such that local information can be accessed efficiently. For example, map matching, which aligns a sequence of observed positions to a real-world road network, needs to find all the nearby road segments to the individual positions. To this end, the map is discretized by cells and each cell retains a list of road segments coincident with this cell. An efficient method is proposed to form such lists for the cells without costly overlapping tests. Furthermore, the method can be easily extended to 3D scenarios for fast triangle mesh voxelization. Pavement surface distress conditions are critical inputs for quantifying roadway infrastructure serviceability. Existing computer-aided automatic examination techniques are mainly based on 2D image analysis or 3D georeferenced data set. The disadvantage of information losses or extremely high costs impedes their effectiveness iv and applicability. In this study, a cost-effective Kinect-based approach is proposed for 3D pavement surface reconstruction and cracking recognition. Various cracking measurements such as alligator cracking, traverse cracking, longitudinal cracking, etc., are identified and recognized for their severity examinations based on associated geometrical features. Smart transportation is one of the core components in modern urbanization processes. Under this context, the Connected Autonomous Vehicle (CAV) system presents a promising solution towards the enhanced traffic safety and mobility through state-of-the-art wireless communications and autonomous driving techniques. Due to the different nature between the CAVs and the conventional Human- Driven-Vehicles (HDVs), it is believed that CAV-enabled transportation systems will revolutionize the existing understanding of network-wide traffic operations and re-establish traffic ow theory. This study presents a new continuum dynamics model for the future CAV-enabled traffic system, realized by encapsulating mutually-coupled vehicle interactions using virtual internal and external forces. A Smoothed Particle Hydrodynamics (SPH)-based numerical simulation and an interactive traffic visualization framework are also developed

Doctor of Philosophy

dissertationStochastic methods, dense free-form mapping, atlas construction, and total variation are examples of advanced image processing techniques which are robust but computationally demanding. These algorithms often require a large amount of computational power as well as massive memory bandwidth. These requirements used to be ful lled only by supercomputers. The development of heterogeneous parallel subsystems and computation-specialized devices such as Graphic Processing Units (GPUs) has brought the requisite power to commodity hardware, opening up opportunities for scientists to experiment and evaluate the in uence of these techniques on their research and practical applications. However, harnessing the processing power from modern hardware is challenging. The di fferences between multicore parallel processing systems and conventional models are signi ficant, often requiring algorithms and data structures to be redesigned signi ficantly for efficiency. It also demands in-depth knowledge about modern hardware architectures to optimize these implementations, sometimes on a per-architecture basis. The goal of this dissertation is to introduce a solution for this problem based on a 3D image processing framework, using high performance APIs at the core level to utilize parallel processing power of the GPUs. The design of the framework facilitates an efficient application development process, which does not require scientists to have extensive knowledge about GPU systems, and encourages them to harness this power to solve their computationally challenging problems. To present the development of this framework, four main problems are described, and the solutions are discussed and evaluated: (1) essential components of a general 3D image processing library: data structures and algorithms, as well as how to implement these building blocks on the GPU architecture for optimal performance; (2) an implementation of unbiased atlas construction algorithms|an illustration of how to solve a highly complex and computationally expensive algorithm using this framework; (3) an extension of the framework to account for geometry descriptors to solve registration challenges with large scale shape changes and high intensity-contrast di fferences; and (4) an out-of-core streaming model, which enables developers to implement multi-image processing techniques on commodity hardware

Advanced Aviation Weather Radar Data Processing and Real-Time Implementations

The objectives of this dissertation work are developing an enhanced intelligent radar signal and data processing framework for aviation hazard detection, classification and monitoring, and real-time implementation on massive parallel platforms. Variety of radar sensor platforms are used to prove the concept including airborne precipitation radar and different ground weather radars. As a focused example of the proposed approach, this research applies evolutionary machine learning technology to turbulence level classification for civil aviation. An artificial neural network (ANN) machine learning approach based on radar observation is developed for classifying the cubed root of the Eddy Dissipation Rate (EDR), a widely-accepted measure of turbulence intensity. The approach is validated using typhoon weather data collected by Hong Kong Observatory’s (HKO) Terminal Doppler Weather Radar (TDWR) located near Hong Kong International Airport (HKIA) and comparing HKO-TDWR EDR$^{1/3}$ detections and predictions with in situ EDR$^{1/3}$ measured by commercial aircrafts. The testing results verified that machine learning approach performs reasonably well for both detecting and predicting tasks. As the preliminary step to explore the possibility of acceleration by integrating General Purpose Graphic Processing Unit (GPGPU), this research introduces a practical approach to implement real-time processing algorithms for general surveillance radar based on NVIDIA graphical processing units (GPUs). The pulse compression algorithms are implemented using compute unified device architecture (CUDA) libraries such as CUDA basic linear algebra subroutines and CUDA fast Fourier transform library, which are adopted from open source libraries and optimized for the NVIDIA GPUs. For more advanced, adaptive processing algorithms such as adaptive pulse compression, customized kernel optimization is investigated. A statistical optimization approach is developed for this purpose without needing much knowledge of the physical configurations of the kernels. It was found that the kernel optimization approach can significantly improve the performance. Benchmark performance is compared with the CPU performance in terms of processing accelerations. The proposed implementation framework can be used in various radar systems including ground-based phased array radar, airborne sense and avoid radar, and aerospace surveillance radar. After the investigation of the GPGPU on radar signal processing chain, the benchmark of applying machine learning approach on embedded GPU platform was performed. According to the performance, real-time requirement of the machine learning method of turbulence detection developed in this research could be met as well as Size, Weight and Power (SWaP) restrictions on embedded GPGPU platforms