CROSS-LAYER CUSTOMIZATION PLATFORM FOR LOW-POWER AND REAL-TIME EMBEDDED APPLICATIONS

Modern embedded applications have become increasingly complex and diverse in their functionalities and requirements. Data processing, communication and multimedia signal processing, real-time control and various other functionalities can often need to be implemented on the same System-on-Chip(SOC) platform. The significant power constraints and real-time guarantee requirements of these applications have become significant obstacles for the traditional embedded system design methodologies. The general-purpose computing microarchitectures of these platforms are designed to achieve good performance on average, which is far from optimal for any particular application. The system must always assume worst-case scenarios, which results in significant power inefficiencies and resource under-utilization. This dissertation introduces a cross-layer application-customizable embedded platform, which dynamically exploits application information and fine-tunes system components at system software and hardware layers. This is achieved with the close cooperation and seamless integration of the compiler, the operating system, and the hardware architecture. The compiler is responsible for extracting application regularities through static and profile-based analysis. The relevant application knowledge is propagated and utilized at run-time across the system layers through the judiciously introduced reconfigurability at both OS and hardware layers. The introduced framework comprehensively covers the fundamental subsystems of memory management and multi-tasking execution control

TractorEYE: Vision-based Real-time Detection for Autonomous Vehicles in Agriculture

Agricultural vehicles such as tractors and harvesters have for decades been able to navigate automatically and more efficiently using commercially available products such as auto-steering and tractor-guidance systems. However, a human operator is still required inside the vehicle to ensure the safety of vehicle and especially surroundings such as humans and animals. To get fully autonomous vehicles certified for farming, computer vision algorithms and sensor technologies must detect obstacles with equivalent or better than human-level performance. Furthermore, detections must run in real-time to allow vehicles to actuate and avoid collision.This thesis proposes a detection system (TractorEYE), a dataset (FieldSAFE), and procedures to fuse information from multiple sensor technologies to improve detection of obstacles and to generate a map. TractorEYE is a multi-sensor detection system for autonomous vehicles in agriculture. The multi-sensor system consists of three hardware synchronized and registered sensors (stereo camera, thermal camera and multi-beam lidar) mounted on/in a ruggedized and water-resistant casing. Algorithms have been developed to run a total of six detection algorithms (four for rgb camera, one for thermal camera and one for a Multi-beam lidar) and fuse detection information in a common format using either 3D positions or Inverse Sensor Models. A GPU powered computational platform is able to run detection algorithms online. For the rgb camera, a deep learning algorithm is proposed DeepAnomaly to perform real-time anomaly detection of distant, heavy occluded and unknown obstacles in agriculture. DeepAnomaly is -- compared to a state-of-the-art object detector Faster R-CNN -- for an agricultural use-case able to detect humans better and at longer ranges (45-90m) using a smaller memory footprint and 7.3-times faster processing. Low memory footprint and fast processing makes DeepAnomaly suitable for real-time applications running on an embedded GPU. FieldSAFE is a multi-modal dataset for detection of static and moving obstacles in agriculture. The dataset includes synchronized recordings from a rgb camera, stereo camera, thermal camera, 360-degree camera, lidar and radar. Precise localization and pose is provided using IMU and GPS. Ground truth of static and moving obstacles (humans, mannequin dolls, barrels, buildings, vehicles, and vegetation) are available as an annotated orthophoto and GPS coordinates for moving obstacles. Detection information from multiple detection algorithms and sensors are fused into a map using Inverse Sensor Models and occupancy grid maps. This thesis presented many scientific contribution and state-of-the-art within perception for autonomous tractors; this includes a dataset, sensor platform, detection algorithms and procedures to perform multi-sensor fusion. Furthermore, important engineering contributions to autonomous farming vehicles are presented such as easily applicable, open-source software packages and algorithms that have been demonstrated in an end-to-end real-time detection system. The contributions of this thesis have demonstrated, addressed and solved critical issues to utilize camera-based perception systems that are essential to make autonomous vehicles in agriculture a reality

Multimedia Forensics

This book is open access. Media forensics has never been more relevant to societal life. Not only media content represents an ever-increasing share of the data traveling on the net and the preferred communications means for most users, it has also become integral part of most innovative applications in the digital information ecosystem that serves various sectors of society, from the entertainment, to journalism, to politics. Undoubtedly, the advances in deep learning and computational imaging contributed significantly to this outcome. The underlying technologies that drive this trend, however, also pose a profound challenge in establishing trust in what we see, hear, and read, and make media content the preferred target of malicious attacks. In this new threat landscape powered by innovative imaging technologies and sophisticated tools, based on autoencoders and generative adversarial networks, this book fills an important gap. It presents a comprehensive review of state-of-the-art forensics capabilities that relate to media attribution, integrity and authenticity verification, and counter forensics. Its content is developed to provide practitioners, researchers, photo and video enthusiasts, and students a holistic view of the field

Multimedia Forensics

This book is open access. Media forensics has never been more relevant to societal life. Not only media content represents an ever-increasing share of the data traveling on the net and the preferred communications means for most users, it has also become integral part of most innovative applications in the digital information ecosystem that serves various sectors of society, from the entertainment, to journalism, to politics. Undoubtedly, the advances in deep learning and computational imaging contributed significantly to this outcome. The underlying technologies that drive this trend, however, also pose a profound challenge in establishing trust in what we see, hear, and read, and make media content the preferred target of malicious attacks. In this new threat landscape powered by innovative imaging technologies and sophisticated tools, based on autoencoders and generative adversarial networks, this book fills an important gap. It presents a comprehensive review of state-of-the-art forensics capabilities that relate to media attribution, integrity and authenticity verification, and counter forensics. Its content is developed to provide practitioners, researchers, photo and video enthusiasts, and students a holistic view of the field

Fundamentals

Volume 1 establishes the foundations of this new field. It goes through all the steps from data collection, their summary and clustering, to different aspects of resource-aware learning, i.e., hardware, memory, energy, and communication awareness. Machine learning methods are inspected with respect to resource requirements and how to enhance scalability on diverse computing architectures ranging from embedded systems to large computing clusters

Image and Video Forensics

Nowadays, images and videos have become the main modalities of information being exchanged in everyday life, and their pervasiveness has led the image forensics community to question their reliability, integrity, confidentiality, and security. Multimedia contents are generated in many different ways through the use of consumer electronics and high-quality digital imaging devices, such as smartphones, digital cameras, tablets, and wearable and IoT devices. The ever-increasing convenience of image acquisition has facilitated instant distribution and sharing of digital images on digital social platforms, determining a great amount of exchange data. Moreover, the pervasiveness of powerful image editing tools has allowed the manipulation of digital images for malicious or criminal ends, up to the creation of synthesized images and videos with the use of deep learning techniques. In response to these threats, the multimedia forensics community has produced major research efforts regarding the identification of the source and the detection of manipulation. In all cases (e.g., forensic investigations, fake news debunking, information warfare, and cyberattacks) where images and videos serve as critical evidence, forensic technologies that help to determine the origin, authenticity, and integrity of multimedia content can become essential tools. This book aims to collect a diverse and complementary set of articles that demonstrate new developments and applications in image and video forensics to tackle new and serious challenges to ensure media authenticity

Discrete Wavelet Transforms

The discrete wavelet transform (DWT) algorithms have a firm position in processing of signals in several areas of research and industry. As DWT provides both octave-scale frequency and spatial timing of the analyzed signal, it is constantly used to solve and treat more and more advanced problems. The present book: Discrete Wavelet Transforms: Algorithms and Applications reviews the recent progress in discrete wavelet transform algorithms and applications. The book covers a wide range of methods (e.g. lifting, shift invariance, multi-scale analysis) for constructing DWTs. The book chapters are organized into four major parts. Part I describes the progress in hardware implementations of the DWT algorithms. Applications include multitone modulation for ADSL and equalization techniques, a scalable architecture for FPGA-implementation, lifting based algorithm for VLSI implementation, comparison between DWT and FFT based OFDM and modified SPIHT codec. Part II addresses image processing algorithms such as multiresolution approach for edge detection, low bit rate image compression, low complexity implementation of CQF wavelets and compression of multi-component images. Part III focuses watermaking DWT algorithms. Finally, Part IV describes shift invariant DWTs, DC lossless property, DWT based analysis and estimation of colored noise and an application of the wavelet Galerkin method. The chapters of the present book consist of both tutorial and highly advanced material. Therefore, the book is intended to be a reference text for graduate students and researchers to obtain state-of-the-art knowledge on specific applications

Fundamentals

Volume 1 establishes the foundations of this new field. It goes through all the steps from data collection, their summary and clustering, to different aspects of resource-aware learning, i.e., hardware, memory, energy, and communication awareness. Machine learning methods are inspected with respect to resource requirements and how to enhance scalability on diverse computing architectures ranging from embedded systems to large computing clusters

Investigation of the Physical Properties of Dirac Materials

This thesis focuses on the investigation of two types of Dirac materials: topological insulators (TI) and graphene. Both materials have received much attention and stimulated intense research activities over the last decade. Although massless Dirac electron are wonderful, there will be more industrial applications if we can open the gap and make Dirac electrons massive. For topological insulators, we focus on studies of the TI/Magnetic TI (MTI) bilayer structures to induce a gap on the surface state. For graphene, the author focuses on the Moiré pattern and interlayer interaction. For bilayer TI/MTI samples, they were investigated with scanning tunneling microscopy and spectroscopy (STM/STS), and with electrical transport measurements by means of a Physical Property Measurement System (PPMS). Details of the experimental setups for this research and their upgrades were described. For the current STM system, both the tube scanner and sample stage in the STM head had been redesigned and rebuilt, which led to better XYZ fine approach control, improved wire protection, and enhanced noise shielding. A new back gate capability was added to the sample stage. A customized commercial STM system has been commissioned, which is expected to provide a better sample holder with improved vacuum seals and easier temperature control, as well as more convenient approaches to loading samples and switching STM or AFM (atomic force microscope) tips. For PPMS, an optical probe had been designed and constructed, which enabled light-induced effects on the electrical transport properties of TIs. A new custom-made glove box has been installed, which provides a computer-controlled and self-circling gas environment to minimize the concentration of air while reduces the waste of argon. The glove box is also easy to use. This upgrade helps expand our abilities to conduct research more efficiently. STM/STS studies of both the binary and ternary types of magnetic topological insulators (MTIs) are presented. For both binary and ternary bilayer TI/MTI systems, the majority of the density of states (DOS) spectra evolved with the temperature. At room temperature, all samples showed massless Dirac spectra. However, for temperatures below 200 K, all bilayer samples with the top pure TI layer thinner than 5QL revealed opening of a surface gap. Generally, binary TI/MTI samples exhibited smaller gapped domains, which was consistent with the finding of nearly negligible hysteretic behavior for Hall resistance vs, magnetic field sweeps at low temperatures. In contrast, ternary TI/MTI samples exhibited larger gapped domains, which implied longer range ferromagnetic order and was indeed corroborated by the apparent hysteretic behavior in the electrical transport measurements at low temperatures. Additionally, the application of c-axis magnetic fields led to slighter larger surface gaps and more uniform gap distributions, which further confirmed the physical origin of the surface gap as magnetic in nature. Besides the U or V-shaped DOS spectra, double-peak or single peak impurity resonances were also observed. These spatially localized minority spectra were found to mostly appear along the boundaries of gapped and gapless domains. Moreover, the number of impurities was founded to reach a maximum around 240 K, which corresponded to the onset temperature of localized surface gaps. Detailed studies of the electrical transport properties of both the binary and ternary MTIs by the PPMS provided a comparison between the macroscopic information thus obtained with the microscopic information derived from STS studies. Binary TI/MTI showed an anonymous Hall effect (AHE) at 25 K while ternary TI/MTI showed AHE around 20 K. Binary TI/MTI systems exhibited weak localization (WL) behavior in the longitudinal resistance vs. magnetic field data at 2 K. The binary TI/MTI samples with a thinner top pure TI layer revealed sharper and stronger WL behavior. In contrast, for the 3QL-TI/6QL-MTI ternary sample, weak antilocalization (WAL) behavior was present for all temperatures, while WL also showed up below 13 K. The Hall resistance vs. magnetic field data for all samples of ternary TI/MTI bilayers and ternary MTI monolayer samples revealed strong hysteresis at low temperatures, in contrast to the negligible hysteretic behavior in all binary TI/MTI samples. Finally, circularly polarized light was found to enhance the AHE of the bilayer ternary TI/MTI sample while weakening that of the monolayer ternary MTI. These experimental phenomena may be mostly attributed to the different band structures and Fermi levels among the binary and ternary TI/MTI samples. In particular, we note that the observation of quantum anomalous Hall effect (QAHE) only in ternary MTI monolayers at extremely low temperatures (at T ≤ 30 mK &lt; &lt; Tcbulk ~ 30 K) may be attributed to the finite contributions of bulk carriers to excess conduction unless T → 0. Simulations have been carried out to account for the Moiré patterns of graphene on Cu (111), graphene on Cu (100), twisted bilayer graphene, and Cr-doped topological insulators. The physical origin for empirically observed structural superlubricity between graphene layers has also been modeled by simulations based on the density functional theory (DFT). Finally, the key findings of this thesis work and the suggested future research directions are summarized.</p