Design of an image acquisition and processing system using configurable devices

This thesis consists of the evaluation of the possibility to implement a Neural Network in an FPGA instead on the more used GPU. Theoretically, an FPGA is a better choice in terms of processing power, latency, or flexibility but its configuration is harder. In this report, the implementation process for an FPGA is followed, including the creation of an embedded Operating System, video capture and display pipelines, testing of the chosen model and the final implementation of the model in the board. As a result of the evaluation, the conclusion is that nowadays the way to implement a neural network in an FPGA is not mature enough to compete with GPU alternative. The tools needed to achieve this implementation are very limited and the process is confusing. In the other hand, the GPU implementations has a huge catalogue of HW options and one can choose the better solution for its model

Runtime methods for energy-efficient, image processing using significance driven learning.

Ph. D. Thesis.Image and Video processing applications are opening up a whole range of opportunities for processing at the "edge" or IoT applications as the demand for high accuracy processing high resolution images increases. However this comes with an increase in the quantity of data to be processed and stored, thereby causing a significant increase in the computational challenges. There is a growing interest in developing hardware systems that provide energy efficient solutions to this challenge. The challenges in Image Processing are unique because the increase in resolution, not only increases the data to be processed but also the amount of information detail scavenged from the data is also greatly increased. This thesis addresses the concept of extracting the significant image information to enable processing the data intelligently within a heterogeneous system. We propose a unique way of defining image significance, based on what causes us to react when something "catches our eye", whether it be static or dynamic, whether it be in our central field of focus or our peripheral vision. This significance technique proves to be a relatively economical process in terms of energy and computational effort. We investigate opportunities for further computational and energy efficiency that are available by elective use of heterogeneous system elements. We utilise significance to adaptively select regions of interest for selective levels of processing dependent on their relative significance. We further demonstrate that exploiting the computational slack time released by this process, we can apply throttling of the processor speed to effect greater energy savings. This demonstrates a reduction in computational effort and energy efficiency a process that we term adaptive approximate computing. We demonstrate that our approach reduces energy in a range of 50 to 75%, dependent on user quality demand, for a real-time performance requirement of 10 fps for a WQXGA image, when compared with the existing approach that is agnostic of significance. We further hypothesise that by use of heterogeneous elements that savings up to 90% could be achievable in both performance and energy when compared with running OpenCV on the CPU alone

Image and Video Coding Techniques for Ultra-low Latency

The next generation of wireless networks fosters the adoption of latency-critical applications such as XR, connected industry, or autonomous driving. This survey gathers implementation aspects of different image and video coding schemes and discusses their tradeoffs. Standardized video coding technologies such as HEVC or VVC provide a high compression ratio, but their enormous complexity sets the scene for alternative approaches like still image, mezzanine, or texture compression in scenarios with tight resource or latency constraints. Regardless of the coding scheme, we found inter-device memory transfers and the lack of sub-frame coding as limitations of current full-system and software-programmable implementations.publishedVersionPeer reviewe

Using Radio Frequency and Motion Sensing to Improve Camera Sensor Systems

Camera-based sensor systems have advanced significantly in recent years. This advancement is a combination of camera CMOS (complementary metal-oxide-semiconductor) hardware technology improvement and new computer vision (CV) algorithms that can better process the rich information captured. As the world becoming more connected and digitized through increased deployment of various sensors, cameras have become a cost-effective solution with the advantages of small sensor size, intuitive sensing results, rich visual information, and neural network-friendly. The increased deployment and advantages of camera-based sensor systems have fueled applications such as surveillance, object detection, person re-identification, scene reconstruction, visual tracking, pose estimation, and localization. However, camera-based sensor systems have fundamental limitations such as extreme power consumption, privacy-intrusive, and inability to see-through obstacles and other non-ideal visual conditions such as darkness, smoke, and fog. In this dissertation, we aim to improve the capability and performance of camera-based sensor systems by utilizing additional sensing modalities such as commodity WiFi and mmWave (millimeter wave) radios, and ultra-low-power and low-cost sensors such as inertial measurement units (IMU). In particular, we set out to study three problems: (1) power and storage consumption of continuous-vision wearable cameras, (2) human presence detection, localization, and re-identification in both indoor and outdoor spaces, and (3) augmenting the sensing capability of camera-based systems in non-ideal situations. We propose to use an ultra-low-power, low-cost IMU sensor, along with readily available camera information, to solve the first problem. WiFi devices will be utilized in the second problem, where our goal is to reduce the hardware deployment cost and leverage existing WiFi infrastructure as much as possible. Finally, we will use a low-cost, off-the-shelf mmWave radar to extend the sensing capability of a camera in non-ideal visual sensing situations.Doctor of Philosoph

Study on Real-time Video Processing with Data Processing Structure Optimized for Characteristics of Application

長崎大学学位論文 学位記番号:博(工)甲第96号 学位授与年月日:令和3年3月22日Nagasaki University (長崎大学)課程博

Wireless indoor localisation within the 5G internet of radio light

This thesis was submitted for the award of Doctor of Philosophy and was awarded by Brunel University LondonNumerous applications can be enhanced by accurate and efficient indoor localisation using wireless sensor networks, however trade-offs often exist between these two parameters. In this thesis, realworld and simulation data is used to examine the hybrid millimeter wave and Visible Light Communications (VLC) architecture of the 5G Internet of Radio Light (IoRL) Horizon 2020 project. Consequently, relevant localisation challenges within Visible Light Positioning (VLP) and asynchronous sampling networks are identified, and more accurate and efficient solutions are developed. Currently, VLP relies strongly on the assumed Lambertian properties of light sources. However, in practice, not all lights are Lambertian. To support the widespread deployment of VLC technology in numerous environments, measurements from non-Lambertian sources are analysed to provide new insights into the limitations of existing VLP techniques. Subsequently, a novel VLP calibration technique is proposed, and results indicate a 59% accuracy improvement against existing methods. This solution enables high accuracy centimetre level VLP to be achieved with non- Lambertian sources. Asynchronous sampling of range-based measurements is known to impact localisation performance negatively. Various Asynchronous Sampling Localisation Techniques (ASLT) exist to mitigate these effects. While effective at improving positioning performance, the exact suitability of such solutions is not evident due to their additional processes, subsequent complexity, and increased costs. As such, extensive simulations are conducted to study the effectiveness of ASLT under variable sampling latencies, sensor measurement noise, and target trajectories. Findings highlight the computational demand of existing ASLT and motivate the development of a novel solution. The proposed Kalman Extrapolated Least Squares (KELS) method achieves optimal localisation performance with a significant energy reduction of over 50% when compared to current leading ASLT. The work in this thesis demonstrates both the capability for high performance VLP from non- Lambertian sources as well as the potential for energy efficient localisation for sequentially sampled range measurements.Horizon 202