Adaptively Lossy Image Compression for Onboard Processing

More efficient image-compression codecs are an emerging requirement for spacecraft because increasingly complex, onboard image sensors can rapidly saturate downlink bandwidth of communication transceivers. While these codecs reduce transmitted data volume, many are compute-intensive and require rapid processing to sustain sensor data rates. Emerging next-generation small satellite (SmallSat) computers provide compelling computational capability to enable more onboard processing and compression than previously considered. For this research, we apply two compression algorithms for deployment on modern flight hardware: (1) end-to-end, neural-network-based, image compression (CNN-JPEG); and (2) adaptive image compression through feature-point detection (FPD-JPEG). These algorithms rely on intelligent data-processing pipelines that adapt to sensor data to compress it more effectively, ensuring efficient use of limited downlink bandwidths. The first algorithm, CNN-JPEG, employs a hybrid approach adapted from literature combining convolutional neural networks (CNNs) and JPEG; however, we modify and tune the training scheme for satellite imagery to account for observed training instabilities. This hybrid CNN-JPEG approach shows 23.5% better average peak signal-to-noise ratio (PSNR) and 33.5% better average structural similarity index (SSIM) versus standard JPEG on a dataset collected on the Space Test Program – Houston 5 (STP-H5-CSP) mission onboard the International Space Station (ISS). For our second algorithm, we developed a novel adaptive image-compression pipeline based upon JPEG that leverages the Oriented FAST and Rotated BRIEF (ORB) feature-point detection algorithm to adaptively tune the compression ratio to allow for a tradeoff between PSNR/SSIM and combined file size over a batch of STP-H5-CSP images. We achieve a less than 1% drop in average PSNR and SSIM while reducing the combined file size by 29.6% compared to JPEG using a static quality factor (QF) of 90

Evaluation and implementation of an auto-encoder for compression of satellite images in the ScOSA project

The thesis evaluates the efficiency of various autoencoder neural networks for image compression regarding satellite imagery. The results highlight the evaluation and implementation of autoencoder architectures and the procedures required to deploy neural networks to reliable embedded devices. The developed autoencoders evaluated, targeting a ZYNQ 7020 FPGA (Field Programmable Gate Array) and a ZU7EV FPGA

Reconfigurable Architectures and Systems for IoT Applications

abstract: Internet of Things (IoT) has become a popular topic in industry over the recent years, which describes an ecosystem of internet-connected devices or things that enrich the everyday life by improving our productivity and efficiency. The primary components of the IoT ecosystem are hardware, software and services. While the software and services of IoT system focus on data collection and processing to make decisions, the underlying hardware is responsible for sensing the information, preprocess and transmit it to the servers. Since the IoT ecosystem is still in infancy, there is a great need for rapid prototyping platforms that would help accelerate the hardware design process. However, depending on the target IoT application, different sensors are required to sense the signals such as heart-rate, temperature, pressure, acceleration, etc., and there is a great need for reconfigurable platforms that can prototype different sensor interfacing circuits. This thesis primarily focuses on two important hardware aspects of an IoT system: (a) an FPAA based reconfigurable sensing front-end system and (b) an FPGA based reconfigurable processing system. To enable reconfiguration capability for any sensor type, Programmable ANalog Device Array (PANDA), a transistor-level analog reconfigurable platform is proposed. CAD tools required for implementation of front-end circuits on the platform are also developed. To demonstrate the capability of the platform on silicon, a small-scale array of 24×25 PANDA cells is fabricated in 65nm technology. Several analog circuit building blocks including amplifiers, bias circuits and filters are prototyped on the platform, which demonstrates the effectiveness of the platform for rapid prototyping IoT sensor interfaces. IoT systems typically use machine learning algorithms that run on the servers to process the data in order to make decisions. Recently, embedded processors are being used to preprocess the data at the energy-constrained sensor node or at IoT gateway, which saves considerable energy for transmission and bandwidth. Using conventional CPU based systems for implementing the machine learning algorithms is not energy-efficient. Hence an FPGA based hardware accelerator is proposed and an optimization methodology is developed to maximize throughput of any convolutional neural network (CNN) based machine learning algorithm on a resource-constrained FPGA.Dissertation/ThesisDoctoral Dissertation Electrical Engineering 201

Evaluation and implementation of an auto-encoder for compression of satellite images in the ScOSA project

The thesis evaluates the efficiency of various autoencoder neural networks for image compression regarding satellite imagery. The results highlight the evaluation and implementation of autoencoder architectures and the procedures required to deploy neural networks to reliable embedded devices. The developed autoencoders evaluated, targeting a ZYNQ 7020 FPGA (Field Programmable Gate Array) and a ZU7EV FPGA

Fast fluorescence lifetime imaging and sensing via deep learning

Error on title page – year of award is 2023.Fluorescence lifetime imaging microscopy (FLIM) has become a valuable tool in diverse disciplines. This thesis presents deep learning (DL) approaches to addressing two major challenges in FLIM: slow and complex data analysis and the high photon budget for precisely quantifying the fluorescence lifetimes. DL's ability to extract high-dimensional features from data has revolutionized optical and biomedical imaging analysis. This thesis contributes several novel DL FLIM algorithms that significantly expand FLIM's scope. Firstly, a hardware-friendly pixel-wise DL algorithm is proposed for fast FLIM data analysis. The algorithm has a simple architecture yet can effectively resolve multi-exponential decay models. The calculation speed and accuracy outperform conventional methods significantly. Secondly, a DL algorithm is proposed to improve FLIM image spatial resolution, obtaining high-resolution (HR) fluorescence lifetime images from low-resolution (LR) images. A computational framework is developed to generate large-scale semi-synthetic FLIM datasets to address the challenge of the lack of sufficient high-quality FLIM datasets. This algorithm offers a practical approach to obtaining HR FLIM images quickly for FLIM systems. Thirdly, a DL algorithm is developed to analyze FLIM images with only a few photons per pixel, named Few-Photon Fluorescence Lifetime Imaging (FPFLI) algorithm. FPFLI uses spatial correlation and intensity information to robustly estimate the fluorescence lifetime images, pushing this photon budget to a record-low level of only a few photons per pixel. Finally, a time-resolved flow cytometry (TRFC) system is developed by integrating an advanced CMOS single-photon avalanche diode (SPAD) array and a DL processor. The SPAD array, using a parallel light detection scheme, shows an excellent photon-counting throughput. A quantized convolutional neural network (QCNN) algorithm is designed and implemented on a field-programmable gate array as an embedded processor. The processor resolves fluorescence lifetimes against disturbing noise, showing unparalleled high accuracy, fast analysis speed, and low power consumption.Fluorescence lifetime imaging microscopy (FLIM) has become a valuable tool in diverse disciplines. This thesis presents deep learning (DL) approaches to addressing two major challenges in FLIM: slow and complex data analysis and the high photon budget for precisely quantifying the fluorescence lifetimes. DL's ability to extract high-dimensional features from data has revolutionized optical and biomedical imaging analysis. This thesis contributes several novel DL FLIM algorithms that significantly expand FLIM's scope. Firstly, a hardware-friendly pixel-wise DL algorithm is proposed for fast FLIM data analysis. The algorithm has a simple architecture yet can effectively resolve multi-exponential decay models. The calculation speed and accuracy outperform conventional methods significantly. Secondly, a DL algorithm is proposed to improve FLIM image spatial resolution, obtaining high-resolution (HR) fluorescence lifetime images from low-resolution (LR) images. A computational framework is developed to generate large-scale semi-synthetic FLIM datasets to address the challenge of the lack of sufficient high-quality FLIM datasets. This algorithm offers a practical approach to obtaining HR FLIM images quickly for FLIM systems. Thirdly, a DL algorithm is developed to analyze FLIM images with only a few photons per pixel, named Few-Photon Fluorescence Lifetime Imaging (FPFLI) algorithm. FPFLI uses spatial correlation and intensity information to robustly estimate the fluorescence lifetime images, pushing this photon budget to a record-low level of only a few photons per pixel. Finally, a time-resolved flow cytometry (TRFC) system is developed by integrating an advanced CMOS single-photon avalanche diode (SPAD) array and a DL processor. The SPAD array, using a parallel light detection scheme, shows an excellent photon-counting throughput. A quantized convolutional neural network (QCNN) algorithm is designed and implemented on a field-programmable gate array as an embedded processor. The processor resolves fluorescence lifetimes against disturbing noise, showing unparalleled high accuracy, fast analysis speed, and low power consumption

Efficient Hardware Architectures for Accelerating Deep Neural Networks: Survey

In the modern-day era of technology, a paradigm shift has been witnessed in the areas involving applications of Artificial Intelligence (AI), Machine Learning (ML), and Deep Learning (DL). Specifically, Deep Neural Networks (DNNs) have emerged as a popular field of interest in most AI applications such as computer vision, image and video processing, robotics, etc. In the context of developed digital technologies and the availability of authentic data and data handling infrastructure, DNNs have been a credible choice for solving more complex real-life problems. The performance and accuracy of a DNN is a way better than human intelligence in certain situations. However, it is noteworthy that the DNN is computationally too cumbersome in terms of the resources and time to handle these computations. Furthermore, general-purpose architectures like CPUs have issues in handling such computationally intensive algorithms. Therefore, a lot of interest and efforts have been invested by the research fraternity in specialized hardware architectures such as Graphics Processing Unit (GPU), Field Programmable Gate Array (FPGA), Application Specific Integrated Circuit (ASIC), and Coarse Grained Reconfigurable Array (CGRA) in the context of effective implementation of computationally intensive algorithms. This paper brings forward the various research works carried out on the development and deployment of DNNs using the aforementioned specialized hardware architectures and embedded AI accelerators. The review discusses the detailed description of the specialized hardware-based accelerators used in the training and/or inference of DNN. A comparative study based on factors like power, area, and throughput, is also made on the various accelerators discussed. Finally, future research and development directions are discussed, such as future trends in DNN implementation on specialized hardware accelerators. This review article is intended to serve as a guide for hardware architectures for accelerating and improving the effectiveness of deep learning research.publishedVersio

6G White Paper on Machine Learning in Wireless Communication Networks

The focus of this white paper is on machine learning (ML) in wireless communications. 6G wireless communication networks will be the backbone of the digital transformation of societies by providing ubiquitous, reliable, and near-instant wireless connectivity for humans and machines. Recent advances in ML research has led enable a wide range of novel technologies such as self-driving vehicles and voice assistants. Such innovation is possible as a result of the availability of advanced ML models, large datasets, and high computational power. On the other hand, the ever-increasing demand for connectivity will require a lot of innovation in 6G wireless networks, and ML tools will play a major role in solving problems in the wireless domain. In this paper, we provide an overview of the vision of how ML will impact the wireless communication systems. We first give an overview of the ML methods that have the highest potential to be used in wireless networks. Then, we discuss the problems that can be solved by using ML in various layers of the network such as the physical layer, medium access layer, and application layer. Zero-touch optimization of wireless networks using ML is another interesting aspect that is discussed in this paper. Finally, at the end of each section, important research questions that the section aims to answer are presented

Application of wavelets and artificial neural network for indoor optical wireless communication systems

Abstract This study investigates the use of error control code, discrete wavelet transform (DWT) and artificial neural network (ANN) to improve the link performance of an indoor optical wireless communication in a physical channel. The key constraints that barricade the realization of unlimited bandwidth in optical wavelengths are the eye-safety issue, the ambient light interference and the multipath induced intersymbol interference (ISI). Eye-safety limits the maximum average transmitted optical power. The rational solution is to use power efficient modulation techniques. Further reduction in transmitted power can be achieved using error control coding. A mathematical analysis of retransmission scheme is investigated for variable length modulation techniques and verified using computer simulations. Though the retransmission scheme is simple to implement, the shortfall in terms of reduced throughput will limit higher code gain. Due to practical limitation, the block code cannot be applied to the variable length modulation techniques and hence the convolutional code is the only possible option. The upper bound for slot error probability of the convolutional coded dual header pulse interval modulation (DH-PIM) and digital pulse interval modulation (DPIM) schemes are calculated and verified using simulations. The power penalty due to fluorescent light interference (FL I) is very high in indoor optical channel making the optical link practically infeasible. A denoising method based on a DWT to remove the FLI from the received signal is devised. The received signal is first decomposed into different DWT levels; the FLI is then removed from the signal before reconstructing the signal. A significant reduction in the power penalty is observed using DWT. Comparative study of DWT based denoising scheme with that of the high pass filter (HPF) show that DWT not only can match the best performance obtain using a HPF, but also offers a reduced complexity and design simplicity. The high power penalty due to multipath induced ISI makes a diffuse optical link practically infeasible at higher data rates. An ANN based linear and DF architectures are investigated to compensation the ISI. Unlike the unequalized cases, the equalized schemes don‘t show infinite power penalty and a significant performance improvement is observed for all modulation schemes. The comparative studies substantiate that ANN based equalizers match the performance of the traditional equalizers for all channel conditions with a reduced training data sequence. The study of the combined effect of the FLI and ISI shows that DWT-ANN based receiver perform equally well in the present of both interference. Adaptive decoding of error control code can offer flexibility of selecting the best possible encoder in a given environment. A suboptimal ?soft‘ sliding block convolutional decoder based on the ANN and a 1/2 rate convolutional code with a constraint length is investigated. Results show that the ANN decoder can match the performance of optimal Viterbi decoder for hard decision decoding but with slightly inferior performance compared to soft decision decoding. This provides a foundation for further investigation of the ANN decoder for convolutional code with higher constraint length values. Finally, the proposed DWT-ANN receiver is practically realized in digital signal processing (DSP) board. The output from the DSP board is compared with the computer simulations and found that the difference is marginal. However, the difference in results doesn‘t affect the overall error probability and identical error probability is obtained for DSP output and computer simulations

Application of wavelets and artificial neural network for indoor optical wireless communication systems

This study investigates the use of error control code, discrete wavelet transform (DWT) and artificial neural network (ANN) to improve the link performance of an indoor optical wireless communication in a physical channel. The key constraints that barricade the realization of unlimited bandwidth in optical wavelengths are the eye-safety issue, the ambient light interference and the multipath induced intersymbol interference (ISI). Eye-safety limits the maximum average transmitted optical power. The rational solution is to use power efficient modulation techniques. Further reduction in transmitted power can be achieved using error control coding. A mathematical analysis of retransmission scheme is investigated for variable length modulation techniques and verified using computer simulations. Though the retransmission scheme is simple to implement, the shortfall in terms of reduced throughput will limit higher code gain. Due to practical limitation, the block code cannot be applied to the variable length modulation techniques and hence the convolutional code is the only possible option. The upper bound for slot error probability of the convolutional coded dual header pulse interval modulation (DH-PIM) and digital pulse interval modulation (DPIM) schemes are calculated and verified using simulations. The power penalty due to fluorescent light interference (FL I) is very high in indoor optical channel making the optical link practically infeasible. A denoising method based on a DWT to remove the FLI from the received signal is devised. The received signal is first decomposed into different DWT levels; the FLI is then removed from the signal before reconstructing the signal. A significant reduction in the power penalty is observed using DWT. Comparative study of DWT based denoising scheme with that of the high pass filter (HPF) show that DWT not only can match the best performance obtain using a HPF, but also offers a reduced complexity and design simplicity. The high power penalty due to multipath induced ISI makes a diffuse optical link practically infeasible at higher data rates. An ANN based linear and DF architectures are investigated to compensation the ISI. Unlike the unequalized cases, the equalized schemes don‘t show infinite power penalty and a significant performance improvement is observed for all modulation schemes. The comparative studies substantiate that ANN based equalizers match the performance of the traditional equalizers for all channel conditions with a reduced training data sequence. The study of the combined effect of the FLI and ISI shows that DWT-ANN based receiver perform equally well in the present of both interference. Adaptive decoding of error control code can offer flexibility of selecting the best possible encoder in a given environment. A suboptimal 'soft' sliding block convolutional decoder based on the ANN and a 1/2 rate convolutional code with a constraint length is investigated. Results show that the ANN decoder can match the performance of optimal Viterbi decoder for hard decision decoding but with slightly inferior performance compared to soft decision decoding. This provides a foundation for further investigation of the ANN decoder for convolutional code with higher constraint length values. Finally, the proposed DWT-ANN receiver is practically realized in digital signal processing (DSP) board. The output from the DSP board is compared with the computer simulations and found that the difference is marginal. However, the difference in results doesn‘t affect the overall error probability and identical error probability is obtained for DSP output and computer simulations.EThOS - Electronic Theses Online ServiceGBUnited Kingdo