Neural-based Compression Scheme for Solar Image Data

Studying the solar system and especially the Sun relies on the data gathered daily from space missions. These missions are data-intensive and compressing this data to make them efficiently transferable to the ground station is a twofold decision to make. Stronger compression methods, by distorting the data, can increase data throughput at the cost of accuracy which could affect scientific analysis of the data. On the other hand, preserving subtle details in the compressed data requires a high amount of data to be transferred, reducing the desired gains from compression. In this work, we propose a neural network-based lossy compression method to be used in NASA's data-intensive imagery missions. We chose NASA's SDO mission which transmits 1.4 terabytes of data each day as a proof of concept for the proposed algorithm. In this work, we propose an adversarially trained neural network, equipped with local and non-local attention modules to capture both the local and global structure of the image resulting in a better trade-off in rate-distortion (RD) compared to conventional hand-engineered codecs. The RD variational autoencoder used in this work is jointly trained with a channel-dependent entropy model as a shared prior between the analysis and synthesis transforms to make the entropy coding of the latent code more effective. Our neural image compression algorithm outperforms currently-in-use and state-of-the-art codecs such as JPEG and JPEG-2000 in terms of the RD performance when compressing extreme-ultraviolet (EUV) data. As a proof of concept for use of this algorithm in SDO data analysis, we have performed coronal hole (CH) detection using our compressed images, and generated consistent segmentations, even at a compression rate of $\sim0.1$ bits per pixel (compared to 8 bits per pixel on the original data) using EUV data from SDO.Comment: Accepted for publication in IEEE Transactions on Aerospace and Electronic Systems (TAES). arXiv admin note: text overlap with arXiv:2210.0647

Stochastic dynamics and wavelets techniques for system response analysis and diagnostics: Diverse applications in structural and biomedical engineering

In the first part of the dissertation, a novel stochastic averaging technique based on a Hilbert transform definition of the oscillator response displacement amplitude is developed. In comparison to standard stochastic averaging, the requirement of “a priori” determination of an equivalent natural frequency is bypassed, yielding flexibility in the ensuing analysis and potentially higher accuracy. Further, the herein proposed Hilbert transform based stochastic averaging is adapted for determining the time-dependent survival probability and first-passage time probability density function of stochastically excited nonlinear oscillators, even endowed with fractional derivative terms. To this aim, a Galerkin scheme is utilized to solve approximately the backward Kolmogorov partial differential equation governing the survival probability of the oscillator response. Next, the potential of the stochastic averaging technique to be used in conjunction with performance-based engineering design applications is demonstrated by proposing a stochastic version of the widely used incremental dynamic analysis (IDA). Specifically, modeling the excitation as a non-stationary stochastic process possessing an evolutionary power spectrum (EPS), an approximate closed-form expression is derived for the parameterized oscillator response amplitude probability density function (PDF). In this regard, IDA surfaces are determined providing the conditional PDF of the engineering demand parameter (EDP) for a given intensity measure (IM) value. In contrast to the computationally expensive Monte Carlo simulation, the methodology developed herein determines the IDA surfaces at minimal computational cost. In the second part of the dissertation, a novel multiple-input/single-output (MISO) system identification technique is developed for parameter identification of nonlinear and time-variant oscillators with fractional derivative terms subject to incomplete non-stationary data. The technique utilizes a representation of the nonlinear restoring forces as a set of parallel linear sub-systems. Next, a recently developed L1-norm minimization procedure based on compressive sensing theory is applied for determining the wavelet coefficients of the available incomplete non-stationary input-output (excitation-response) data. Several numerical examples are considered for assessing the reliability of the technique, even in the presence of incomplete and corrupted data. These include a 2-DOF time-variant Duffing oscillator endowed with fractional derivative terms, as well as a 2-DOF system subject to flow-induced forces where the non-stationary sea state possesses a recently proposed evolutionary version of the JONSWAP spectrum. In the third part of this dissertation, a joint time-frequency analysis technique based on generalized harmonic wavelets (GHWs) is developed for dynamic cerebral autoregulation (DCA) performance quantification. DCA is the continuous counter-regulation of the cerebral blood flow by the active response of cerebral blood vessels to the spontaneous or induced blood pressure fluctuations. Specifically, various metrics of the phase shift and magnitude of appropriately defined GHW-based transfer functions are determined based on data points over the joint time-frequency domain. The potential of these metrics to be used as a diagnostics tool for indicating healthy versus impaired DCA function is assessed by considering both healthy individuals and patients with unilateral carotid artery stenosis. Next, another application in biomedical engineering is pursued related to the Pulse Wave Imaging (PWI) technique. This relies on ultrasonic signals for capturing the propagation of pressure pulses along the carotid artery, and eventually for prognosis of focal vascular diseases (e.g., atherosclerosis and abdominal aortic aneurysm). However, to obtain a high spatio-temporal resolution the data are acquired at a high rate, in the order of kilohertz, yielding large datasets. To address this challenge, an efficient data compression technique is developed based on the multiresolution wavelet decomposition scheme, which exploits the high correlation of adjacent RF-frames generated by the PWI technique. Further, a sparse matrix decomposition is proposed as an efficient way to identify the boundaries of the arterial wall in the PWI technique

A Survey of Deep Face Restoration: Denoise, Super-Resolution, Deblur, Artifact Removal

Face Restoration (FR) aims to restore High-Quality (HQ) faces from Low-Quality (LQ) input images, which is a domain-specific image restoration problem in the low-level computer vision area. The early face restoration methods mainly use statistic priors and degradation models, which are difficult to meet the requirements of real-world applications in practice. In recent years, face restoration has witnessed great progress after stepping into the deep learning era. However, there are few works to study deep learning-based face restoration methods systematically. Thus, this paper comprehensively surveys recent advances in deep learning techniques for face restoration. Specifically, we first summarize different problem formulations and analyze the characteristic of the face image. Second, we discuss the challenges of face restoration. Concerning these challenges, we present a comprehensive review of existing FR methods, including prior based methods and deep learning-based methods. Then, we explore developed techniques in the task of FR covering network architectures, loss functions, and benchmark datasets. We also conduct a systematic benchmark evaluation on representative methods. Finally, we discuss future directions, including network designs, metrics, benchmark datasets, applications,etc. We also provide an open-source repository for all the discussed methods, which is available at https://github.com/TaoWangzj/Awesome-Face-Restoration.Comment: 21 pages, 19 figure

A network transparent, retained mode multimedia processing framework for the Linux operating system environment

Die Arbeit präsentiert ein Multimedia-Framework für Linux, das im Unterschied zu früheren Arbeiten auf den Ideen "retained-mode processing" und "lazy evaluation" basiert: Statt Transformationen unmittelbar auszuführen, wird eine abstrakte Repräsentation aller Medienelemente aufgebaut. "renderer"-Treiber fungieren als Übersetzer, die diese Darstellung zur Laufzeit in konkrete Operationen umsetzen, wobei das Datenmodell zahlreiche Optimierungen zur Reduktion der Anzahl der Schritte oder der Minimierung von Kommunikation erlaubt. Dies erlaubt ein stark vereinfachtes Programmiermodell bei gleichzeitiger Effizienzsteigerung. "renderer"-Treiber können zur Ausführung von Transformationen den lokalen Prozessor verwenden, oder können die Operationen delegieren. In der Arbeit wird eine Erweiterung des X Window Systems um Mechanismen zur Medienverarbeitung vorgestellt, sowie ein "renderer"-Treiber, der diese zur Delegation der Verarbeitung nutzt

An efficient telemetry system for restoring sight

PhD ThesisThe human nervous system can be damaged as a result of disease or trauma, causing conditions such as Parkinson’s disease. Most people try pharmaceuticals as a primary method of treatment. However, drugs cannot restore some cases, such as visual disorder. Alternatively, this impairment can be treated with electronic neural prostheses. A retinal prosthesis is an example of that for restoring sight, but it is not efficient and only people with retinal pigmentosa benefit from it. In such treatments, stimulation of the nervous system can be achieved by electrical or optical means. In the latter case, the nerves need to be rendered light sensitive via genetic means (optogenetics). High radiance photonic devices are then required to deliver light to the target tissue. Such optical approaches hold the potential to be more effective while causing less harm to the brain tissue. As these devices are implanted in tissue, wireless means need to be used to communicate with them. For this, IEEE 802.15.6 or Bluetooth protocols at 2.4GHz are potentially compatible with most advanced electronic devices, and are also safe and secure. Also, wireless power delivery can operate the implanted device. In this thesis, a fully wireless and efficient visual cortical stimulator was designed to restore the sight of the blind. This system is likely to address 40% of the causes of blindness. In general, the system can be divided into two parts, hardware and software. Hardware parts include a wireless power transfer design, the communication device, power management, a processor and the control unit, and the 3D design for assembly. The software part contains the image simplification, image compression, data encoding, pulse modulation, and the control system. Real-time video streaming is processed and sent over Bluetooth, and data are received by the LPC4330 six layer implanted board. After retrieving the compressed data, the processed data are again sent to the implanted electrode/optrode to stimulate the brain’s nerve cells

Neural Image Restoration for Images with Diverse Distortion Factors

Tohoku University岡谷貴之課

Remote Sensing Data Compression

A huge amount of data is acquired nowadays by different remote sensing systems installed on satellites, aircrafts, and UAV. The acquired data then have to be transferred to image processing centres, stored and/or delivered to customers. In restricted scenarios, data compression is strongly desired or necessary. A wide diversity of coding methods can be used, depending on the requirements and their priority. In addition, the types and properties of images differ a lot, thus, practical implementation aspects have to be taken into account. The Special Issue paper collection taken as basis of this book touches on all of the aforementioned items to some degree, giving the reader an opportunity to learn about recent developments and research directions in the field of image compression. In particular, lossless and near-lossless compression of multi- and hyperspectral images still remains current, since such images constitute data arrays that are of extremely large size with rich information that can be retrieved from them for various applications. Another important aspect is the impact of lossless compression on image classification and segmentation, where a reasonable compromise between the characteristics of compression and the final tasks of data processing has to be achieved. The problems of data transition from UAV-based acquisition platforms, as well as the use of FPGA and neural networks, have become very important. Finally, attempts to apply compressive sensing approaches in remote sensing image processing with positive outcomes are observed. We hope that readers will find our book useful and interestin