AM-FM methods for image and video processing

This dissertation is focused on the development of robust and efficient Amplitude-Modulation Frequency-Modulation (AM-FM) demodulation methods for image and video processing (there is currently a patent pending that covers the AM-FM methods and applications described in this dissertation). The motivation for this research lies in the wide number of image and video processing applications that can significantly benefit from this research. A number of potential applications are developed in the dissertation. First, a new, robust and efficient formulation for the instantaneous frequency (IF) estimation: a variable spacing, local quadratic phase method (VS-LQP) is presented. VS-LQP produces much more accurate results than current AM-FM methods. At significant noise levels (SNR \u3c 30dB), for single component images, the VS-LQP method produces better IF estimation results than methods using a multi-scale filterbank. At low noise levels (SNR \u3e 50dB), VS-LQP performs better when used in combination with a multi-scale filterbank. In all cases, VS-LQP outperforms the Quasi-Eigen Approximation algorithm by significant amounts (up to 20dB). New least squares reconstructions using AM-FM components from the input signal (image or video) are also presented. Three different reconstruction approaches are developed: (i) using AM-FM harmonics, (ii) using AM-FM components extracted from different scales and (iii) using AM-FM harmonics with the output of a low-pass filter. The image reconstruction methods provide perceptually lossless results with image quality index values bigger than 0.7 on average. The video reconstructions produced image quality index values, frame by frame, up to more than 0.7 using AM-FM components extracted from different scales. An application of the AM-FM method to retinal image analysis is also shown. This approach uses the instantaneous frequency magnitude and the instantaneous amplitude (IA) information to provide image features. The new AM-FM approach produced ROC area of 0.984 in classifying Risk 0 versus Risk 1, 0.95 in classifying Risk 0 versus Risk 2, 0.973 in classifying Risk 0 versus Risk 3 and 0.95 in classifying Risk 0 versus all images with any sign of Diabetic Retinopathy. An extension of the 2D AM-FM demodulation methods to three dimensions is also presented. New AM-FM methods for motion estimation are developed. The new motion estimation method provides three motion estimation equations per channel filter (AM, IF motion equations and a continuity equation). Applications of the method in motion tracking, trajectory estimation and for continuous-scale video searching are demonstrated. For each application, we discuss the advantages of the AM-FM methods over current approaches

High-Fidelity and Perfect Reconstruction Techniques for Synthesizing Modulation Domain Filtered Images

Biomimetic processing inspired by biological vision systems has long been a goal of the image processing research community, both to further understanding of what it means to perceive and interpret image content and to facilitate advancements in applications ranging from processing large volumes of image data to engineering artificial intelligence systems. In recent years, the AM-FM transform has emerged as a useful tool that enables processing that is intuitive to human observers but would be difficult or impossible to achieve using traditional linear processing methods. The transform makes use of the multicomponent AM-FM image model, which represents imagery in terms of amplitude modulations, representative of local image contrast, and frequency modulations, representative of local spacing and orientation of lines and patterns. The model defines image components using an array of narrowband filterbank channels that is designed to be similar to the spatial frequency channel decomposition that occurs in the human visual system. The AM-FM transform entails the computation of modulation functions for all components of an image and the subsequent exact recovery of the image from those modulation functions. The process of modifying the modulation functions to alter visual information in a predictable way and then recovering the modified image through the AM-FM transform is known as modulation domain filtering. Past work in modulation domain filtering has produced dramatic results, but has faced challenges due to phase wrapping inherent in the transform computations and due to unknown integration constants associated with modified frequency content. The approaches developed to overcome these challenges have led to a loss of both stability and intuitive simplicity within the AM-FM model. In this dissertation, I have made significant advancements in the underlying processes that comprise the AM-FM transform. I have developed a new phase unwrapping method that increases the stability of the AM-FM transform, allowing higher quality modulation domain filtering results. I have designed new reconstruction techniques that allow for successful recovery from modified frequency modulations. These developments have allowed the design of modulation domain filters that, for the first time, do not require any departure from the simple and intuitive nature of the basic AM-FM model. Using the new modulation domain filters, I have produced new and striking results that achieve a variety of image processing tasks which are motivated by biological visual perception. These results represent a significant advancement relative to the state of the art and are a foundation from which future advancements in the field may be attained

Multirate Frequency Transformations: Wideband AM-FM Demodulation with Applications to Signal Processing and Communications

The AM-FM (amplitude & frequency modulation) signal model finds numerous applications in image processing, communications, and speech processing. The traditional approaches towards demodulation of signals in this category are the analytic signal approach, frequency tracking, or the energy operator approach. These approaches however, assume that the amplitude and frequency components are slowly time-varying, e.g., narrowband and incur significant demodulation error in the wideband scenarios. In this thesis, we extend a two-stage approach towards wideband AM-FM demodulation that combines multirate frequency transformations (MFT) enacted through a combination of multirate systems with traditional demodulation techniques, e.g., the Teager-Kasiser energy operator demodulation (ESA) approach to large wideband to narrowband conversion factors. The MFT module comprises of multirate interpolation and heterodyning and converts the wideband AM-FM signal into a narrowband signal, while the demodulation module such as ESA demodulates the narrowband signal into constituent amplitude and frequency components that are then transformed back to yield estimates for the wideband signal. This MFT-ESA approach is then applied to the various problems of: (a) wideband image demodulation and fingerprint demodulation, where multidimensional energy separation is employed, (b) wideband first-formant demodulation in vowels, and (c) wideband CPM demodulation with partial response signaling, to demonstrate its validity in both monocomponent and multicomponent scenarios as an effective multicomponent AM-FM signal demodulation and analysis technique for image processing, speech processing, and communications based applications

Modulation Domain Image Processing

The classical Fourier transform is the cornerstone of traditional linearsignal and image processing. The discrete Fourier transform (DFT) and thefast Fourier transform (FFT) in particular led toprofound changes during the later decades of the last century in howwe analyze and process 1D and multi-dimensional signals.The Fourier transform represents a signal as an infinite superpositionof stationary sinusoids each of which has constant amplitude and constantfrequency. However, many important practical signals such as radar returnsand seismic waves are inherently nonstationary. Hence, more complextechniques such as the windowed Fourier transform and the wavelet transformwere invented to better capture nonstationary properties of these signals.In this dissertation, I studied an alternative nonstationary representationfor images, the 2D AM-FM model. In contrast to thestationary nature of the classical Fourier representation, the AM-FM modelrepresents an image as a finite sum of smoothly varying amplitudesand smoothly varying frequencies. The model has been applied successfullyin image processing applications such as image segmentation, texture analysis,and target tracking. However, these applications are limitedto \emph{analysis}, meaning that the computed AM and FM functionsare used as features for signal processing tasks such as classificationand recognition. For synthesis applications, few attempts have been madeto synthesize the original image from the AM and FM components. Nevertheless,these attempts were unstable and the synthesized results contained artifacts.The main reason is that the perfect reconstruction AM-FM image model waseither unavailable or unstable. Here, I constructed the first functionalperfect reconstruction AM-FM image transform that paves the way for AM-FMimage synthesis applications. The transform enables intuitive nonlinearimage filter designs in the modulation domain. I showed that these filtersprovide important advantages relative to traditional linear translation invariant filters.This dissertation addresses image processing operations in the nonlinearnonstationary modulation domain. In the modulation domain, an image is modeledas a sum of nonstationary amplitude modulation (AM) functions andnonstationary frequency modulation (FM) functions. I developeda theoretical framework for high fidelity signal and image modeling in themodulation domain, constructed an invertible multi-dimensional AM-FMtransform (xAMFM), and investigated practical signal processing applicationsof the transform. After developing the xAMFM, I investigated new imageprocessing operations that apply directly to the transformed AM and FMfunctions in the modulation domain. In addition, I introduced twoclasses of modulation domain image filters. These filters produceperceptually motivated signal processing results that are difficult orimpossible to obtain with traditional linear processing or spatial domainnonlinear approaches. Finally, I proposed three extensions of the AM-FMtransform and applied them in image analysis applications.The main original contributions of this dissertation include the following.- I proposed a perfect reconstruction FM algorithm. I used aleast-squares approach to recover the phase signal from itsgradient. In order to allow perfect reconstruction of the phase function, Ienforced an initial condition on the reconstructed phase. The perfectreconstruction FM algorithm plays a critical role in theoverall AM-FM transform.- I constructed a perfect reconstruction multi-dimensional filterbankby modifying the classical steerable pyramid. This modified filterbankensures a true multi-scale multi-orientation signal decomposition. Such adecomposition is required for a perceptually meaningful AM-FM imagerepresentation.- I rotated the partial Hilbert transform to alleviate ripplingartifacts in the computed AM and FM functions. This adjustment results inartifact free filtering results in the modulation domain.- I proposed the modulation domain image filtering framework. Iconstructed two classes of modulation domain filters. I showed that themodulation domain filters outperform traditional linear shiftinvariant (LSI) filters qualitatively and quantitatively in applicationssuch as selective orientation filtering, selective frequency filtering,and fundamental geometric image transformations.- I provided extensions of the AM-FM transform for image decompositionproblems. I illustrated that the AM-FM approach can successfullydecompose an image into coherent components such as textureand structural components.- I investigated the relationship between the two prominentAM-FM computational models, namely the partial Hilbert transformapproach (pHT) and the monogenic signal. The established relationshiphelps unify these two AM-FM algorithms.This dissertation lays a theoretical foundation for future nonlinearmodulation domain image processing applications. For the first time, onecan apply modulation domain filters to images to obtain predictableresults. The design of modulation domain filters is intuitive and simple,yet these filters produce superior results compared to those of pixeldomain LSI filters. Moreover, this dissertation opens up other research problems.For instance, classical image applications such as image segmentation andedge detection can be re-formulated in the modulation domain setting.Modulation domain based perceptual image and video quality assessment andimage compression are important future application areas for the fundamentalrepresentation results developed in this dissertation

Detection of Spatial and Temporal Interactions in Renal Autoregulation Dynamics

Renal autoregulation stabilizes renal blood flow to protect the glomerular capillaries and maintain glomerular filtration rates through two mechanisms: tubuloglomerular feedback (TGF) and the myogenic response (MR). It is considered that the feedback mechanisms operate independently in each nephron (the functional unit of the kidney) within a kidney, but renal autoregulation dynamics can be coupled between vascular connected nephrons. It has also been shown that the mechanisms are time-varying and interact with each other. Understanding of the significance of such complex behavior has been limited by absence of techniques capable of monitoring renal flow signals among more than 2 or 3 nephrons simultaneously. The purpose of this thesis was to develop approaches to allow the identification and characterization of spatial and temporal properties of renal autoregulation dynamics. We present evidence that laser speckle perfusion imaging (LSPI) effectively captures renal autoregulation dynamics in perfusion signals across the renal cortex of anaesthetized rats and that spatial heterogeneity of the dynamics is present and can be investigated using LSPI. Next, we present a novel approach to segment LSPI of the renal surface into phase synchronized clusters representing areas with coupled renal autoregulation dynamics. Results are shown for the MR and demonstrate that when a signal is present phase synchronized regions can be identified. We then describe an approach to identify quadratic phase coupling between the TGF and MR mechanisms in time and space. Using this approach we can identify locations across the renal surface where both mechanisms are operating cooperatively. Finally, we show how synchronization between nephrons can be investigated in relation to renal autoregulation effectiveness by comparing phase synchronization estimates from LSPI with renal autoregulation system properties estimated from renal blood flow and blood pressure measurements. Overall, we have developed approaches to 1) capture renal autoregulation dynamics across the renal surface, 2) identify regions with phase synchronized renal autoregulation dynamics, 3) quantify the presence of the TGF-MR interaction across the renal surface, and 4) determine how the above vary over time. The described tools allow for investigations of the significance and mechanisms behind the complex spatial interactions and time-varying properties of renal autoregulation dynamics

Distributed Fiber Ultrasonic Sensor and Pattern Recognition Analytics

Ultrasound interrogation and structural health monitoring technologies have found a wide array of applications in the health care, aerospace, automobile, and energy sectors. To achieve high spatial resolution, large array electrical transducers have been used in these applications to harness sufficient data for both monitoring and diagnoses. Electronic-based sensors have been the standard technology for ultrasonic detection, which are often expensive and cumbersome for use in large scale deployments. Fiber optical sensors have advantageous characteristics of smaller cross-sectional area, humidity-resistance, immunity to electromagnetic interference, as well as compatibility with telemetry and telecommunications applications, which make them attractive alternatives for use as ultrasonic sensors. A unique trait of fiber sensors is its ability to perform distributed acoustic measurements to achieve high spatial resolution detection using a single fiber. Using ultrafast laser direct-writing techniques, nano-reflectors can be induced inside fiber cores to drastically improve the signal-to-noise ratio of distributed fiber sensors. This dissertation explores the applications of laser-fabricated nano-reflectors in optical fiber cores for both multi-point intrinsic Fabry–Perot (FP) interferometer sensors and a distributed phase-sensitive optical time-domain reflectometry (φ-OTDR) to be used in ultrasound detection. Multi-point intrinsic FP interferometer was based on swept-frequency interferometry with optoelectronic phase-locked loop that interrogated cascaded FP cavities to obtain ultrasound patterns. The ultrasound was demodulated through reassigned short time Fourier transform incorporating with maximum-energy ridges tracking. With tens of centimeters cavity length, this approach achieved 20kHz ultrasound detection that was finesse-insensitive, noise-free, high-sensitivity and multiplex-scalability. The use of φ-OTDR with enhanced Rayleigh backscattering compensated the deficiencies of low inherent signal-to-noise ratio (SNR). The dynamic strain between two adjacent nano-reflectors was extracted by using 3×3 coupler demodulation within Michelson interferometer. With an improvement of over 35 dB SNR, this was adequate for the recognition of the subtle differences in signals, such as footstep of human locomotion and abnormal acoustic echoes from pipeline corrosion. With the help of artificial intelligence in pattern recognition, high accuracy of events’ identification can be achieved in perimeter security and structural health monitoring, with further potential that can be harnessed using unsurprised learning

Analysis of motion in scale space

This work includes some new aspects of motion estimation by the optic flow method in scale spaces. The usual techniques for motion estimation are limited to the application of coarse to fine strategies. The coarse to fine strategies can be successful only if there is enough information in every scale. In this work we investigate the motion estimation in the scale space more basically. The wavelet choice for scale space decomposition of image sequences is discussed in the first part of this work. We make use of the continuous wavelet transform with rotationally symmetric wavelets. Bandpass decomposed sequences allow the replacement of the structure tensor by the phase invariant energy operator. The structure tensor is computationally more expensive because of its spatial or spatio-temporal averaging. The energy operator needs in general no further averaging. The numerical accuracy of the motion estimation with the energy operator is compared to the results of usual techniques, based on the structure tensor. The comparison tests are performed on synthetic and real life sequences. Another practical contribution is the accuracy measurement for motion estimation by adaptive smoothed tensor fields. The adaptive smoothing relies on nonlinear anisotropic diffusion with discontinuity and curvature preservation. We reached an accuracy gain under properly chosen parameters for the diffusion filter. A theoretical contribution from mathematical point of view is a new discontinuity and curvature preserving regularization for motion estimation. The convergence of solutions for the isotropic case of the nonlocal partial differential equation is shown. For large displacements between two consecutive frames the optic flow method is systematically corrupted because of the violence of the sampling theorem. We developed a new method for motion analysis by scale decomposition, which allows to circumvent the systematic corruption without using the coarse to fine strategy. The underlying assumption is, that in a certain neighborhood the grey value undergoes the same displacement. If this is fulfilled, then the same optic flow should be measured in all scales. If there arise inconsistencies in a pixel across the scale space, so they can be detected and the scales containing this inconsistencies are not taken into account

Design and numerical simulation of the real-time particle charge and size analyser

The electrostatic charge and size distribution of aerosol particles play a very important role in many industrial applications. Due to the complexity and the probabilistic nature of the different charging mechanisms often acting simultaneously, it is difficult to theoretically predict the charge distribution of aerosol particles or even estimate the relative effect of the different mechanisms. Therefore, it is necessary to measure the size and also the bipolar charge distribution on aerosol particles. The main aim of this research project was to design, implement and simulate a signal processing system for novel, fully functional measurement instrument capable of simultaneously measuring in real time the bipolar charge and size distribution of medical aerosols. The Particle Size and Charge Analyser (PSCA), investigated in this thesis, uses Phase Doppler Anemometry (PDA) technique. The PDA system was used to track the motion of charged particles in the presence of an electric field. By solving the equation of particle motion in a viscous medium combined with the simultaneous measurement of its size and velocity, the magnitude as well as the polarity of the particle charge can be obtained. Different signal processing systems in different excitation fields have been designed and implemented. These systems include: velocity estimation system using spectral analysis in DC excitation field, velocity estimation system based on Phase Locked Loop (PLL) technique working in DC as well as sine-wave excitation fields, velocity estimation system based on Quadrature Demodulation (QD) technique under sine-wave excitation method, velocity estimation system using spectral analysis in square-wave excitation field and phase shift estimation based on Hilbert transformation and correlation technique in both sine-wave and square-wave excitation fields. The performances of these systems were evaluated using Monte Carlo (MC) simulations obtained from the synthesized Doppler burst signals generated from the mathematical models implemented in MATLAB. The synthesized Doppler Burst Signal (DBS) was subsequently corrupted with the added Gaussian noise. Cross validation of the results was performed using hardware signal processing system employing Arbitrary Waveform Generator and also NASA simulator to further confirm the validity of the estimation